18:00:14 <phsmoura> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting
18:00:14 <zodbot> Meeting started Tue Apr 11 18:00:14 2023 UTC.
18:00:14 <zodbot> This meeting is logged and archived in a public location.
18:00:14 <zodbot> The chair is phsmoura. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
18:00:14 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:14 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:00:14 <phsmoura> #chair nirik smooge phsmoura
18:00:14 <phsmoura> #meetingname fedora_infrastructure_ops_daily_standup_meeting
18:00:14 <zodbot> Current chairs: nirik phsmoura smooge
18:00:14 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:00:14 <phsmoura> #info meeting is 30 minutes MAX. At the end of 30, its stops
18:00:14 <phsmoura> #info agenda is at https://board.net/p/fedora-infra-daily
18:00:21 <smooge> hello
18:00:22 <phsmoura> hello
18:00:53 <nirik> morning
18:01:25 <phsmoura> #info reminder: speak up if you want to work on a ticket!
18:01:26 <phsmoura> #topic Tickets needing review
18:01:40 <phsmoura> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1
18:01:53 <phsmoura> .ticket 11233
18:01:54 <zodbot> phsmoura: Issue #11233: File status mismatch in EPEL mirror ib01 but looks good in iad01 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/11233
18:02:17 <nirik> med med ops...
18:02:27 <nirik> Hopefully it's going to catch up soon
18:03:07 <nirik> it had several issues. Also, it's on an old machine that we were hoping to replace before now. ;(
18:03:53 <phsmoura> done
18:03:55 <phsmoura> #info https://pagure.io/releng/issues?status=Open
18:04:04 <phsmoura> .releng 11382
18:04:07 <zodbot> phsmoura: Issue #11382: Retire Node.js modules - releng - Pagure.io - https://pagure.io/releng/issue/11382
18:04:20 <nirik> low low ops?
18:05:36 <phsmoura> +1
18:05:45 <phsmoura> thats it, open floor now
18:05:47 <phsmoura> #topic Planning, Upcoming work and Open floor
18:06:29 <smooge> what modules are left?
18:06:41 <nirik> today has been a second monday. ;)
18:06:53 <smooge> I am on the 11th monday in a row
18:07:03 <smooge> i expect it has been longer...
18:07:20 <smooge> anything that can help nirik ?
18:07:43 <JamesAntill[m]1> So I have a quick question ... would it be ok if I ran something like: gunzip /mnt/fedora_stats/combined-http/2023///mirrors.fedoraproject.org-access.log.gz
18:07:48 <nirik> smooge: there's a bunch left.
18:08:08 <nirik> James Antill: not sure, smooge?
18:08:10 <phsmoura> Im still catching up with emails, but yesterday I was working on fedora wiki the issue persists with that user Im trying to reset his images and see if it works, was having trouble with permissions in the end of the day. Also the disk mounted in stg is the same as in prod just fyi
18:08:18 <JamesAntill[m]1> Should be * in those dirs.
18:08:20 <smooge> JamesAntill[m]1: oh I gzip'd all those up recently up until March 1st
18:08:51 <smooge> it was mainly to save space on the share before it hit 85%
18:09:22 <nirik> phsmoura: oh weird. it's suppsoed to be a different/staging volume. :( if it's the prod one, don't change it. ;)
18:09:22 <JamesAntill[m]1> Yeh, I just need the access ones ... the import is taking ages and it not being gzipped might help it out
18:09:34 <smooge> I would go with
18:09:56 <JamesAntill[m]1> I could do a month at a time, and then gzip them back up after doing each month?
18:10:12 <smooge> for i in $( find /mnt/fedora_stats/combined-http/2023/ -type f -name "mirrors.fedoraproject.org-access.log.gz" ) ; do gunzip -v $i done
18:10:14 <phsmoura> nirik: too late, but still only the reporter user is broke
18:10:27 <nirik> phsmoura: oh, it's staging... ntap-iad2-c02-fedora01-nfs01a:/fedora_app_staging/app/attachments  920G  239G  682G  26% /mnt/web/attachments... note the 'fedora_app_staging' there. It was copied from the prod one tho, so it's likely very similar.
18:10:48 <smooge> JamesAntill[m]1: it should be fine.. koji logs are usually what takes up a lot of space
18:10:57 <JamesAntill[m]1> smooge: Sure, that should be the same but I'll do it that way if you want.
18:10:59 <smooge> mirrors used to be the biggest.. but well
18:10:59 <JamesAntill[m]1> Ahh, cool.
18:11:07 <JamesAntill[m]1> koji ftw
18:11:43 <phsmoura> nirik: weird, because yesterday I uploaded an image on my user in stg and it appeared on prod.. will check later
18:12:01 <smooge> JamesAntill[m]1: the other would have worked also.. I know after a couple of 'months' gunzip says too many files
18:12:11 <nirik> strange.
18:13:13 <nirik> sometimes redirects get you... ie, you go to stg, but somewhere it redirects you to production...
18:13:41 <nirik> James Antill: have you found the problem with it yet? or still digging?
18:16:28 <smooge> oh JamesAntill[m]1 there is a 'warning' with the raw countme tool. I think I fixed it in the base code, but I didn't go any further because of time
18:16:36 <phsmoura> nirik: https://paste.centos.org/view/49579efa
18:16:39 <JamesAntill[m]1> atm. I'm just trying to make it usable, and import this years data. If I can hit the original problem I can maybe fix it :(
18:16:54 <phsmoura> ah you re right
18:17:19 <phsmoura> I saw now, but weird... Ill check it later
18:17:36 <smooge> basically a python library update no longer likes a seperator which used to be ok, and anytime it shows up in a log file it spits out a complaint about this causing an error in the future. I didn't see if impacting running code however
18:17:37 <nirik> ok
18:17:55 <JamesAntill[m]1> smooge: The query thing?
18:18:13 <JamesAntill[m]1> smooge: Yeh, I saw that ... wasn't sure if I could ignore it ... did you have a patch to fix it?
18:19:32 <JamesAntill[m]1> I have a couple of minor patches for upstream atm. ... and FWIW the gunzip thing has made it a lot faster.
18:20:38 <smooge> I honestly thought I had made a PR for it.
18:23:40 <phsmoura> Is there anything else?
18:24:14 <smooge> not from me
18:24:39 <nirik> looks like we will be doing a rc2 today... otherwise nothing from me.
18:25:08 <phsmoura> thank you all :)
18:25:10 <phsmoura> #endmeeting