18:00:14 #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 18:00:14 Meeting started Tue Apr 11 18:00:14 2023 UTC. 18:00:14 This meeting is logged and archived in a public location. 18:00:14 The chair is phsmoura. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 18:00:14 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:14 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:14 #chair nirik smooge phsmoura 18:00:14 #meetingname fedora_infrastructure_ops_daily_standup_meeting 18:00:14 Current chairs: nirik phsmoura smooge 18:00:14 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:14 #info meeting is 30 minutes MAX. At the end of 30, its stops 18:00:14 #info agenda is at https://board.net/p/fedora-infra-daily 18:00:21 hello 18:00:22 hello 18:00:53 morning 18:01:25 #info reminder: speak up if you want to work on a ticket! 18:01:26 #topic Tickets needing review 18:01:40 #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:01:53 .ticket 11233 18:01:54 phsmoura: Issue #11233: File status mismatch in EPEL mirror ib01 but looks good in iad01 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/11233 18:02:17 med med ops... 18:02:27 Hopefully it's going to catch up soon 18:03:07 it had several issues. Also, it's on an old machine that we were hoping to replace before now. ;( 18:03:53 done 18:03:55 #info https://pagure.io/releng/issues?status=Open 18:04:04 .releng 11382 18:04:07 phsmoura: Issue #11382: Retire Node.js modules - releng - Pagure.io - https://pagure.io/releng/issue/11382 18:04:20 low low ops? 18:05:36 +1 18:05:45 thats it, open floor now 18:05:47 #topic Planning, Upcoming work and Open floor 18:06:29 what modules are left? 18:06:41 today has been a second monday. ;) 18:06:53 I am on the 11th monday in a row 18:07:03 i expect it has been longer... 18:07:20 anything that can help nirik ? 18:07:43 So I have a quick question ... would it be ok if I ran something like: gunzip /mnt/fedora_stats/combined-http/2023///mirrors.fedoraproject.org-access.log.gz 18:07:48 smooge: there's a bunch left. 18:08:08 James Antill: not sure, smooge? 18:08:10 Im still catching up with emails, but yesterday I was working on fedora wiki the issue persists with that user Im trying to reset his images and see if it works, was having trouble with permissions in the end of the day. Also the disk mounted in stg is the same as in prod just fyi 18:08:18 Should be * in those dirs. 18:08:20 JamesAntill[m]1: oh I gzip'd all those up recently up until March 1st 18:08:51 it was mainly to save space on the share before it hit 85% 18:09:22 phsmoura: oh weird. it's suppsoed to be a different/staging volume. :( if it's the prod one, don't change it. ;) 18:09:22 Yeh, I just need the access ones ... the import is taking ages and it not being gzipped might help it out 18:09:34 I would go with 18:09:56 I could do a month at a time, and then gzip them back up after doing each month? 18:10:12 for i in $( find /mnt/fedora_stats/combined-http/2023/ -type f -name "mirrors.fedoraproject.org-access.log.gz" ) ; do gunzip -v $i done 18:10:14 nirik: too late, but still only the reporter user is broke 18:10:27 phsmoura: oh, it's staging... ntap-iad2-c02-fedora01-nfs01a:/fedora_app_staging/app/attachments 920G 239G 682G 26% /mnt/web/attachments... note the 'fedora_app_staging' there. It was copied from the prod one tho, so it's likely very similar. 18:10:48 JamesAntill[m]1: it should be fine.. koji logs are usually what takes up a lot of space 18:10:57 smooge: Sure, that should be the same but I'll do it that way if you want. 18:10:59 mirrors used to be the biggest.. but well 18:10:59 Ahh, cool. 18:11:07 koji ftw 18:11:43 nirik: weird, because yesterday I uploaded an image on my user in stg and it appeared on prod.. will check later 18:12:01 JamesAntill[m]1: the other would have worked also.. I know after a couple of 'months' gunzip says too many files 18:12:11 strange. 18:13:13 sometimes redirects get you... ie, you go to stg, but somewhere it redirects you to production... 18:13:41 James Antill: have you found the problem with it yet? or still digging? 18:16:28 oh JamesAntill[m]1 there is a 'warning' with the raw countme tool. I think I fixed it in the base code, but I didn't go any further because of time 18:16:36 nirik: https://paste.centos.org/view/49579efa 18:16:39 atm. I'm just trying to make it usable, and import this years data. If I can hit the original problem I can maybe fix it :( 18:16:54 ah you re right 18:17:19 I saw now, but weird... Ill check it later 18:17:36 basically a python library update no longer likes a seperator which used to be ok, and anytime it shows up in a log file it spits out a complaint about this causing an error in the future. I didn't see if impacting running code however 18:17:37 ok 18:17:55 smooge: The query thing? 18:18:13 smooge: Yeh, I saw that ... wasn't sure if I could ignore it ... did you have a patch to fix it? 18:19:32 I have a couple of minor patches for upstream atm. ... and FWIW the gunzip thing has made it a lot faster. 18:20:38 I honestly thought I had made a PR for it. 18:23:40 Is there anything else? 18:24:14 not from me 18:24:39 looks like we will be doing a rc2 today... otherwise nothing from me. 18:25:08 thank you all :) 18:25:10 #endmeeting