18:00:00 #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 18:00:00 Meeting started Mon Aug 31 18:00:00 2020 UTC. 18:00:00 This meeting is logged and archived in a public location. 18:00:00 The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:00 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:00 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:00 #chair mboddu nirik smooge pingou mobrien 18:00:01 #meetingname fedora_infrastructure_ops_daily_standup_meeting 18:00:01 #info meeting is 30 minutes MAX. At the end of 30, its stops 18:00:01 #info agenda is at https://board.net/p/fedora-infra-daily 18:00:01 Current chairs: mboddu mobrien nirik pingou smooge 18:00:01 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:01 #topic Tickets needing review 18:00:03 #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:00:17 * nirik waits to see who's around and who wants to modify tickets. 18:01:21 I am here 18:01:27 And I can modify tickets 18:01:35 cool. 18:01:43 .ticket 9281 18:01:44 nirik: Issue #9281: Logging in bugzilla via ipsilon returns a 500 error - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9281 18:01:56 I wonder on this one if we shouldn't file an upstream ticket... 18:02:32 should or shouldn't? 18:02:33 where? 18:03:05 https://pagure.io/ipsilon/ 18:03:14 should file upstream instead 18:03:29 because I don't know that we have the understanding to fix this... 18:03:39 or I suppose it could be on the bugzilla side, I am not sure. 18:04:00 pagure.issue.edit -- mobrien edited the close_status and status fields of ticket fedora-infrastructure#9283 https://pagure.io/fedora-infrastructure/issue/9283 18:04:01 pagure.issue.comment.added -- mobrien commented on ticket fedora-infrastructure#9283: "Update Varnish configuration to work with version 6.4.0" https://pagure.io/fedora-infrastructure/issue/9283#comment-674651 18:04:17 nirik: Lets start with ipsilon and see what they can find 18:04:36 ok, should we ask pingou to refile? or just do it ourselves? 18:04:37 And we dont know if its persistent as Pierre is not around atm 18:04:55 * mboddu can file the ticket 18:05:04 Close and move upstream? 18:05:06 yep. 18:05:24 ack 18:05:25 I think it's sporadic... might be bugzilla is sometimes throwing the wrong thing 18:05:46 .ticket 9282 18:05:47 nirik: Issue #9282: chromium builds hanging on aarch64 builders in koji - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9282 18:06:05 so, this was due to both heavybuilder aarch64 boxes not checking in... 18:06:10 because kojid was oom killed. 18:06:27 I'd like to push a change to set kojid to Restart on failure... 18:06:48 so, keep this open to track that? or we could just close it. 18:06:55 pagure.issue.new -- mohanboddu opened a new ticket ipsilon#343: "Logging in bugzilla via ipsilon returns a 500 error" https://pagure.io/ipsilon/issue/343 18:07:19 pagure.issue.edit -- mohanboddu edited the close_status and status fields of ticket fedora-infrastructure#9281 https://pagure.io/fedora-infrastructure/issue/9281 18:07:20 pagure.issue.comment.added -- mohanboddu commented on ticket fedora-infrastructure#9281: "Logging in bugzilla via ipsilon returns a 500 error" https://pagure.io/fedora-infrastructure/issue/9281#comment-674653 18:07:26 or... another solution: 18:07:28 I would say keep it open as then you can reference the ticket in the freeze break request 18:07:49 +1 18:08:04 set: OOMScoreAdjust=-1000 in the kojid unit, so the oom killer never kills it. 18:08:07 pagure.issue.comment.added -- ngompa commented on ticket ipsilon#343: "Logging in bugzilla via ipsilon returns a 500 error" https://pagure.io/ipsilon/issue/343#comment-674657 18:08:44 ok, so med/med/ops? 18:08:47 I am not sure not killing it is a good idea 18:08:53 +1 18:09:11 pagure.issue.tag.added -- mohanboddu tagged ticket fedora-infrastructure#9282: medium-gain, medium-trouble, and ops https://pagure.io/fedora-infrastructure/issue/9282 18:09:12 pagure.issue.edit -- mohanboddu edited the priority fields of ticket fedora-infrastructure#9282 https://pagure.io/fedora-infrastructure/issue/9282 18:09:35 well, restarting also can be bad... if there's a config issue or something it will keep cycling. 18:09:44 True 18:09:51 There is not good solution 18:09:59 mobrien: you want to discuss 9283 ? 18:10:03 Just a series of bad ones like a weekend bender 18:10:11 Haha, true smooge 18:11:14 nirik I downgraded varnish to version 6.3.2 which was on another proxy which is working and its stiill bust so I closed the ticket as invalid as that is not the cause of it 18:11:33 huh. ;( 18:12:01 mobrien: different gcc versions? 18:12:13 I'm not really sure why its breaking, you'd think since its ansible its the same everywhere 18:12:18 I'll checkk gcc 18:12:33 varnish amusingly uses gcc to 'compile' it's rules... 18:12:58 thats all the infra ones. 18:13:02 mboddu: any releng ones? 18:13:16 Well, none right now as I fixed them already :) 18:13:23 But I have a ticket to discuss 18:13:32 huh, the newer proxy has an older version of gcc 18:13:35 😎 18:13:44 man varnish should use yacc 18:13:55 .releng 9468 18:13:56 mboddu: go head 18:13:56 mboddu: Issue #9468: Please send openh264 2.1.1 builds to Cisco - releng - Pagure.io - https://pagure.io/releng/issue/9468 18:14:10 nirik: The binaries are deployed on Cisco CDN 18:14:28 But the compose dir has changed the structure 18:14:30 cool. Did we get rawhide too? 18:14:34 Yup 18:14:36 oh hum. 18:14:47 did repodata move? 18:15:14 Yup 18:15:29 nirik: The old os/ vs non os/ dir 18:15:44 Well, the newer composes has os/ in them 18:16:25 so, I'd say: sync the new one and make sure it's in place... get adrianr to change it in the mirrormanager db, remove the old ones? 18:17:09 Thats what I am thinking 18:17:17 yeah, should work fine 18:17:26 mv the old ones and then remove them if no one complains 18:17:47 sure, to be safe. ;) 18:18:12 ok, anything else on that or from releng? 18:18:16 nirik: And with the metalink change, they are still pointing to /srv/web/codecs.fedoraproject.org/openh264/ on sundries01, right? 18:18:36 Just making sure as I forgot what we changed :D 18:19:03 mboddu: no, they point to the proxies... but the proxies sync from sundries 18:19:23 https://codecs.fedoraproject.org/openh264/33/x86_64/repodata/repomd.xml 18:19:28 Ah right, so sync to sundries should just work fine as we used to, thanks 18:19:42 yep 18:20:03 Thanks nirik 18:20:05 Thats all I got 18:20:30 Oh, I had one releng thing: after a really long path, I have the windows signing cert now, can pass that on to mboddu and jednorozec. I am still working on the apple one. 18:20:46 ok, any other tickets or issues or prs or anything folks want to bring up? 18:20:51 nirik: Awesome news 18:21:08 None from me 18:21:50 I'm planning on working on stg deployment today. Hopefully unblock the noggin team at least 18:22:00 staging++ 18:23:09 Sorry, guys, the candidates to join the apprentice team should wait to the weekly meeting? 18:23:12 pagure.issue.comment.added -- simo commented on ticket ipsilon#343: "Logging in bugzilla via ipsilon returns a 500 error" https://pagure.io/ipsilon/issue/343#comment-674660 18:23:53 miausX: nope, you can ask here... ;) did you send your into email to the list yet? and see https://fedoraproject.org/wiki/Infrastructure/GettingStarted 18:23:59 miausX: If you already attended one of the weekly meetings, then you can request it here 18:24:13 nirik: Yup! 18:24:54 mboddu: Nope, never attended one. 18:25:52 nirik: I know I have a long way behind me to be useful. Just wanted to join the apprentice team to learn. 18:26:01 I guess the email to the list is good enough 18:26:20 miausX: is your fas login the same as your irc nick? or ? 18:26:35 if nothing more for the meeting, closing that out 18:26:37 #endmeeting