18:00:32 #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 18:00:32 Meeting started Thu Apr 2 18:00:32 2020 UTC. 18:00:32 This meeting is logged and archived in a public location. 18:00:32 The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:32 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:32 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:32 #chair cverna mboddu nirik smooge 18:00:32 Current chairs: cverna mboddu nirik smooge 18:00:33 #info meeting is 30 minutes MAX. At the end of 30, its stops 18:00:33 #info agenda is at https://board.net/p/fedora-infra-daily 18:00:33 #topic Tickets needing review 18:00:33 #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:00:42 ha nice did not know about the sop 18:00:55 .hello mohanboddu 18:00:55 mattdm: https://docs.pagure.org/infra-docs/sysadmin-guide/sops/freenode-irc-channel.html 18:00:56 mboddu: mohanboddu 'Mohan Boddu' 18:01:20 hllo 18:01:28 just made it bck 18:01:31 .ticket 8802 18:01:32 nirik: Issue #8802: Fedora planet is not updating (again) - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8802 18:01:34 hell 18:01:36 mattdm, what is the difference between #fedora-social-hour and #fedora-social? 18:01:52 this just got stuck again... I don't know if we want to come up with a longer term solution or not here. 18:02:28 pagure.issue.edit -- kevin edited the close_status and status fields of ticket fedora-infrastructure#8802 https://pagure.io/fedora-infrastructure/issue/8802 18:02:29 pagure.issue.comment.added -- kevin commented on ticket fedora-infrastructure#8802: "Fedora planet is not updating (again)" https://pagure.io/fedora-infrastructure/issue/8802#comment-639699 18:02:43 I guess we could stick in a 'pkill -f planet' in cron. ;) 18:02:48 not sure how much effort it would need 18:02:56 do we know what is failing ? 18:03:15 it's hanging... some bad/crap web server is just hanging and it never finishes it's run. 18:04:18 I guess we can ponder and revisit if it keeps happening more? 18:04:19 I think it is multiple ones 18:04:30 .ticket 8803 18:04:31 nirik: Issue #8803: Link sonarqube to noggin - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8803 18:04:34 ok yeah I guess the cron might not be a bad idea 18:04:37 this is done? 18:04:41 if we don't want to spend much more time on it :) 18:04:43 I did this a while back. 18:05:12 or perhaps something else. 18:05:34 I guess a webhook? 18:05:56 I could check with james tomorrow, and try to clarify what is needed 18:06:01 I can do this but we need the info for the webhook. 18:06:02 I ll assign this to me 18:06:06 or you could cverna, sure. 18:06:15 pagure.issue.assigned.added -- cverna assigned ticket fedora-infrastructure#8803 to cverna https://pagure.io/fedora-infrastructure/issue/8803 18:06:24 .ticket 8804 18:06:25 nirik: Issue #8804: Playing with packages app // CommunityShift access - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8804 18:06:38 pagure.issue.tag.added -- cverna tagged ticket fedora-infrastructure#8803: low-trouble and medium-gain https://pagure.io/fedora-infrastructure/issue/8803 18:06:39 pagure.issue.edit -- cverna edited the priority fields of ticket fedora-infrastructure#8803 https://pagure.io/fedora-infrastructure/issue/8803 18:06:40 I can do this one, or someone else can, but we should note to them the upcoming downtime. 18:06:48 so, move to waiting on assignee 18:07:06 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#8804 https://pagure.io/fedora-infrastructure/issue/8804 18:07:15 .ticket 8805 18:07:16 yeah I believe smooge did on the ticket and I did mentioned it on the mailing list 18:07:16 nirik: Issue #8805: Break backups into smaller volumes - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8805 18:07:31 how do we give access to communishift ? a FAS group ? 18:07:32 my first unscheduled meeting of the day 18:07:35 smooge: where did this request get to you? 18:07:43 nirik, chat.google.com to you and me 18:07:48 * nirik sighs 18:08:23 so... this can be probibly reduced to less than 10TB 18:08:30 I even suggested I could do that. 18:08:40 but then everyone said "no no no, not before the move!" 18:08:51 nb I guess we could bridge to #fedora-social, but this is kind of meant to be a event-based thing rather than ongoing 18:09:11 so anyhow, assign it to me, waiting on assignee, and I can look at it. 18:09:27 so they need the various volumes down to 10 TB because we are now causing an impact. they can be up to 15 but no larger 18:09:28 will do so 18:09:50 mattdm, I would suggest re-using #fedora-social, to cut down on making yet another channel, although if you want another channel, sure, that can be done 18:10:04 ok, I can look at this (again) :) 18:10:05 you'd probably get more people participating if you used -social 18:10:10 since people are already there 18:10:17 .ticket 8806 18:10:18 nirik: Issue #8806: MBS builds fail with "[Errno 12] Cannot allocate memory" - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8806 18:10:35 pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#8805 to kevin https://pagure.io/fedora-infrastructure/issue/8805 18:10:36 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#8805 https://pagure.io/fedora-infrastructure/issue/8805 18:10:36 I fixed this (I think) by restarting fedmsg-hub on mbs-backend01. 18:10:48 however, I think there's a ticket to fix this so we don't have to do that all the time... 18:10:51 ok will ask in ticket 18:11:05 still using fedmsg :( 18:11:18 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#8806: mbs https://pagure.io/fedora-infrastructure/issue/8806 18:11:19 pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#8806 to kevin https://pagure.io/fedora-infrastructure/issue/8806 18:11:20 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#8806 https://pagure.io/fedora-infrastructure/issue/8806 18:11:58 Iam looking for it. 18:12:08 smooge: we should just close 8806 I think 18:12:21 .ticket 8445 18:12:23 nirik: Issue #8445: Provide newer versions of python-kerberos and python-requests-kerberos in epel7-infra-stg - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8445 18:12:31 this is the one ^ we need some rpms to make it not leak 18:12:47 pagure.issue.edit -- smooge edited the close_status and status fields of ticket fedora-infrastructure#8806 https://pagure.io/fedora-infrastructure/issue/8806 18:13:05 so, perhaps we should try and get that done soon... 18:13:21 I can try or smooge was going to? or does someone else want it? :) 18:13:51 MBS just fell over again, I think -- getting error 500 when submitting a build 18:14:00 could it be something we can try to do during the packaging hackfest tomorrow 18:14:16 I was supposed to do so and it fell into a hole 18:14:46 kalev, there is an ongoing outage in proxies at the moment 18:14:56 kalev: it's ipsilon. I am working on it. 18:15:07 cverna: oh, yeah, +1 18:15:23 kalev: might be back... try again? 18:16:19 so, thats all the needs review. 18:16:21 nirik: smooge: yep, works now, thanks! 18:16:29 any blockers, reviews, other tickets to discuss? 18:17:10 nothing here that I can think of 18:17:20 do we have any releng tickets ? 18:17:43 * nirik looks 18:17:57 .releng 9380 18:18:04 nirik: Issue #9380: Clean up side-tags in koji.stg - releng - Pagure.io - https://pagure.io/releng/issue/9380 18:18:14 this one should be easy to do... just fedpkg remove-side-tag on them (if you are a admin there) 18:18:28 .releng 9364 18:18:31 nirik: Issue #9364: drop fedora-release and fedora-repos from the secure-boot channel - releng - Pagure.io - https://pagure.io/releng/issue/9364 18:18:41 this is done, just waiting on more history, but I think we can just close it. 18:18:54 nirik: what does one need for 9380? 18:18:56 2 unretirement requests 18:18:59 koji admin priv? 18:19:06 pingou: yeah, I think so. 18:19:08 ok 18:20:08 you can list them easily with 'koji list-targets' too 18:21:19 so my updates are still running on some s390x builders. ;( probibly stuck. I will clean them up... 18:21:27 and we can start reboot later / tomorrow. 18:21:34 I don't have anything else off hand. 18:21:48 I keep running into transient s390x builder issues all the time 18:22:00 should I note what builders are failing and file tickets or is it expected? 18:22:33 it's not much of a trouble to resubmit builds, just wondering if it would be helpful to gather failing host names or not 18:22:35 depends... whats the issue? 18:22:41 unpacking the src.rpm? 18:22:59 I don't know, they have been failing so much that I've not bothered to even look lately :) it used to be that, yes 18:23:23 well, thats still the issue I am aware of. it's not fixed. 18:23:28 https://pagure.io/koji/issue/1974 is the upstream issue 18:23:46 * kalev nods. 18:23:59 I have had 0 time to apply that debugging. 18:24:01 anyway, I didn't at all want to cause more work here, just wondered if it would be helpful :) 18:24:32 if you could look and see if it's the src.rpm or something else we don't know about yet that could be handy 18:24:44 ok, will do 18:24:45 if it's the src.rpm thing... we know. ;) 18:24:48 ok 18:25:37 MBS fell over again :( 18:25:37 Reason: [Errno 12] Cannot allocate memory 18:25:49 ok, if nothing else I have pleanty to do and we can end here? or anything from anyone else? 18:26:23 #endmeeting