18:01:02 #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 18:01:02 Meeting started Tue Jul 7 18:01:02 2020 UTC. 18:01:02 This meeting is logged and archived in a public location. 18:01:02 The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:02 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:01:02 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:01:03 #chair cverna mboddu nirik smooge 18:01:03 #meetingname fedora_infrastructure_ops_daily_standup_meeting 18:01:03 Current chairs: cverna mboddu nirik smooge 18:01:03 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:01:03 #info meeting is 30 minutes MAX. At the end of 30, its stops 18:01:03 #info agenda is at https://board.net/p/fedora-infra-daily 18:01:04 #topic Tickets needing review 18:01:05 #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:01:20 oh hai 18:01:27 .hello2 18:01:28 nb: nb 'Nick Bebout' 18:01:41 Hello 18:01:53 who wants to run things and who wants to edit? :) 18:02:13 .ticket 9115 18:02:14 smooge: Issue #9115: Broken builder buildvm-ppc64le-05.iad2.fedoraproject.org - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9115 18:02:33 reinstall the host? 18:02:48 I haven't looked at it yet, rebooting might fix it... 18:02:55 but I need to reinstall them anyhow 18:03:00 to move them around 18:03:23 groomed, medium-trouble, medium-? 18:03:40 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9115: groomed, medium-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/9115 18:03:41 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9115 https://pagure.io/fedora-infrastructure/issue/9115 18:03:43 ack 18:04:40 .ticket 9116 18:04:42 smooge: Issue #9116: Container builds don't work - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9116 18:05:09 only random thought here: might be firewall rules? but not sure how to fix them... 18:05:16 me either.. 18:05:24 not sure what builds them or where 18:05:32 osbs* 18:05:46 ah 18:06:07 FYI, this is related to https://pagure.io/releng/issue/9581 18:06:16 yeah, I bet thats the issue... could try rebooting them? 18:06:27 Not related, same issue 18:06:34 well, making sure the iptables fw is cleared and rebooting them 18:06:38 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9116: OSBS, high-trouble, and medium-gain https://pagure.io/fedora-infrastructure/issue/9116 18:06:39 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9116 https://pagure.io/fedora-infrastructure/issue/9116 18:06:54 oh wait. 18:06:59 thats the registry it can't talk to? 18:07:33 dunno, need more poking 18:08:05 pagure.issue.comment.added -- nphilipp commented on ticket fedora-infrastructure#8213: "fedmsg -> fedora-messaging migration tracker" https://pagure.io/fedora-infrastructure/issue/8213#comment-663927 18:08:51 .ticket 9117 18:08:51 pagure.issue.comment.added -- mohanboddu commented on ticket releng#9581: "Container images builds fail constantly for couple of days" https://pagure.io/releng/issue/9581#comment-663928 18:08:52 smooge: Issue #9117: Add oamg group to the list of src.fedoraproject.org access groups - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9117 18:09:09 looks like pingou had this one... 18:09:38 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9117: groomed, medium-gain, and 2 others https://pagure.io/fedora-infrastructure/issue/9117 18:09:39 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9117 https://pagure.io/fedora-infrastructure/issue/9117 18:09:54 .ticket 9118 18:09:55 smooge: Issue #9118: src.fp.o fork stuck in "updating permissions" (not writable, not deletable) - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9118 18:10:22 need pingou to look I guess. 18:10:37 src.fpo high-trouble medium-gain groomed ? 18:10:52 ack 18:11:04 sure. +1 18:11:04 with a ping in the ticket to pingou 18:11:08 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9118: groomed, high-gain, and 2 others https://pagure.io/fedora-infrastructure/issue/9118 18:11:09 pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#9118: "src.fp.o fork stuck in "updating permissions" (not writable, not deletable)" https://pagure.io/fedora-infrastructure/issue/9118#comment-663931 18:11:30 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9118 https://pagure.io/fedora-infrastructure/issue/9118 18:11:51 .ticket 9119 18:11:52 smooge: Issue #9119: FedoraMessaging: unsubscribe osci-pipelines-queue-5 from topics - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9119 18:11:56 bw? 18:12:04 bw? 18:12:11 bw?> 18:12:14 bwa? no idea 18:12:22 just needs doing. I can do it... 18:12:35 you can do this kind of thing from the web interface... 18:12:43 at least I think you can. 18:12:45 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9119: groomed, medium-gain, and 2 others https://pagure.io/fedora-infrastructure/issue/9119 18:12:46 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9119 https://pagure.io/fedora-infrastructure/issue/9119 18:13:06 well I put it as medium trouble.. if it turns out the web interface can't do it.. then it is high trouble 18:13:48 that is all for infra tickets.. how goes the releng queue mboddu ? 18:15:16 smooge: I just closed the container build issue as a duplicate to the infra one 18:15:28 Thats the only one releng has 18:16:04 ok cool. I am still working on the inventory of systems to get an idea of how many are still needing to be reinstalled/setup etc. 18:16:35 once I get that done it is a matter of reallocation and vlan changes 18:17:05 smooge: When do you think all the phx2 infra will be setup in iad2? 18:17:14 August? 18:17:21 Okay 18:17:37 It has taken me 5 days to get just an idea of what hardware we have left to do and I keep getting pulled into other thigns 18:17:50 for some reason people think its all done 18:17:55 after we get things back up and installed, we still need to rebuild up staging... thats gonna take some time 18:18:05 I understand 18:18:06 yeah... ;( 18:19:00 But luckily most of stuff is running at higher performance compared to phx2 18:19:27 yep... 18:19:54 Either its the bump in networking speeds or something else, but its good that we have better performance with lesser hardware 18:20:24 the new systems have more cores, more ram, faster SSD drives and 10 gig network 18:21:12 the older systems will be spinning rust with less ram and cores. they will have 10 gig.. so we can compare that apples and oranges 18:21:45 ok if there isn't anything else today.. I will call this meeting closed. have a good day and see you all tomorrow.. same bat channel, same bat time 18:21:49 #endmeeting