19:01:18 <nirik> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting
19:01:18 <zodbot> Meeting started Wed Feb  9 19:01:18 2022 UTC.
19:01:18 <zodbot> This meeting is logged and archived in a public location.
19:01:18 <zodbot> The chair is nirik. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
19:01:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:01:18 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
19:01:18 <nirik> #chair mboddu nirik pingou mobrien nb
19:01:18 <zodbot> Current chairs: mboddu mobrien nb nirik pingou
19:01:18 <nirik> #meetingname fedora_infrastructure_ops_daily_standup_meeting
19:01:18 <nirik> #info meeting is 30 minutes MAX. At the end of 30, its stops
19:01:19 <nirik> #info agenda is at https://board.net/p/fedora-infra-daily
19:01:19 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
19:01:45 <eddiejennings> .hi
19:01:46 <zodbot> eddiejennings: eddiejennings 'Eddie Jennings' <eddie@eddiejennings.net>
19:01:55 <phsmoura> hello
19:02:28 <nirik> morning everyone
19:02:42 <mboddu> Hello
19:03:30 <nirik> lets dive in...
19:03:50 <nirik> .ticket 10539
19:03:51 <zodbot> nirik: Issue #10539: Branch Koschei - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10539
19:03:58 <nirik> just needs 'waiting on assignee'
19:04:26 <mboddu> +1
19:04:38 <nirik> .ticket 10541
19:04:41 <zodbot> nirik: Issue #10541: Nagios checks for pagure.io - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10541
19:04:55 <nirik> so, we do monitor it... just perhaps not whatever is failing
19:05:30 <nirik> likely we need a website check for some page it needs to generate...
19:05:44 <nirik> anyhow, med gain, low trouble, ops?
19:06:00 <mboddu> +1
19:06:01 <eddiejennings> I thought pagure.io was outside the control of Fedora Infra
19:06:13 <eddiejennings> or maybe I'm thinking of the meeting agenda site
19:06:37 <nirik> no, we run pagure.io... it's called 'pagure02.fedoraproject.org' internally to our ansible tho
19:06:49 <nirik> .ticket 10543
19:06:50 <zodbot> nirik: Issue #10543: Add IP v6 verification for fedorapeople - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10543
19:07:02 <eddiejennings> noted
19:07:33 <nirik> this almost seems like a duplicate of that last one, just need to add some monitoring. note that only noc02 (nagios-external) can monitor ipv6 ips
19:07:45 <nirik> low low ops? or duplicate and add in with the last one?
19:08:11 <mboddu> Maybe mark it low, low, ops because of https://pagure.io/fedora-infrastructure/issue/10541#comment-780283
19:08:20 <mboddu> Just to make sure they might be different
19:09:22 <nirik> ok.
19:09:34 <nirik> .ticket 10544
19:09:35 <zodbot> nirik: Issue #10544: Unhealthy armv7hl builder - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10544
19:09:49 <nirik> so the builder seems fine from a quick glance. I am not sure why it would be so slow...
19:10:36 <mboddu> If its only armv7hl, maybe just wait until next week and we can just remove it?
19:10:45 <mboddu> and also just that one builder
19:11:00 <nirik> ha. I wish. no, we have to have builders until f35 goes eol
19:11:16 <nirik> but anyhow, I guess med med ops and we can investigate more.
19:11:16 <mboddu> Oh crap, right, its been a long day
19:11:23 <mboddu> Yeah, +1
19:11:53 <mboddu> nirik: FYI, not f35 eol, they should live until F36 EOL
19:11:57 <nirik> .ticket 10545
19:11:58 <zodbot> nirik: Issue #10545: Assistance during libravatar system upgrade - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10545
19:12:09 <nirik> yeah, f36.
19:12:22 <nirik> low low ops, and mobrien said he could do this I think...
19:13:00 <mboddu> Okay, I will assign the ticket to him
19:13:52 <nirik> thats it for new infra ones, how about releng?
19:14:21 <mboddu> Yeah, there are a few
19:14:45 <mboddu> .releng 10630
19:14:46 <zodbot> mboddu: Issue #10630: Delete pagure branch - releng - Pagure.io - https://pagure.io/releng/issue/10630
19:14:48 <mboddu> low, low, ops
19:15:04 <nirik> +1
19:15:11 <mboddu> .releng 10631
19:15:12 <zodbot> mboddu: Issue #10631: Unable to create new update for dropwatch on budhi - releng - Pagure.io - https://pagure.io/releng/issue/10631
19:15:26 <mboddu> This needs some investigation
19:15:30 <nirik> I'm not sure what could be going on there...
19:16:12 <mboddu> I asked the reporter to logout and login again, but still no luck
19:16:19 <mboddu> So, med, med, ops
19:16:26 <mboddu> Or even high trouble
19:16:55 <nirik> +1 to med/med/ops
19:17:19 <mboddu> .releng 10632
19:17:20 <zodbot> mboddu: Issue #10632: Issues with conmon package - releng - Pagure.io - https://pagure.io/releng/issue/10632
19:17:24 <nirik> might try removing .fedora/openidbaseclient-sessions.cache and/or making sure they have a valid kerberos ticket and/or make sure they can login to accounts ok
19:17:31 <mboddu> low, low, ops (or even might close it)
19:18:03 <nirik> +1
19:18:12 <nirik> low/low/ops for now and close in a few days?
19:19:29 <mboddu> .releng 10633
19:19:30 <zodbot> mboddu: Issue #10633: Wrong version of sssd in repos - releng - Pagure.io - https://pagure.io/releng/issue/10633
19:19:34 <mboddu> I will just fix it now and close it
19:20:04 <nirik> we might run that check script
19:21:05 <mboddu> Yeah, generally we run it after a freeze but due to some bodhi wonky stuff in between we should run it again and verify if everything is good
19:21:12 * nirik can do that
19:21:17 <mboddu> Anyway, I fixed it and we close the ticket
19:21:20 <nirik> +1
19:21:29 <mboddu> nirik: I can take care of it, dont worry
19:22:09 <mboddu> Thats all from releng
19:22:17 <nirik> no worries. Its easy to run...
19:22:19 <nirik> ok.
19:22:24 <nirik> #topic Planning / Upcoming work
19:22:32 <nirik> so, lets see...
19:23:12 <nirik> signing is still slowly going on f37. It kept getting stuck with batches, so I started it doing 1 at a time. ;( I might try some big batches later when things are more quiet.
19:23:27 <nirik> the f36 branched sync is going now... hopefully that will finish soon.
19:23:59 <nirik> mboddu: where exactly is the mass branching sop? this one: https://docs.pagure.org/releng/sop_mass_branching.html ?
19:24:18 <mboddu> nirik: Yup
19:24:18 <nirik> It was suggested we disable builds next time during mass branching and change wording on some of the announcements.
19:24:39 <mboddu> Yeah, but should we get the approval from fesco about disabling builds?
19:24:53 <mboddu> Announcements wording can be changed easily
19:25:10 <nirik> are the announcements just reuse from last time? perhaps we should stick them in git...
19:25:23 <mboddu> Yeah, they are from last time
19:25:32 <nirik> yeah, we could run it by fesco, but it's just another way of freezeing... but we can
19:25:44 <nirik> we should put them all in git so we can get pr's to improve them, etc.
19:25:55 <mboddu> Yeah, that would be nice
19:25:58 <mboddu> I will work on that
19:26:03 <mboddu> For all the announcements
19:26:05 <mboddu> In one page
19:26:35 <mboddu> I can also create a ticket with FESCo about disabling building
19:26:36 <nirik> I'd say make a subdir and have them one per announcement?
19:26:49 <nirik> well, before we do that, we need to figure out how we are going to do it.
19:27:04 <mboddu> I am thinking of disabling the build targets
19:27:31 <mboddu> I wonder how the bots will react
19:27:34 <nirik> well, that would work, but... give a kind of confusing error and also not allow us to make fedora-release/fedora-repos.
19:28:08 <mboddu> I mean disable them after building those two packages
19:28:41 <nirik> well, part of the problem is that maintainers build while we are doing things... so they are not sure what is going to happen with their build.
19:28:58 <nirik> it would be nice to have a way to stop at the beginning, do everything and then re-enable
19:29:57 <mboddu> We could do that as well and build those two packages with `koji build...` command
19:30:03 <nirik> anyhow, I told the requestor to file a releng ticket on it, so we can discuss there and move to fesco once we decide how
19:31:01 <mboddu> +1 and koji build doesn't work as it needs a target
19:31:56 <nirik> we could see about preventing commits on branched/rawhide... no commits, no builds.
19:32:10 <nirik> anyhow, can ponder on it.
19:32:18 <nirik> Also on the upcoming slate:
19:32:44 <nirik> #info next week mass update/reboot cycle... monday: staging, tuesday: proxies and non outage causing, wed: the rest.
19:33:00 <nirik> #info next friday is a recharge day at Red Hat, many RH folks will be away.
19:33:12 <nirik> anything else for upcoming work/planning?
19:33:35 <nirik> #topic Discussion / AOB
19:33:44 <mboddu> Nothing from me
19:33:45 <nirik> anything to discuss? we are over time I guess.
19:34:25 <nirik> ok, thanks for coming everyone!
19:34:28 <nirik> #endmeeting