16:01:31 <Penguinpee> #startmeeting Infrastructure (2023-03-09)
16:01:31 <zodbot> Meeting started Thu Mar  9 16:01:31 2023 UTC.
16:01:31 <zodbot> This meeting is logged and archived in a public location.
16:01:31 <zodbot> The chair is Penguinpee. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:01:31 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:01:31 <zodbot> The meeting name has been set to 'infrastructure_(2023-03-09)'
16:01:39 <aheath1992> .hi
16:01:40 <zodbot> aheath1992: aheath1992 'Andrew Heath' <aheath1992@gmail.com>
16:01:48 <Penguinpee> #meetingname infrastructure
16:01:48 <zodbot> The meeting name has been set to 'infrastructure'
16:01:57 <Penguinpee> #chair nirik zlopez nb bodanel dtometzki jnsamyak
16:01:57 <zodbot> Current chairs: Penguinpee bodanel dtometzki jnsamyak nb nirik zlopez
16:02:08 <Penguinpee> #info Agenda is at: https://board.net/p/fedora-infra
16:02:27 <mkonecny> .hello zlopez
16:02:28 <zodbot> mkonecny: zlopez 'Michal Konecny' <michal.konecny@pacse.eu>
16:02:30 <Penguinpee> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:02:41 <Penguinpee> #info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
16:02:50 <Penguinpee> #topic greetings!
16:02:52 <phsmoura> .hi
16:02:54 <zodbot> phsmoura: phsmoura 'Pedro Moura' <pmoura@redhat.com>
16:03:17 <Penguinpee> Hello everyone!
16:03:35 * Penguinpee looks outside and is glad to be inside ;)
16:04:30 <Penguinpee> I guess we wait a bit for people to show up?
16:04:36 <nirik> morning
16:04:44 <eddiejenningsjr> .hello eddiejennings
16:04:44 <Penguinpee> Hello sir!
16:04:45 <zodbot> eddiejenningsjr: eddiejennings 'Eddie Jennings' <eddie@eddiejennings.net>
16:05:15 <Penguinpee> #topic New folks introductions
16:05:20 <Penguinpee> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:05:28 <Penguinpee> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:05:45 <Penguinpee> Do we have any new people among us? Please say hello.
16:07:01 <darknao> .hi
16:07:02 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:07:29 <Penguinpee> I guess we have no new arrivals this week. Moving on...
16:07:32 <Penguinpee> #topic Next chair
16:07:44 <Penguinpee> #info magic eight ball says:
16:07:54 <Penguinpee> #info chair 2023-03-16 - aheath1992
16:07:55 <Penguinpee> #info chair 2023-03-23 - Canceled due to F2F CPE meeting
16:07:55 <Penguinpee> #info chair 2023-03-30 - nirik
16:08:12 <Penguinpee> Any volunteers for April 6th?
16:08:30 <mkonecny> I can take it
16:08:41 <Penguinpee> Sold to mkonecny!
16:09:04 <Penguinpee> #info chair 2023-04-06 - mkonecny
16:09:42 <Penguinpee> I guess we are good for the coming weeks. Unless someone really wants the 13th (no a Friday). ;)
16:10:46 <Penguinpee> Moving right along...
16:10:49 <Penguinpee> #topic announcements and information
16:10:49 <Penguinpee> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:10:49 <Penguinpee> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:10:49 <Penguinpee> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request
16:10:51 <Penguinpee> #info oncall should also handling #fedora-releng pings if possible
16:10:53 <Penguinpee> #info Fedora 38 beta freeze in effect
16:11:35 <Penguinpee> #info Penguinpee will miss next week's meeting due to work
16:12:00 <Penguinpee> Any more announcements?
16:12:07 <eddiejenningsjr> Sorry, I was pulled onto another call.
16:12:14 <eddiejenningsjr> I can do April 6th chair.
16:12:31 <Penguinpee> eddiejenningsjr: That's been sold. ;)
16:12:37 <mkonecny> #info Fedora 38 Beta RC 1.3 available for testing
16:12:37 <nirik> #info nirik is out on PTO tomorrow and monday. :) (But of course I will still be around some)
16:13:06 <Penguinpee> Make way for nirik-away...
16:13:18 <eddiejenningsjr> ah ok.  :)
16:13:33 <Penguinpee> eddiejenningsjr: I can put you down for the 13th.
16:13:41 <eddiejenningsjr> 13th is good
16:14:14 <Penguinpee> #info chair 2023-04-13 - eddijenningsjr
16:14:23 <Penguinpee> #info chair 2023-04-13 - eddiejenningsjr
16:14:57 <Penguinpee> #topic Oncall
16:14:57 <Penguinpee> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:14:57 <Penguinpee> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:14:57 <Penguinpee> ## .oncalltakeeu .oncalltakeus
16:14:57 <Penguinpee> #info eddiejennings is on call from 2023-03-02 to 2023-03-09
16:14:58 <Penguinpee> #info darknao is on call from 2023-03-10 to 2023-03-16
16:15:37 <Penguinpee> On call volunteers for week 14/15?
16:16:40 <Penguinpee> That's wrong. Week 12 (2023-03-17) and week 13 (2023-03-24)
16:17:40 <nirik> I'm gonna be traveling week 12/13. We may want to just put up a 'please file a ticket' oncall.
16:18:10 <eddiejenningsjr> I can take the 2023-03-17 week.
16:18:22 <Penguinpee> Sold to eddiejenningsjr. Thank you!
16:18:41 <Penguinpee> #info eddiejenningsjr is on call from 2023-03-17 to 2023-03-23
16:19:05 <Penguinpee> Anyone for week 13? If not I'll leave it open.
16:19:16 <darknao> .oncalltakeeu
16:19:16 <zodbot> darknao: Kneel before zod!
16:19:20 <eddiejenningsjr> Since I haven't had time to actually deal with tickets in a while, it's the least I can do :)
16:19:44 <Penguinpee> #info Summary of last week: (from current oncall eddiejenningsjr)
16:19:49 <Penguinpee> eddiejenningsjr: take it away
16:20:46 <eddiejenningsjr> Thankfully this week was quiet.  If there was a ping it wasn't made to oncall or me
16:21:15 <Penguinpee> Nice and quite. Our favorite flavor.
16:21:22 <Penguinpee> #topic Monitoring discussion [nirik]
16:21:22 <Penguinpee> #info https://nagios.fedoraproject.org/nagios
16:21:22 <Penguinpee> #info Go over existing out items and fix
16:22:10 <Penguinpee> nirik: What's burning at the moment?
16:22:34 <nirik> oh lets see...
16:23:00 <nirik> The usual things...
16:23:03 <nirik> and 3 new boxes I haven't yet installed are (not surprisingly) down. ;)
16:23:23 <nirik> also there's some copr machines that aren't allowing monitoring. we should ask them to fix that.
16:23:40 <mkonecny> nirik: We already have them in nagios?
16:23:43 <nirik> otherwise the usual down machines needing reboots, etc.
16:24:09 <nirik> yeah, I added them to dhcp and inventory and that adds them automagically to nagios.
16:24:13 <mkonecny> That's nice
16:24:47 <Penguinpee> Regarding Copr machines: Do we have a ticket for that already?
16:25:23 <nirik> no. I haven't had a chance. I was just going to talk to copr developers...
16:25:41 <mkonecny> I have a question about the bvmhost socket timeout, where are you checking what is happening?
16:25:44 <nirik> it's likely that aws needs to allow nagios into those, but might be firewall on the local instances or something
16:26:12 <Penguinpee> All right. As long as Copr people are aware and know what to check.
16:26:37 <nirik> The ones this morning? so thats basically saying the machine is down... so I usually try and ping/ssh to it to make sure...
16:26:55 <nirik> then, I look up machinename.mgmt ip. Thats the ip for the management interface for that machine.
16:27:10 <nirik> Then, I use ipmitool or login to the web console to look at whats on the console.
16:27:33 <Penguinpee> It seems we do not have any learning topics for future meetings. Suggestions are welcome. Maybe this week we do some backlog refinement instead?
16:27:49 <nirik> With the old a64 machines they just sometimes throw some oops and aren't responsive. You can't login to console even
16:27:52 <nirik> then, I use ipmitool to power cycle them (or web console)
16:28:16 <nirik> and watch console as they boot... in case they get stuck and then they should be back up. ;)
16:28:39 <mkonecny> Is there any guide I can follow? If I would like to check it by myself, ssh and ping was my first guess, but I didn't worked with ipmitool yet in Fedora
16:29:35 <nirik> not sure. Note that this needs you to have mgmt admin passwords.
16:29:42 <nirik> If we don't have something we should make something. ;)
16:30:24 <mkonecny> I would be glad if I could assist with this
16:30:40 <nirik> excellent. thanks. ;)
16:30:54 <Penguinpee> Anything else to discuss regarding Nagios?
16:31:20 <nirik> I really hope to work on docs more. I added it as a goal for me this year...
16:31:49 <Penguinpee> mkonecny++ for the assistance offer
16:31:49 <zodbot> Penguinpee: Karma for zlopez changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:32:23 <Penguinpee> #topic Revisit blocked tickets
16:32:23 <Penguinpee> #info Check if any blocked ticket is unblocked
16:32:23 <Penguinpee> #link https://pagure.io/fedora-infrastructure/issues?status=Open&tags=blocked&priority=0&close_status=
16:33:07 <Penguinpee> That looks good! Only four tickets blocked.
16:33:16 <nirik> there's progress on mailman, but it's not really unblocked fully
16:33:48 <nirik> otherwise nothing there off hand
16:33:54 <Penguinpee> Nice! Slow movement is also movement. Ask the slug!
16:34:13 <mkonecny> FYI: The CentOS has the same issue with mailman
16:34:22 <Penguinpee> #topic Fedora Infra backlog refinement
16:34:22 <Penguinpee> #info Refine oldest tickets on Fedora Infra tracker
16:34:22 <Penguinpee> #link https://pagure.io/fedora-infrastructure/issues?status=Open&order_key=last_updated&order=asc
16:34:45 <nirik> it's I think all in fedora now... but ideally we would want epel9. ;)
16:35:03 <Penguinpee> Do we just go through the backlog top down?
16:35:23 <mkonecny> EPEL9 should be our target
16:35:52 <Penguinpee> .ticket 10372
16:35:52 <mkonecny> Penguinpee: Yeah, just open the link and go them one by one
16:35:52 <zodbot> Penguinpee: Issue #10372: [proposal] Move Pagure (pagure.io and src.fp.o) to use OIDC for web login - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10372
16:35:53 * nirik has to go feed cats soon, they are circling. ;)
16:36:35 <nirik> this is still blocking on ipsilon supporting the scopes it needs
16:37:00 <mkonecny> We should probably start looking into that
16:37:12 <Penguinpee> Okay. So it doesn't need a nudge?
16:38:33 <nirik> I think abompard knows the most about ipsilon, but he's not had cycles to look at it I don't think.
16:38:57 <Penguinpee> Let's move on to the next ticket...
16:39:12 <Penguinpee> .ticket 7361
16:39:14 <zodbot> Penguinpee: Issue #7361: Signing infrastructure for aarch64 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/7361
16:39:16 <mkonecny> +1
16:40:25 <nirik> So, there's a big of discussion around this... possibly moving secure boot signing over to sigul, then we wouldn't need any special hardware. Other than that, we don't have hardware for it currently.
16:41:09 <Penguinpee> So, it's blocked on missing hardware for the time being. Should this be added to the ticket?
16:41:38 <mkonecny> I already commented with backlog refinement
16:41:46 <nirik> perhaps. well, I guess we do have hardware, but it's not in the right place...
16:41:51 <Penguinpee> I saw it. Thanks mkonecny!
16:41:56 <nirik> anyhow, it's not something we can do right now.
16:42:17 <nirik> next?
16:42:31 <Penguinpee> .ticket 10485
16:42:32 <zodbot> Penguinpee: Issue #10485: IAD2: certificate renewal stopped on updates.coreos.fedoraproject.org - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10485
16:43:10 <nirik> thats closed?
16:43:15 <mkonecny> This is already closed
16:43:29 <Penguinpee> Sorry! Typo!
16:43:46 <Penguinpee> .ticket 10458
16:43:47 <zodbot> Penguinpee: Issue #10458: Help me move my discourse bots to production? - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10458
16:44:11 <nirik> thats waiting on mattdm now...
16:44:16 <mkonecny> This is waiting for mattdm now
16:44:30 <nirik> however...
16:44:30 <mkonecny> nirik: Quicker :-D
16:44:46 <nirik> we now have a communishift. we could set him up a project there to help? or offer to...
16:45:22 <mkonecny> We can, will this help with the request?
16:45:23 <Penguinpee> Oh, mattdm loves playing around with new stuff. Will you leave a comment?
16:45:29 <nirik> That would be a place to run them short term and help configure them to move to staging.
16:46:54 <nirik> added
16:47:14 <Penguinpee> nirik: you beat me to it. ;)
16:47:34 <Penguinpee> One more:
16:47:42 <Penguinpee> .ticket 10963
16:47:43 <zodbot> Penguinpee: Issue #10963: Close out unused mailing lists - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10963
16:48:12 <mkonecny> Do we want to set is blocked by mailman update?
16:48:28 <Penguinpee> mkonecny: You are reading my mind...
16:48:32 <nirik> well, reading the additional comment, perhaps we should just close this?
16:48:42 <nirik> or try and get more info?
16:49:06 <mkonecny> We can, they are already closed
16:49:32 <nirik> right, but I don't know if there's still an outstanding bug or if it got cleaned up.
16:49:43 <nirik> closed/archived lists have posts set to 'reject' everything.
16:49:52 <Penguinpee> +1 for getting more info
16:49:58 <nirik> but I don't see what lists it was in there.
16:50:20 <mkonecny> It seems that the best course is to ask if this is still happening and wait for response
16:50:36 <nirik> I dimly remember this incident... but I don't know if there was still a bug or what
16:50:43 <nirik> or close and ask to reopen if it's still an issue?
16:51:03 <mkonecny> That is more preferable
16:51:06 <Penguinpee> I like that approach as well, nirik.
16:51:40 <Penguinpee> Do you or shall I, nirik?
16:51:44 <nirik> shall I? or one of you wants to?
16:51:54 <Penguinpee> Go ahead, nirik!
16:52:08 <mkonecny> I will close it
16:52:14 <Penguinpee> #topic Upcoming learning topics
16:52:22 <nirik> thanks zlopez
16:52:40 <Penguinpee> We don't have any learning topics listed for the coming weeks. Any suggestions or offers?
16:52:46 <nirik> I have a suggestion for a learning topic...
16:52:55 * Penguinpee listening
16:53:16 <nirik> learn how do contribute to our infra docs and we can fix one or more live. ;)
16:53:30 <eddiejenningsjr> That would be a good one.
16:53:37 <Penguinpee> That sounds like fun.
16:53:57 <aheath1992> +1 to that
16:53:58 <Penguinpee> nirik: Got a preferred date?
16:54:23 <nirik> I'd be happy to run that, or someone like darknao might know more about the current docs flow
16:54:24 <mkonecny> +1
16:54:52 <nirik> 30th? or in april? I'm traveling soon and don't want to do it then...
16:55:02 <darknao> yeah I might be able to help on that topic if needed
16:55:10 <Penguinpee> 30th sounds good to me
16:55:46 <Penguinpee> #info 2023-03-30 Learn how do contribute to our infra docs + live fixes [darknao, nirik]
16:56:07 <Penguinpee> We are approaching the top of the hour.
16:56:15 <Penguinpee> #topic Open Floor
16:57:12 <Penguinpee> More learning topic suggestions are welcome, of course.
16:57:38 <eddiejenningsjr> <work rant>Let it be known that the fact that dnf config supports installonly_limit= parameter makes it superior to apt</work rant>
16:57:38 <Penguinpee> Does anybody have anything for open floor?
16:57:48 <eddiejenningsjr> Had to let that out for open floor :)
16:58:16 <mkonecny> Eddie Jennings, Jr.: I really liked that you can just blacklist something in dnf config, so it doesn't get updated
16:58:42 * Penguinpee hears the reant echoing through the hallway - all the way to the Debian office... ;)
16:58:50 <Penguinpee> *rant
16:58:52 <eddiejenningsjr> Ha!
16:59:27 <Penguinpee> mkonecny: I made good use of it when Thunderbird was messing around with the UI again...
17:00:02 <Penguinpee> For whom the bell tolls...
17:00:04 <Penguinpee> #endmeeting