16:01:55 <eddiejennings> #startmeeting Infrastructure (2023-09-07)
16:01:55 <zodbot> Meeting started Thu Sep  7 16:01:55 2023 UTC.
16:01:55 <zodbot> This meeting is logged and archived in a public location.
16:01:55 <zodbot> The chair is eddiejennings. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:01:55 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:01:55 <zodbot> The meeting name has been set to 'infrastructure_(2023-09-07)'
16:02:09 <eddiejennings> #meetingname infrastructure
16:02:09 <zodbot> The meeting name has been set to 'infrastructure'
16:02:09 <eddiejennings> #chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg patrikp
16:02:09 <eddiejennings> #info Agenda is at: https://board.net/p/fedora-infra
16:02:09 <zodbot> Current chairs: bodanel dtometzki eddiejennings jnsamyak lenkaseg nb nirik patrikp zlopez
16:02:09 <eddiejennings> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:02:09 <eddiejennings> #info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
16:02:09 <eddiejennings> #topic greetings!
16:02:14 <nirik> morning party people.
16:02:30 <patrikp_> .hi
16:02:30 <eddiejennings> I may or may not be ending a Fantasy Football draft at work :)
16:02:30 <zodbot> patrikp_: Something blew up, please try again
16:02:34 <zodbot> patrikp_: An error has occurred and has been logged. Please contact this bot's administrator for more information.
16:02:39 <eddiejennings> .hi
16:02:40 <zodbot> eddiejennings: Something blew up, please try again
16:02:43 <zodbot> eddiejennings: An error has occurred and has been logged. Please contact this bot's administrator for more information.
16:02:49 <zlopez> .hi
16:02:50 <zodbot> zlopez: Something blew up, please try again
16:02:53 <zodbot> zlopez: An error has occurred and has been logged. Please contact this bot's administrator for more information.
16:03:06 <eddiejennings> Welcome to the party everyone!
16:03:30 <jednorozec> .hello humaton
16:03:31 <zodbot> jednorozec: Something blew up, please try again
16:03:34 <zodbot> jednorozec: An error has occurred and has been logged. Please contact this bot's administrator for more information.
16:05:40 <nirik> zodbot: reload Fedora
16:05:40 <zodbot> nirik: Kneel before zod!
16:05:40 <eddiejennings> #topic New folks introductions
16:05:41 <eddiejennings> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:05:41 <eddiejennings> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:05:41 <eddiejennings> Do we have any new folks today?  If so, let yourself be known and introduce yourself!
16:06:22 <eddiejennings> Hearing none, let's move along
16:06:38 <eddiejennings> #topic Next chair
16:06:39 <eddiejennings> #info magic eight ball says:
16:06:39 <eddiejennings> #info chair 2023-09-07 eddiejennings
16:06:39 <eddiejennings> #info chair 2023-09-14 nirik
16:06:39 <eddiejennings> #info chair 2023-09-21 phsmoura
16:06:39 <eddiejennings> #info chair 2023-09-28 pcreech
16:07:04 <eddiejennings> Looks like we're set through September for meeting chairs.  Unless someone's interested in an October slot, we'll move along.
16:09:07 <eddiejennings> For the record, chairing is a fun, easy way to be involved.  I'm allowed to do it, so anyone can :D
16:09:24 <eddiejennings> And on to everyone's favorite topic . . .
16:09:35 <eddiejennings> #topic Oncall
16:09:35 <eddiejennings> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:09:35 <eddiejennings> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:09:35 <eddiejennings> ## .oncalltakeeu .oncalltakeus
16:09:35 <eddiejennings> #info darknao  is on call from 2023-09-01 to 2023-09-07
16:09:35 <eddiejennings> #info nirik  is on call from 2023-09-08 to 2023-09-14
16:09:37 <eddiejennings> #info ???  is on call from 2023-09-15 to 2023-09-21
16:09:39 <eddiejennings> #info pcreech  is on call from 2023-09-22 to 2023-09-28
16:10:01 <eddiejennings> We're covered for this week's oncall. Are there any volunteers for next week?
16:10:06 <zlopez> I can take the week from 15-21
16:10:22 <eddiejennings> sold to zlopez!
16:10:37 <nirik> .oncalltakus
16:10:41 <eddiejennings> #info zlopez is on call from 2023-09-15 to 2023-09-21
16:10:43 <zlopez> \o/
16:10:50 <nirik> .oncalltakeus
16:10:50 <zodbot> nirik: Kneel before zod!
16:11:09 <nirik> note that things are kinda wacky right now with the bridge being gone.
16:11:14 <eddiejennings> Like the meeting chair, we're covered through September.
16:11:39 <eddiejennings> Yeah.  I had to remind myself to keep IRC open when I was on a computer.
16:12:30 <eddiejennings> #info Summary of last week: (from current oncall )
16:12:35 <eddiejennings> darknao the floor is your's!
16:14:37 <eddiejennings> Is darknao with us today?
16:15:13 <nirik> might not be around.
16:16:18 <eddiejennings> Seems not.  I assume it was a quiet week.  Likely so because of the freeze.
16:16:30 <nirik> yeah, not too crazy
16:17:00 <eddiejennings> Rolling along. . .
16:17:15 <eddiejennings> #topic Monitoring discussion [nirik]
16:17:15 <eddiejennings> #info https://nagios.fedoraproject.org/nagios
16:17:15 <eddiejennings> #info Go over existing out items and fix
16:17:20 <eddiejennings> nirik, take it away :D
16:17:22 <nirik> lets see
16:17:47 <nirik> pretty normal. a few small issues we haven't fixed yet... but nothing too big.
16:18:20 <nirik> we can probibly just move on
16:18:43 <eddiejennings> Sounds good and noted.  We're flying through today :)
16:19:28 <eddiejennings> Last week we did some brainstorming of learning topics, so we'll do ticket backlog this week.
16:19:50 <eddiejennings> Before we do, do we want to pick a topic for next meeting?
16:19:57 <zlopez> What about announcements?
16:20:02 <eddiejennings> !
16:20:05 <eddiejennings> You're right
16:20:11 <eddiejennings> I misread my script :(
16:20:40 <eddiejennings> Shifting into reverse momentarily...
16:20:42 <eddiejennings> #topic announcements and information
16:20:42 <eddiejennings> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0730 UTC in #centos-meeting
16:20:42 <eddiejennings> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:20:42 <eddiejennings> #info we are in f39 beta freeze
16:20:51 <eddiejennings> thanks zlopez for keeping me on track
16:21:00 <eddiejennings> Do we have any new announcements?
16:22:35 <nirik> we are no go for beta this week. ;) next go/nogo is next thursday
16:23:25 <eddiejennings> But the freeze is still in effect right?
16:23:46 <zlopez> Yes, till the Beta is released
16:24:03 <eddiejennings> #info next go/nogo for f39 beta is September 14
16:24:53 <nirik> yes
16:25:03 <eddiejennings> Any other announcements?
16:25:25 <nirik> nope
16:25:46 <eddiejennings> Ok.  Shifting back into a forward gear...
16:26:24 <smooge> make sure to skip 2nd, we haven't fixed that gear
16:26:45 <eddiejennings> Starting at line 132 on https://board.net/p/fedora-infra, you can see the topic choices we came up with the brainstorm.  Which topic would we want to do next week?
16:27:23 <eddiejennings> smooge my father had a van a few years used for bringing supplies to vending machines, and he had to do just that, shift from 1 to 3
16:27:52 <eddiejennings> Personally, I'd like to hear about the storage setup / backups
16:27:54 <nirik> I could do koji intro/session next time? or pungi info from jednorozec would be nice
16:28:12 <jednorozec> pungi might also be multipart
16:28:41 <nirik> I could do backups, but I plan to rework how they work... so might be good to wait and talk about the new setup? or I suppose I could talk about exisintg and do another one later after the rework
16:28:57 <eddiejennings> +1 for waiting on backups since it's being reworked
16:28:58 <smooge> i was thinking it might be good to do backups for two reasons
16:29:16 <smooge> 1. talking about how it is will help you show/see yourself the problems
16:29:27 <smooge> 2. it may also help figure out any problems with the new one
16:29:38 <eddiejennings> smooge has valid points we well
16:29:54 <smooge> that way you get to think out loud and we can be rubber ducks
16:30:01 <smooge> quack
16:30:11 <nirik> sure... happy to share...
16:30:13 <jednorozec> quack quack
16:30:26 <eddiejennings> Perhaps we plan 9/14 - backups, 9/28 - Pungi Part 1, 10/12 - Pungi Part 2
16:30:39 <nirik> gonna be a lot of typing for me next week, since I am running the meeting too, but thats fine.
16:31:05 <smooge> sorry I didn't think about that
16:31:18 <eddiejennings> Maybe hexchat < ./backup_spiel.sh :)
16:31:19 <jednorozec> so if nirik is running next week
16:31:22 <jednorozec> let me du pungi
16:31:50 <zlopez> I can talk about registry.fp.o and the migration to quay.io
16:32:28 <eddiejennings> When do you think the reworking of backups will happen, nirik?
16:32:49 <nirik> well, hard to say... needs the fires to stop burning for a while so I can get to it. ;)
16:33:14 <nirik> but I can talk about current setup anytime
16:33:19 <eddiejennings> how about this?
16:34:12 <eddiejennings> 9/14 zlopez and quay.io, 9/28 nirik on current backups, then the two October slots be for jednorozec for Pungi
16:34:26 <smooge> sounds good to me
16:34:31 <jednorozec> ack
16:34:38 <zlopez> That sounds good :-)
16:34:57 <eddiejennings> That way nirik doesn't break his keyboard typing :)
16:35:11 <nirik> sure. sounds good.
16:35:37 <eddiejennings> Excellent.  I'll update the agenda.  In the meantime
16:35:39 <eddiejennings> #topic Revisit blocked tickets
16:35:39 <eddiejennings> #info Check if any blocked ticket is unblocked
16:35:39 <eddiejennings> #link https://pagure.io/fedora-infrastructure/issues?status=Open&tags=blocked&priority=0&close_status=
16:36:24 <nirik> some progress on some but nothing unblocked really
16:38:09 <eddiejennings> Noted.
16:38:17 <eddiejennings> #topic Fedora Infra backlog refinement
16:38:17 <eddiejennings> #info Refine oldest tickets on Fedora Infra tracker
16:38:18 <eddiejennings> #link https://pagure.io/fedora-infrastructure/issues?status=Open&order_key=last_updated&order=asc
16:40:43 <eddiejennings> Anything in the backlog of note?
16:41:09 <zlopez> .ticket 10358
16:41:11 <nirik> I wonder if we can just close 11331
16:41:12 <zodbot> zlopez: Issue #10358: IPA backup fails on ipa03 and ipa02.stg - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10358
16:41:31 <nirik> this is blocked on me upgrading our clusters to rhel9, which I keep not getting to. ;(
16:42:17 <zlopez> How are you doing that?
16:42:36 <zlopez> I can probably upgrade something
16:42:37 <nirik> you mean not doing it? ;)
16:42:42 <nirik> it's not simple
16:43:16 <nirik> you have to install a new rhel9 one, add it to replication from the other ones, wait for it to replicate, then replace one by one the other ones.
16:43:59 <zlopez> I assume you need access to management console
16:44:34 <nirik> no, it likely needs command line ipa commands and might need out ansible playbooks that deploy adjusted (carefully)
16:44:40 <nirik> I'm looking for the doc
16:45:31 <zlopez> I talked about the installation of RHEL9
16:46:03 <nirik> oh, no installing a rhel9 vm should all be done via ansible...
16:46:24 <zlopez> That sounds convenient :-)
16:46:54 <nirik> but the thing is that it will also configure things... but might now be wrong for rhel9.
16:47:03 <nirik> I can't find the doc. I will dig more and look for it.
16:48:12 <nirik> ah ha, there it is!
16:48:16 <zlopez> If the doc will be available I can look into that
16:48:29 <nirik> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/migrating_to_identity_management_on_rhel_9/index
16:48:55 <nirik> ok, that would be great, but... make sure we have a backup of the data and be careful. If it's broken, no one can login in staging.
16:49:18 <zlopez> I will add the link to the ticket, so it could be easily found :-)
16:49:47 <nirik> it's possible it will all just work to add a new rhel9 ipa04.stg... but I bet it will be broken somehow. ;)
16:50:45 <zlopez> Maybe best time to look at the SOP for IPA and see if there is any backup procedure :-)
16:50:48 <nirik> I suppose we could also just manually configure the rhel9 one... then reinstall the other rhel8 ones and then at the end delete the temp rhel9 one...
16:51:05 <nirik> there is and we do make backups. ;) (Thats what the ticket was orig about)
16:51:41 <zlopez> I'm working on the docs most of the time for last few days, so trying to improve them
16:51:49 <nirik> great!
16:51:50 <smooge> zlopez++
16:52:14 <eddiejennings> Good discussion.
16:52:59 <eddiejennings> Since we're in our last 10 minutes, any other backlog tickets of note, or shall we proceed to end-meeting things?
16:53:14 <zlopez> .ticket 11331
16:53:15 <zodbot> zlopez: Issue #11331: Syncing of mirror log files to log01 broke around May 5th-10th, bad DNF countme data - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/11331
16:53:24 <zlopez> This was mentioned by nirik before
16:54:16 <zlopez> I think we can close that as this only happened for few days few months ago
16:55:00 <nirik> yeah, and it's unlikely we can see what happened then or why...
16:56:24 <zlopez> Let's close it then
16:56:47 <zlopez> As Insufficient data
16:56:52 <nirik> +1
16:57:30 <eddiejennings> Any other tickets for our final 3 minutes?
16:58:03 <zlopez> We can go to Open Floor
16:58:17 <eddiejennings> To have it on the record. . .
16:58:18 <eddiejennings> #topic Upcoming learning topics
16:58:18 <eddiejennings> ## info registry.fp.o and the migration to quay.io [zlopez] on 2023-09-14
16:58:18 <eddiejennings> ## info current storage setup and backups [nirik] on 2023-09-28
16:58:19 <eddiejennings> ## info pungi intro/crash course [jednorozec] on 2023-10-12 and 2023-10-26
16:58:27 <eddiejennings> and now
16:58:28 <eddiejennings> #topic Open Floor
16:58:42 <eddiejennings> Anything for the good of us all? :)
17:00:14 <eddiejennings> Hearning none; thus endeth the meeting.
17:00:18 <eddiejennings> Thanks for coming everyone!
17:00:20 <eddiejennings> #endmeeting