16:02:27 #startmeeting Infrastructure (2023-06-29) 16:02:27 Meeting started Thu Jun 29 16:02:27 2023 UTC. 16:02:27 This meeting is logged and archived in a public location. 16:02:27 The chair is phsmoura. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 16:02:27 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:02:27 The meeting name has been set to 'infrastructure_(2023-06-29)' 16:02:27 #meetingname infrastructure 16:02:27 The meeting name has been set to 'infrastructure' 16:02:27 #chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg 16:02:27 #info Agenda is at: https://board.net/p/fedora-infra 16:02:27 #info About our team: https://docs.fedoraproject.org/en-US/cpe/ 16:02:27 Current chairs: bodanel dtometzki jnsamyak lenkaseg nb nirik phsmoura zlopez 16:02:28 #info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra 16:02:32 #topic greetings! 16:02:52 .hello zlopez 16:02:53 mkonecny: zlopez 'Michal Konecny' 16:03:18 .hi 16:03:19 smooge: smooge 'Stephen Smoogen' 16:03:31 good morning everyone. Happy monday! 16:03:35 heelo 16:04:15 did my command to start the meeting worked? got disconnected after sending the msgs 16:04:46 It didi 16:05:02 ok, thanks :) 16:05:15 nirik: Monday? 16:05:35 it's always monday in the sysadmin world. ;) 16:05:39 4th monday of the week 16:05:40 .hi 16:05:40 darknao: darknao 'Francois Andrieu' 16:05:54 Lol! 16:06:02 .hello jnsamyak 16:06:03 jnsamyak: jnsamyak 'Samyak Jain' 16:06:24 It is also the 180th Monday of 2023. There are only 175 more Monday's until Christmas 16:07:04 #topic New folks introductions 16:07:04 #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves 16:07:04 #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted 16:07:14 do we have new people around? 16:08:17 moving on then 16:08:19 #topic Next chair 16:08:20 #info magic eight ball says: 16:08:20 #info chair 2023-07-06 - dtometzki 16:08:20 #info chair 2023-07-13 - lenkaseg 16:08:20 #info chair 2023-07-20 - ??? 16:08:20 #info chair 2023-07-27 - ??? 16:08:53 I can take the 2023-07-20 16:09:27 anyone wants to take 2023-07-27? 16:10:37 ok, I think we are fine for the following weeks here is what we have: 16:10:42 #info chair 2023-07-06 - dtometzki 16:10:43 #info chair 2023-07-13 - lenkaseg 16:10:43 #info chair 2023-07-20 - mkonecny 16:11:05 next topic 16:11:09 #topic announcements and information 16:11:09 #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0730 UTC in #centos-meeting 16:11:09 #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3 16:11:09 #info flock to fedora coming up in early aug (2nd-4th in cork, ie) https://flocktofedora.org 16:11:41 any other announcement? 16:12:44 #info next tuesday is a holiday in the US 16:12:52 I think there's a holiday in cz also next week 16:14:08 #info Next Wednesday and Thursday are holidays in CZ 16:15:16 #topic Oncall 16:15:31 #info https://fedoraproject.org/wiki/Infrastructure/Oncall 16:15:31 #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/ 16:15:31 ## .oncalltakeeu .oncalltakeus 16:15:31 #info nirik is on call from 2023-06-23 to 2023-06-29 16:15:31 #info darknao is on call from 2023-06-30 to 2023-07-06 16:15:32 #info eddiejenningsjr is on call from 2023-07-07 to 2023-07-13 16:15:34 #info ??? is on call from 2023-07-14 to 2023-07-20 16:15:35 #info ??? is on call from 2023-07-21 to 2023-07-27 16:15:54 anyone wants to take 2023-07-14 to 2023-07-20 or 2023-07-21 to 2023-07-27? 16:17:02 I'm here. I promise. I may or may not have been distracted by work :) 16:17:54 lets move on to summary then 16:18:04 #info Summary of last week: nirik 16:18:26 let me see. I think there was one ping... 16:19:04 yeah, it was a releng one about some eln issues. 16:19:08 Otherwise nothing much 16:19:41 .oncalltakeeu 16:19:41 darknao: Kneel before zod! 16:20:08 ok, do you have anything about monitoring nirik 16:20:12 #topic Monitoring discussion [nirik] 16:20:12 #info https://nagios.fedoraproject.org/nagios 16:20:12 #info Go over existing out items and fix 16:20:31 yeah, lets see. 16:21:22 There's an issue with out fedmsg monitoring script still... we need to adjust it to work with the python2 version as well as the python3 one 16:21:57 and there's 2 alerts on notifs-backend01 still... it's reading the right socket there, but not working for some reason. 16:22:10 I was hoping Zlopez might be able to look, because I have no idea on that one. ;) 16:22:11 Couldn't we update it to python 3 version everywhere? 16:22:43 no. badges is still python2, fedimg is still python2. busgateway is still python2. 16:22:59 unless we want to port those all to python3... which would be great, but... 16:23:01 Ok, that makes sense 16:23:36 I think we can just make the script check -3 first, then look for without it... 16:23:55 the notifs-backend01 alert is a timeout... UNKNOWN - ZMQ timeout. No message received in 20000 ms 16:24:20 That sounds reasonable for the script 16:24:39 I think aheath1992 was gonna do a pr, but we still had questions? I am not sure where we left it... 16:24:59 I will look into the notifs-backend issue tomorrow 16:25:27 that would be great! 16:25:46 also, we are continuing to get new fmn alerts. We might be able to tune that to not alert as much 16:26:15 I actually forgot about the alert :/ 16:26:40 (by 'new fmn' I mean the new notifications app, the one in openshift/new) 16:26:47 I can try and adjust it some 16:26:55 thats all I had on monitoring off hand. 16:28:23 ok, lets go through backlog refinement very quick and then move to learning topic 16:28:38 #topic Revisit blocked tickets 16:28:38 #info Check if any blocked ticket is unblocked 16:28:38 #link https://pagure.io/fedora-infrastructure/issues?status=Open&tags=blocked&priority=0&close_status= 16:29:19 all remain blocked? 16:29:27 all still blocked. There is hope for the aarch64 signing stuff on the horizon tho. 16:29:45 I'm currently working on https://pagure.io/fedora-infrastructure/issue/10372, hopefully I will move it forward 16:30:35 ok :) 16:30:38 #topic Fedora Infra backlog refinement 16:30:38 #info Refine oldest tickets on Fedora Infra tracker 16:30:38 #link https://pagure.io/fedora-infrastructure/issues?status=Open&order_key=last_updated&order=asc 16:31:06 I wonder... 16:31:12 any of those tickets we should review? 16:31:27 if we should move to doing this with releng tickets now...but I guess thats more for the releng meeting. 16:32:06 That is more for releng meeting 16:32:35 .ticket 7361 16:32:36 mkonecny: Issue #7361: Signing infrastructure for aarch64 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/7361 16:33:02 Are we still blocked on this? 16:33:23 there's some progress on this, but nothing really to say right now. Still needs more things to get into place. 16:34:19 .ticket 10241 16:34:20 mkonecny: Issue #10241: Port apps to OIDC - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10241 16:35:05 I'm working on the Pagure right now, but otherwise it's still the same 16:35:56 +1 16:36:15 .ticket 10845 16:36:16 mkonecny: Issue #10845: reply by email to Pagure issues bounce - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/10845 16:36:57 I'm not sure at all whats going on there... need someone to spend time making sure it's still happening, duplicate it, etc. 16:39:07 I could say I will, but... I haven't gotten to it yet. ;) 16:39:25 I'm not sure how I can help 16:40:46 any other ticket we should take a look? 16:40:50 well to see if it is still happening.. someone just needs to try and reply to a ticket in say fedora-infra and if they get a bounce or something? 16:41:04 yep 16:41:13 or any pagure.io project 16:41:35 trying with that ticket 16:41:36 if it's not and it's still happening to the reporter, there's something going on with their account or email or something. 16:41:49 will let you know and update ticket 16:42:12 smooge++ 16:42:14 thanks smooge 16:42:24 looks like with them 16:42:41 .ticket 11245 16:42:42 mkonecny: Issue #11245: DANE TLSA record for fedoraproject.org seems outdated - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/11245 16:43:30 ryan had a PR... but it needed adjustment 16:43:43 I can ping him about it next time I "see" him 16:44:08 basically it was fixed, but we want a nagios check to make sure we know when it breaks again. 16:44:23 PR in question https://pagure.io/fedora-infra/ansible/pull-request/1405 16:45:58 .ticket 8213 16:45:59 mkonecny: Issue #8213: fedmsg -> fedora-messaging migration tracker - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8213 16:46:56 There are few apps we want to get rid off 16:47:09 yep. 16:47:19 And there is some work being done on badges as well 16:47:20 nirik: mkonecny a higher priority issue on pagure may be why it stops logging to rsyslog every logrotate. Had to restart rsyslogd again 16:47:28 I thought the wiki was moved, but it turns out it was not. 16:47:52 smooge: ☹️ 16:48:30 smooge: Doesn't sound good 16:49:18 should we move to learning topic? Im afraid we are running out of time for it :/ 16:50:11 Do we have learning topic for today? 16:50:19 yep 16:50:40 Oh, we should have skipped the backlog refinement than 16:50:51 ok, eddiejennings are you around? 16:51:08 I am. Thank you for ping 16:51:28 I was pulled away once more. 16:51:33 stage is yours 16:51:36 #topic Learning topic 16:51:37 #info Ansible install options [eddiejennings] 2023-06-29 16:52:06 So this will be a little preview of a larger discussion, since we have about 10 minutes left. 16:52:38 There are four may ways of installing Ansible on the server which will serve as your control node. 16:54:30 One method is using the installer for Red Hat's Ansible Automation Platform. This is the paid product from Red Hat, which unfortunately I've never been able to use. 16:55:09 The AWX project is the upstream project of Ansible Automation Platform. Installation of it requires using Kubernets -- which unfortunately I haven't had experience. 16:55:58 The next methods are the ones with which I have experience. 16:56:38 I lied. I said four methods, but there are three. The second method is using a distribution's package manager to install either the ansible package or the ansible-core package. 16:57:02 The final method is installing either ansible or ansible-core using pip, which installs from PyPI. 16:57:55 The next time we have a larger learning discussion block, I'll go into details about the ansible and ansible-core packages. Their differences, and when one may choose to use PyPI instead of a distro's repository and vice-versa. 16:58:23 So not much information, but a little preview of what I'll discuss when we have more time :) 16:59:38 should we reschedule that learning topic for july 13rd? 17:00:04 or next week? 17:00:12 I'm fine with either. 17:00:50 I'd probably need 20-25 minutes to give it justice. 17:01:18 what do you all think? next week or july 13rd? before we end this meeting 17:01:45 Next week 17:02:16 ok, next week it is then 17:02:29 thanks everyone :) 17:02:38 #endmeeting