2026-03-26 17:01:41 <@zlopez:fedora.im> !startmeeting Infrastructure (2026-03-26)
2026-03-26 17:01:41 <@meetbot:fedora.im> Meeting started at 2026-03-26 17:01:41 UTC
2026-03-26 17:01:41 <@meetbot:fedora.im> The Meeting name is 'Infrastructure (2026-03-26)'
2026-03-26 17:01:50 <@zlopez:fedora.im> !info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
2026-03-26 17:01:50 <@zlopez:fedora.im> !topic Hola y bienvenido
2026-03-26 17:01:50 <@zlopez:fedora.im> !info About our team: https://docs.fedoraproject.org/en-US/cle/
2026-03-26 17:01:50 <@zlopez:fedora.im> !info Agenda is at: https://board.net/p/fedora-infra
2026-03-26 17:01:50 <@zlopez:fedora.im> !chair @nirik:matrix.scrye.com @zlopez:fedora.im  @jnsamyak:matrix.org  @james:fedora.im @gwmngilfen:fedora.im @patrikp:matrix.org
2026-03-26 17:01:50 <@zlopez:fedora.im> !meetingname infrastructure
2026-03-26 17:01:51 <@meetbot:fedora.im> The Meeting Name is now infrastructure
2026-03-26 17:01:58 <@zlopez:fedora.im> !hi
2026-03-26 17:01:59 <@zodbot:fedora.im> Michal Konecny (zlopez)
2026-03-26 17:02:01 <@gwmngilfen:fedora.im> !hi
2026-03-26 17:02:02 <@zodbot:fedora.im> Greg Sutcliffe (gwmngilfen) - he / him / his
2026-03-26 17:02:09 <@zlopez:fedora.im> Welcome everyone to today meeting
2026-03-26 17:03:47 <@patrikp:matrix.org> Hi. 👋
2026-03-26 17:04:20 <@ahmedalmeleh:matrix.org> !hi
2026-03-26 17:04:21 <@zodbot:fedora.im> Ahmed Almeleh (ahmedalmeleh) - he / him / his
2026-03-26 17:05:04 <@james:fedora.im> !hi
2026-03-26 17:05:07 <@zlopez:fedora.im> !info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/
2026-03-26 17:05:07 <@zlopez:fedora.im> !topic New folks introductions
2026-03-26 17:05:07 <@zlopez:fedora.im> !info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
2026-03-26 17:05:08 <@zodbot:fedora.im> James Antill (james) - he / him / his
2026-03-26 17:05:11 <@zlopez:fedora.im> So anybody new here?
2026-03-26 17:07:13 <@ahmedalmeleh:matrix.org> I've done qa here but not really infrastructure 
2026-03-26 17:07:32 <@zlopez:fedora.im> What brings you to infra?
2026-03-26 17:08:04 <@ahmedalmeleh:matrix.org> I have a passion for networking and love tinkering and a higher national diploma in computing 
2026-03-26 17:08:45 <@zlopez:fedora.im> Welcome in Fedora infra then
2026-03-26 17:09:18 <@ahmedalmeleh:matrix.org> Thanks 
2026-03-26 17:09:19 <@patrikp:matrix.org> Welcome!
2026-03-26 17:09:47 <@zlopez:fedora.im> Feel free to reach to us on #admin:fedoraproject.org if you need any help
2026-03-26 17:10:50 <@ahmedalmeleh:matrix.org> What is there that is good to do that doesn't involve coding but like systems in infa?
2026-03-26 17:11:13 <@ahmedalmeleh:matrix.org> I've heard about the monitoring and stuff
2026-03-26 17:12:00 <@gwmngilfen:fedora.im> Zabbix is entirely configured from Ansible, so if you want to avoid coding you might not ernjoy that, but you're welcome to look. help with docs/sops is always welcome too
2026-03-26 17:12:12 <@gwmngilfen:fedora.im> Zabbix (our monitoring) is entirely configured from Ansible, so if you want to avoid coding you might not ernjoy that, but you're welcome to look. help with docs/sops is always welcome too
2026-03-26 17:12:22 <@gwmngilfen:fedora.im> Zabbix (our monitoring) is entirely configured from Ansible, so if you want to avoid coding you might not enjoy that, but you're welcome to look. help with docs/sops is always welcome too
2026-03-26 17:13:22 <@zlopez:fedora.im> We can give you read fi-apprentice access and you could troubleshoot some of the issues we have, that would help as well 🙂
2026-03-26 17:13:50 <@gwmngilfen:fedora.im> also true. debugging *why* an alert is going off is frequently helpful 🙂
2026-03-26 17:14:37 <@ahmedalmeleh:matrix.org> What is sops, I've heard of ansible which is scripting and similar to stuff like a incus file for nano or podfile
2026-03-26 17:15:18 <@gwmngilfen:fedora.im> sops = standard operating procedures. https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/#_standard_operating_procedures
2026-03-26 17:15:31 <@ahmedalmeleh:matrix.org> What is sops, I've heard of ansible which is scripting and similar to stuff like an incus file script I used with nano as easg config for vms or podfile
2026-03-26 17:16:04 <@gwmngilfen:fedora.im> Ansible is a configuration management tool, see docs.ansible.com
2026-03-26 17:16:54 <@ahmedalmeleh:matrix.org> So more advanced then a manual nano script then or docker file?
2026-03-26 17:17:26 <@zlopez:fedora.im> We should move on to next topic, Ahmed Almeleh there will be an open floor at the end, where you can ask some questions
2026-03-26 17:17:55 <@zlopez:fedora.im> !info magic eight ball says:
2026-03-26 17:17:55 <@zlopez:fedora.im> !topic Next chair
2026-03-26 17:17:55 <@zlopez:fedora.im> !info chair 2026-04-02 - ???
2026-03-26 17:17:55 <@zlopez:fedora.im> !info chair 2026-03-26 - mkonecny
2026-03-26 17:18:08 <@zlopez:fedora.im> Do we have any volunteers to run next meeting?
2026-03-26 17:18:44 <@patrikp:matrix.org> I would take it but high likelihood I won't be here.
2026-03-26 17:19:09 <@gwmngilfen:fedora.im> usual answer for me, I can't stay til the end
2026-03-26 17:19:21 <@gwmngilfen:fedora.im> (n fact I have to go even earliet today, in a few mins 😕)
2026-03-26 17:19:38 <@zlopez:fedora.im> No problem
2026-03-26 17:20:01 <@zlopez:fedora.im> It would be nice to have volunteer at least for next week
2026-03-26 17:20:52 <@gwmngilfen:fedora.im> I can certainly start it next week. if there's more than me by hafway, someone else can take over 😉
2026-03-26 17:20:58 <@gwmngilfen:fedora.im> I can certainly start it next week. if there's more than me by halfway, someone else can take over 😉
2026-03-26 17:21:00 <@zlopez:fedora.im> So let me do it then
2026-03-26 17:21:06 <@gwmngilfen:fedora.im> if not... well that also solves the problem
2026-03-26 17:21:17 <@zlopez:fedora.im> !info chair 2026-04-02 - mkonecny
2026-03-26 17:21:35 <@zlopez:fedora.im> And what about the week after that
2026-03-26 17:21:36 <@zlopez:fedora.im> !info chair 2026-04-09 - ???
2026-03-26 17:23:39 <@patrikp:matrix.org> Wait, I got my dates mixed up.
2026-03-26 17:23:52 <@patrikp:matrix.org> I can do next week and I won't be here the one after that.
2026-03-26 17:24:12 <@zlopez:fedora.im> Do you want to switch then?
2026-03-26 17:24:28 <@patrikp:matrix.org> I can do next week, sure.
2026-03-26 17:24:46 <@zlopez:fedora.im> !info chair 2026-04-02 - patrikp
2026-03-26 17:24:46 <@zlopez:fedora.im> !info chair 2026-04-09 - mkonecny
2026-03-26 17:24:57 <@zlopez:fedora.im> That looks good, let us continue
2026-03-26 17:25:05 <@zlopez:fedora.im> !topic announcements and information
2026-03-26 17:25:05 <@zlopez:fedora.im> !info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1900 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2026-03-26 17:25:58 <@zlopez:fedora.im> !info F44 final freeze starts next week on Tuesday 31st March
2026-03-26 17:26:53 <@zlopez:fedora.im> !info There is a new way to set up monitoring for packages https://the-new-hotness.readthedocs.io/en/stable/user-guide.html#notifications-settings
2026-03-26 17:28:02 <@gwmngilfen:fedora.im> - We also have PSQL monitoring (net new, wasn't in Nagios) which needs some finetuning (lots of alerts about DB locks) but otherwise is looking good
2026-03-26 17:28:02 <@gwmngilfen:fedora.im> I have to run, so a quick monitoring update from me
2026-03-26 17:28:02 <@gwmngilfen:fedora.im> - lots of new  checks on Zabbix, but notably the DNS ping & management IP monitoring has been moved from Nagios to Zabbix now
2026-03-26 17:28:02 <@gwmngilfen:fedora.im> - I've also reduced the sensitivity of the Load avg checks that were triggering all the time - now it has to be high for 15mins instead of 5mins. There's an open ELN queue alert I'm not sure about, the rest seems pretty sane
2026-03-26 17:28:07 <@gwmngilfen:fedora.im> catch you later folks 😉
2026-03-26 17:29:25 <@zlopez:fedora.im> And that brings us to monitoring
2026-03-26 17:29:28 <@zlopez:fedora.im> !topic Monitoring discussion [nirik]
2026-03-26 17:29:28 <@zlopez:fedora.im> !info https://nagios.fedoraproject.org/nagios & https://zabbix.fedoraproject.org (top 100 triggers: https://zabbix.fedoraproject.org/zabbix.php?action=toptriggers.list)
2026-03-26 17:29:28 <@zlopez:fedora.im> !info Go over existing items and fix them
2026-03-26 17:30:53 <@zlopez:fedora.im> I can only add there there were postfix alert all over the place related to lmdb change, I fixed all I found, but it's possible that some of them will happen later especially on machines where the amount of sent messages is really low
2026-03-26 17:31:05 <@zlopez:fedora.im> I can only add that there were postfix alert all over the place related to lmdb change, I fixed all I found, but it's possible that some of them will happen later especially on machines where the amount of sent messages is really low
2026-03-26 17:31:13 <@zlopez:fedora.im> I can only add that there were postfix alerts all over the place related to lmdb change, I fixed all I found, but it's possible that some of them will happen later especially on machines where the amount of sent messages is really low
2026-03-26 17:32:33 <@zlopez:fedora.im> I don't see anything else that weren't already mentioned by Gwmngilfen, so let's move on
2026-03-26 17:32:51 <@zlopez:fedora.im> !link https://forge.fedoraproject.org/infra/tickets/issues?q=&type=all&sort=leastupdate&state=open&labels=&milestone=0&project=0&assignee=0&poster=0
2026-03-26 17:32:51 <@zlopez:fedora.im> !topic Fedora Infra backlog refinement
2026-03-26 17:32:51 <@zlopez:fedora.im> !info Refine oldest tickets on Fedora Infra tracker
2026-03-26 17:33:13 <@zlopez:fedora.im> So let's go through some tickets and let's see if we can add something to them
2026-03-26 17:33:29 <@zlopez:fedora.im> !link https://forge.fedoraproject.org/infra/tickets/issues/12058
2026-03-26 17:34:42 <@zlopez:fedora.im> This is waiting for davdunc, but he probably didn't have time to check it out or missed it. I will try to ping him again
2026-03-26 17:35:12 <@zlopez:fedora.im> !link https://forge.fedoraproject.org/infra/tickets/issues/12761
2026-03-26 17:36:30 <@zlopez:fedora.im> I see that there was some question from nirik, which got already answered, so it's now waiting on him to move it forward
2026-03-26 17:37:02 <@zlopez:fedora.im> !link https://forge.fedoraproject.org/infra/tickets/issues/11825
2026-03-26 17:38:13 <@zlopez:fedora.im> Hm, I must have missed the last update from David Kirwan here
2026-03-26 17:38:22 <@zlopez:fedora.im> This is still something we would like to do
2026-03-26 17:40:00 <@zlopez:fedora.im> Responded in the ticket
2026-03-26 17:40:34 <@zlopez:fedora.im> Let me do one more ticket and then switch to open floor as there isn't much engagement here anyway
2026-03-26 17:40:49 <@zlopez:fedora.im> !link https://forge.fedoraproject.org/infra/tickets/issues/12234
2026-03-26 17:42:02 <@zlopez:fedora.im> I assume this is now waiting for decision which way to go on the notification implementation
2026-03-26 17:43:32 <@zlopez:fedora.im> Responded in the ticket
2026-03-26 17:43:35 <@patrikp:matrix.org> Looks like it.
2026-03-26 17:43:43 <@patrikp:matrix.org> I don't have much to say on any of these...
2026-03-26 17:43:51 <@zlopez:fedora.im> !topic Open Floor
2026-03-26 17:44:13 <@zlopez:fedora.im> Let's do the open floor, this is the place to talk about anything infra related
2026-03-26 17:44:41 <@zlopez:fedora.im> If there will be nothing, I will close the meeting in 5 minutes
2026-03-26 17:46:33 <@patrikp:matrix.org> Looks like it's a wrap. Have a good day everyone and thanks for running the meeting Michal.
2026-03-26 17:46:43 <@ahmedalmeleh:matrix.org> Thanks
2026-03-26 17:47:21 <@zlopez:fedora.im> In that case let me end the meeting
2026-03-26 17:47:28 <@zlopez:fedora.im> !info Thank you all for coming!
2026-03-26 17:47:28 <@zlopez:fedora.im> !endmeeting