#meeting-3:fedoraproject.org log

<@zlopez:fedora.im>

17:01:41

!startmeeting Infrastructure (2026-03-26)

<@meetbot:fedora.im>

17:01:41

Meeting started at 2026-03-26 17:01:41 UTC

<@meetbot:fedora.im>

17:01:41

The Meeting name is 'Infrastructure (2026-03-26)'

<@zlopez:fedora.im>

17:01:50

!info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra

<@zlopez:fedora.im>

17:01:50

!topic Hola y bienvenido

<@zlopez:fedora.im>

17:01:50

!info About our team: https://docs.fedoraproject.org/en-US/cle/

<@zlopez:fedora.im>

17:01:50

!info Agenda is at: https://board.net/p/fedora-infra

<@zlopez:fedora.im>

17:01:50

!chair @nirik:matrix.scrye.com @zlopez:fedora.im @jnsamyak:matrix.org @james:fedora.im @gwmngilfen:fedora.im @patrikp:matrix.org

<@zlopez:fedora.im>

17:01:50

!meetingname infrastructure

<@meetbot:fedora.im>

17:01:51

The Meeting Name is now infrastructure

<@zlopez:fedora.im>

17:01:58

!hi

<@zodbot:fedora.im>

17:01:59

Michal Konecny (zlopez)

<@gwmngilfen:fedora.im>

17:02:01

!hi

<@zodbot:fedora.im>

17:02:02

Greg Sutcliffe (gwmngilfen) - he / him / his

<@zlopez:fedora.im>

17:02:09

Welcome everyone to today meeting

<@patrikp:matrix.org>

17:03:47

Hi. 👋

<@ahmedalmeleh:matrix.org>

17:04:20

!hi

<@zodbot:fedora.im>

17:04:21

Ahmed Almeleh (ahmedalmeleh) - he / him / his

<@james:fedora.im>

17:05:04

!hi

<@zlopez:fedora.im>

17:05:07

!info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/

<@zlopez:fedora.im>

17:05:07

!topic New folks introductions

<@zlopez:fedora.im>

17:05:07

!info This is a place where people who are interested in Fedora Infrastructure can introduce themselves

<@zodbot:fedora.im>

17:05:08

James Antill (james) - he / him / his

<@zlopez:fedora.im>

17:05:11

So anybody new here?

<@ahmedalmeleh:matrix.org>

17:07:13

I've done qa here but not really infrastructure

<@zlopez:fedora.im>

17:07:32

What brings you to infra?

<@ahmedalmeleh:matrix.org>

17:08:04

I have a passion for networking and love tinkering and a higher national diploma in computing

<@zlopez:fedora.im>

17:08:45

Welcome in Fedora infra then

<@ahmedalmeleh:matrix.org>

17:09:18

Thanks

<@patrikp:matrix.org>

17:09:19

Welcome!

<@zlopez:fedora.im>

17:09:47

Feel free to reach to us on #admin:fedoraproject.org if you need any help

<@ahmedalmeleh:matrix.org>

17:10:50

What is there that is good to do that doesn't involve coding but like systems in infa?

<@ahmedalmeleh:matrix.org>

17:11:13

I've heard about the monitoring and stuff

<@gwmngilfen:fedora.im>

17:12:00

Zabbix is entirely configured from Ansible, so if you want to avoid coding you might not ernjoy that, but you're welcome to look. help with docs/sops is always welcome too

<@gwmngilfen:fedora.im>

17:12:12

Zabbix (our monitoring) is entirely configured from Ansible, so if you want to avoid coding you might not ernjoy that, but you're welcome to look. help with docs/sops is always welcome too

<@gwmngilfen:fedora.im>

17:12:22

Zabbix (our monitoring) is entirely configured from Ansible, so if you want to avoid coding you might not enjoy that, but you're welcome to look. help with docs/sops is always welcome too

<@zlopez:fedora.im>

17:13:22

We can give you read fi-apprentice access and you could troubleshoot some of the issues we have, that would help as well 🙂

<@gwmngilfen:fedora.im>

17:13:50

also true. debugging *why* an alert is going off is frequently helpful 🙂

<@ahmedalmeleh:matrix.org>

17:14:37

What is sops, I've heard of ansible which is scripting and similar to stuff like a incus file for nano or podfile

<@gwmngilfen:fedora.im>

17:15:18

sops = standard operating procedures. https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/#_standard_operating_procedures

<@ahmedalmeleh:matrix.org>

17:15:31

What is sops, I've heard of ansible which is scripting and similar to stuff like an incus file script I used with nano as easg config for vms or podfile

<@gwmngilfen:fedora.im>

17:16:04

Ansible is a configuration management tool, see docs.ansible.com

<@ahmedalmeleh:matrix.org>

17:16:54

So more advanced then a manual nano script then or docker file?

<@zlopez:fedora.im>

17:17:26

We should move on to next topic, Ahmed Almeleh there will be an open floor at the end, where you can ask some questions

<@zlopez:fedora.im>

17:17:55

!info magic eight ball says:

<@zlopez:fedora.im>

17:17:55

!topic Next chair

<@zlopez:fedora.im>

17:17:55

!info chair 2026-04-02 - ???

<@zlopez:fedora.im>

17:17:55

!info chair 2026-03-26 - mkonecny

<@zlopez:fedora.im>

17:18:08

Do we have any volunteers to run next meeting?

<@patrikp:matrix.org>

17:18:44

I would take it but high likelihood I won't be here.

<@gwmngilfen:fedora.im>

17:19:09

usual answer for me, I can't stay til the end

<@gwmngilfen:fedora.im>

17:19:21

(n fact I have to go even earliet today, in a few mins 😕)

<@zlopez:fedora.im>

17:19:38

No problem

<@zlopez:fedora.im>

17:20:01

It would be nice to have volunteer at least for next week

<@gwmngilfen:fedora.im>

17:20:52

I can certainly start it next week. if there's more than me by hafway, someone else can take over 😉

<@gwmngilfen:fedora.im>

17:20:58

I can certainly start it next week. if there's more than me by halfway, someone else can take over 😉

<@zlopez:fedora.im>

17:21:00

So let me do it then

<@gwmngilfen:fedora.im>

17:21:06

if not... well that also solves the problem

<@zlopez:fedora.im>

17:21:17

!info chair 2026-04-02 - mkonecny

<@zlopez:fedora.im>

17:21:35

And what about the week after that

<@zlopez:fedora.im>

17:21:36

!info chair 2026-04-09 - ???

<@patrikp:matrix.org>

17:23:39

Wait, I got my dates mixed up.

<@patrikp:matrix.org>

17:23:52

I can do next week and I won't be here the one after that.

<@zlopez:fedora.im>

17:24:12

Do you want to switch then?

<@patrikp:matrix.org>

17:24:28

I can do next week, sure.

<@zlopez:fedora.im>

17:24:46

!info chair 2026-04-02 - patrikp

<@zlopez:fedora.im>

17:24:46

!info chair 2026-04-09 - mkonecny

<@zlopez:fedora.im>

17:24:57

That looks good, let us continue

<@zlopez:fedora.im>

17:25:05

!topic announcements and information

<@zlopez:fedora.im>

17:25:05

!info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1900 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org

<@zlopez:fedora.im>

17:25:58

!info F44 final freeze starts next week on Tuesday 31st March

<@zlopez:fedora.im>

17:26:53

!info There is a new way to set up monitoring for packages https://the-new-hotness.readthedocs.io/en/stable/user-guide.html#notifications-settings

<@gwmngilfen:fedora.im>

17:28:02

- We also have PSQL monitoring (net new, wasn't in Nagios) which needs some finetuning (lots of alerts about DB locks) but otherwise is looking good

<@gwmngilfen:fedora.im>

17:28:02

I have to run, so a quick monitoring update from me

<@gwmngilfen:fedora.im>

17:28:02

- lots of new checks on Zabbix, but notably the DNS ping & management IP monitoring has been moved from Nagios to Zabbix now

<@gwmngilfen:fedora.im>

17:28:02

- I've also reduced the sensitivity of the Load avg checks that were triggering all the time - now it has to be high for 15mins instead of 5mins. There's an open ELN queue alert I'm not sure about, the rest seems pretty sane

<@gwmngilfen:fedora.im>

17:28:07

catch you later folks 😉

<@zlopez:fedora.im>

17:29:25

And that brings us to monitoring

<@zlopez:fedora.im>

17:29:28

!topic Monitoring discussion [nirik]

<@zlopez:fedora.im>

17:29:28

!info https://nagios.fedoraproject.org/nagios & https://zabbix.fedoraproject.org (top 100 triggers: https://zabbix.fedoraproject.org/zabbix.php?action=toptriggers.list)

<@zlopez:fedora.im>

17:29:28

!info Go over existing items and fix them

<@zlopez:fedora.im>

17:30:53

I can only add there there were postfix alert all over the place related to lmdb change, I fixed all I found, but it's possible that some of them will happen later especially on machines where the amount of sent messages is really low

<@zlopez:fedora.im>

17:31:05

I can only add that there were postfix alert all over the place related to lmdb change, I fixed all I found, but it's possible that some of them will happen later especially on machines where the amount of sent messages is really low

<@zlopez:fedora.im>

17:31:13

I can only add that there were postfix alerts all over the place related to lmdb change, I fixed all I found, but it's possible that some of them will happen later especially on machines where the amount of sent messages is really low

<@zlopez:fedora.im>

17:32:33

I don't see anything else that weren't already mentioned by Gwmngilfen, so let's move on

<@zlopez:fedora.im>

17:32:51

!link https://forge.fedoraproject.org/infra/tickets/issues?q=&type=all&sort=leastupdate&state=open&labels=&milestone=0&project=0&assignee=0&poster=0

<@zlopez:fedora.im>

17:32:51

!topic Fedora Infra backlog refinement

<@zlopez:fedora.im>

17:32:51

!info Refine oldest tickets on Fedora Infra tracker

<@zlopez:fedora.im>

17:33:13

So let's go through some tickets and let's see if we can add something to them

<@zlopez:fedora.im>

17:33:29

!link https://forge.fedoraproject.org/infra/tickets/issues/12058

<@zlopez:fedora.im>

17:34:42

This is waiting for davdunc, but he probably didn't have time to check it out or missed it. I will try to ping him again

<@zlopez:fedora.im>

17:35:12

!link https://forge.fedoraproject.org/infra/tickets/issues/12761

<@zlopez:fedora.im>

17:36:30

I see that there was some question from nirik, which got already answered, so it's now waiting on him to move it forward

<@zlopez:fedora.im>

17:37:02

!link https://forge.fedoraproject.org/infra/tickets/issues/11825

<@zlopez:fedora.im>

17:38:13

Hm, I must have missed the last update from David Kirwan here

<@zlopez:fedora.im>

17:38:22

This is still something we would like to do

<@zlopez:fedora.im>

17:40:00

Responded in the ticket

<@zlopez:fedora.im>

17:40:34

Let me do one more ticket and then switch to open floor as there isn't much engagement here anyway

<@zlopez:fedora.im>

17:40:49

!link https://forge.fedoraproject.org/infra/tickets/issues/12234

<@zlopez:fedora.im>

17:42:02

I assume this is now waiting for decision which way to go on the notification implementation

<@zlopez:fedora.im>

17:43:32

Responded in the ticket

<@patrikp:matrix.org>

17:43:35

Looks like it.

<@patrikp:matrix.org>

17:43:43

I don't have much to say on any of these...

<@zlopez:fedora.im>

17:43:51

!topic Open Floor

<@zlopez:fedora.im>

17:44:13

Let's do the open floor, this is the place to talk about anything infra related

<@zlopez:fedora.im>

17:44:41

If there will be nothing, I will close the meeting in 5 minutes

<@patrikp:matrix.org>

17:46:33

Looks like it's a wrap. Have a good day everyone and thanks for running the meeting Michal.

<@ahmedalmeleh:matrix.org>

17:46:43

Thanks

<@zlopez:fedora.im>

17:47:21

In that case let me end the meeting

<@zlopez:fedora.im>

17:47:28

!info Thank you all for coming!

<@zlopez:fedora.im>

17:47:28

!endmeeting