16:00:07 <nirik> #startmeeting Infrastructure (2023-09-14)
16:00:07 <zodbot> Meeting started Thu Sep 14 16:00:07 2023 UTC.
16:00:07 <zodbot> This meeting is logged and archived in a public location.
16:00:07 <zodbot> The chair is nirik. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:00:07 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:07 <zodbot> The meeting name has been set to 'infrastructure_(2023-09-14)'
16:00:07 <nirik> #meetingname infrastructure
16:00:07 <zodbot> The meeting name has been set to 'infrastructure'
16:00:07 <nirik> #chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg patrikp
16:00:07 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
16:00:07 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:00:07 <zodbot> Current chairs: bodanel dtometzki jnsamyak lenkaseg nb nirik patrikp zlopez
16:00:08 <nirik> #info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
16:00:09 <nirik> #topic ahoy!
16:00:16 <dtometzki> .hi
16:00:18 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <damian@riscv.tometzki.de>
16:00:36 <zlopez> .hi
16:00:38 <zodbot> zlopez: zlopez 'Michal Konecny' <michal.konecny@pacse.eu>
16:01:12 * nirik will wait until 5after for folks to wander in. ;)
16:03:20 <nirik> two min more ⏲️
16:03:38 <dtometzki> it is nice to join the meeting after some busy weeks
16:03:55 <nirik> welcome back. ;)
16:04:05 <nirik> Oh, there's a hourglass emoji... ⏳
16:04:32 <patrikp> Which client do you use Kevin?
16:05:00 <nirik> hexchat for irc. (for emoji's I'm using gnome-characters to find them and copy, then just paste into hexchat)
16:05:08 <nirik> ok, shall we get started then?
16:05:13 <darknao> .hi
16:05:14 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:05:15 <zlopez> +1
16:05:20 <nirik> #topic New folks introductions
16:05:20 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:05:20 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:05:31 <nirik> any new folks today want to give a quick intro?
16:06:01 <nirik> seems not...
16:06:08 <nirik> #topic Next chair
16:06:08 <nirik> #info magic eight ball says:
16:06:08 <nirik> #info chair 2023-09-21 phsmoura
16:06:08 <nirik> #info chair 2023-09-28 pcreech
16:06:09 <nirik> #info chair 2023-10-05 ?
16:06:17 <nirik> anyone interested in the 10-05 meeting char?
16:06:28 <dtometzki> i will take 10-05
16:06:36 <nirik> thanks dtometzki !
16:06:53 <nirik> #topic announcements and information
16:06:54 <nirik> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0730 UTC in #centos-meeting
16:06:54 <nirik> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:06:54 <nirik> #info we are in f39 beta freeze
16:06:54 <nirik> #info next go/nogo for f39 beta is September 14
16:06:59 <nirik> any other announcements?
16:07:08 <nirik> The go/nogo is in about 53min...
16:07:50 <austinpowered> Not new but it's been a minute since I last attended.
16:08:03 <nirik> hey austinpowered. welcome back.
16:08:29 <austinpowered> thanks
16:08:41 <nirik> care to give a short intro/what your interest are, etc?
16:09:53 <austinpowered> Offically, I'm retired. But I still have quite the homelab and a strong interest in DevOps and security.
16:10:41 <nirik> Oh, FYI, I haven't seen much more feedback from the flock workshop/onboarding/etc discussion, so I think soon I am going to see about getting our docs.stg site to update from a staging branch and start making those changes, once they look reasonable, just merge back in to push to prod.
16:10:54 <nirik> austinpowered: great!
16:11:20 <nirik> feel free to ask questions or chime in whenever you like...
16:12:19 <nirik> lets move to oncall...
16:12:22 <nirik> #topic Oncall
16:12:22 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:12:22 <nirik> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:12:22 <nirik> ## .oncalltakeeu .oncalltakeus
16:12:22 <nirik> #info nirik  is on call from 2023-09-08 to 2023-09-14
16:12:23 <nirik> #info zlopez is on call from 2023-09-15 to 2023-09-21
16:12:25 <nirik> #info pcreech  is on call from 2023-09-22 to 2023-09-28
16:12:27 <nirik> #info ? is on call from 2023-09-29 to 2023-10-05
16:12:32 <nirik> Anyone interested in taking the oncall that last week?
16:12:49 <dtometzki> i will take it
16:13:18 <nirik> excellent
16:13:38 <nirik> #info Summary of last week: (from current oncall )
16:13:53 <nirik> No oncall pings.
16:14:19 <zlopez> .oncalltakeeu
16:14:19 <zodbot> zlopez: Kneel before zod!
16:14:21 <nirik> #topic Monitoring discussion [nirik]
16:14:22 <nirik> #info https://nagios.fedoraproject.org/nagios
16:14:22 <nirik> #info Go over existing out items and fix
16:14:53 <nirik> mailman01 is right at the disk alert threshold.
16:15:03 <nirik> so it alerts, then logs rotate and it's ok, then...
16:15:12 <nirik> I guess we could adjust the limit there
16:15:57 <nirik> some copr power8 boxes are locking up... we aren't sure fully why
16:16:12 <nirik> otherwise it's been pretty quiet
16:16:43 <zlopez> Thats nice
16:16:53 <nirik> always
16:17:25 <nirik> zlopez: you good to talk about quay.io/registry today?
16:18:07 <zlopez> Yes, I am
16:18:13 <patrikp> Where is today's go/nogo meeting supposed to take place?
16:18:21 <zlopez> #fedora-meeting
16:18:49 <nirik> #topic Learning topic
16:19:03 <nirik> take it away!
16:19:05 <nirik> oops
16:19:29 <zlopez> So let's start
16:19:29 <nirik> take it away!
16:20:02 <zlopez> I recently started to investigate if the registry.fedoraproject.org could be moved to quay.io
16:20:42 <zlopez> For anybody who wants to read it later the whole investigation is documented here: https://fedora-arc.readthedocs.io/en/latest/registry_to_quay/index.html
16:21:40 <zlopez> The reason for move is to use the features the quay.io has available and get rid of our own hosted registry, which is not that well maintained
16:22:35 <zlopez> The whole investigation was split in three parts: Flatpaks, Toolbx containers and Fedora Base/Minimal/CoreOS containers
16:23:09 <zlopez> Let's start with the Base/Minimal images for example
16:23:47 <zlopez> Here is the workflow for those containers https://fedora-arc.readthedocs.io/en/latest/_images/container_workflow_registry.drawio.png
16:24:03 <zlopez> As you can see with F39 there is a lot changes
16:24:36 <zlopez> It's not using OSBS anymore, but it's for now bypassing Bodhi
16:25:42 <zlopez> This made the investigation much simpler, because the only thing that needs to change currently is the releng script which is located here https://pagure.io/releng/blob/main/f/scripts/sync-latest-container-base-image.sh
16:27:08 <zlopez> In case of Bodhi the change is simple enough, just change where the Bodhi is pushing the container images and from where
16:28:03 <zlopez> In case of OSBS the change is more complex, there is dockercfg file which needs to be updated and it's need to be deployed in specific playbook
16:28:30 <zlopez> Here is the dockercfg file https://pagure.io/fedora-infra/ansible/blob/main/f/files/osbs/dockercfg-production-secret.j2 and here is the playbook https://pagure.io/fedora-infra/ansible/blob/main/f/playbooks/groups/osbs/setup-worker-namespace.yml
16:28:55 <zlopez> This is all about the Base/Minimal/Core OS container images
16:30:02 <zlopez> In case of toolbx images the situation is mostly same, there is just toolbox CLI tool that needs to be updated as well, because the registry.fedoraproject.org is hardcoded in it
16:31:18 <zlopez> And last part were the flatpaks
16:32:31 <zlopez> In case of flatpaks the workflow is like this https://fedora-arc.readthedocs.io/en/latest/_images/flatpaks_workflow_registry.drawio.png
16:33:09 <zlopez> This will change with the F39, where the flatpaks will no longer use OSBS, but the koji-flatpak plugin
16:33:42 <zlopez> Which is easy to change, we just need to adjust configuration file for it https://pagure.io/fedora-infra/ansible/blob/main/f/roles/koji_builder/templates/flatpak.conf
16:34:10 <zlopez> The OSBS and Bodhi change will be the same as in case of containers
16:35:23 <zlopez> We will probably decommision OSBS in future, when it will no longer be needed and flatpaks are in phase of migrating from OSBS
16:36:07 <zlopez> So in case of flatpaks we also need to adjust proxies to point to quay.io instead of registry.fedoraproject.org
16:36:36 <nirik> yeah, imho, we should keep registry.fp.o, just redirect it
16:36:41 <zlopez> And there is also flatpak-indexer which creates metadata for flatpaks https://github.com/owtaylor/flatpak-indexer
16:36:49 <zlopez> nirik: Understood
16:37:25 <zlopez> Ok, that was the investigation and how the setup should look on the quay.io side
16:38:00 <nirik> thanks for doing that investigation....
16:38:03 <zlopez> So we should have four namespaces: staging-candidate, staging, production-candidate, production
16:38:49 <zlopez> The staging ones will be for staging builds, just to test out things before going to production, and the production is well... for production :-)
16:39:36 <zlopez> nirik: Do we plan to keep all the content on the registry?
16:40:09 <nirik> so, i filed a ticket about that...
16:40:30 <nirik> currently we don't delete anything at all from prod
16:40:50 <nirik> but we likely should.
16:40:57 <nirik> or discuss it at least
16:41:42 <zlopez> And a last note, I will have a meeting with the quay.io folks next week on Friday, they look happy to host the things for us :-)
16:42:06 <nirik> https://pagure.io/fedora-infrastructure/issue/11512
16:42:09 <zlopez> nirik: +1, it would be nice to free the resources for something else
16:42:58 <nirik> feel free to add me if you would like me on that meeting...
16:43:16 <zlopez> I don't think I have anything else to had, we will probably start working on that after F39 release
16:43:35 <nirik> sounds reasonable
16:44:13 <zlopez> nirik: I will add you as optional for the meeting
16:44:23 <nirik> thanks
16:44:40 <zlopez> Do we know what amount of traffic we had for registry.fedoraproject.org?
16:44:43 <nirik> any questions or comments or discussion?
16:45:43 <patrikp> zlopez++
16:45:43 <zodbot> patrikp: Karma for zlopez changed to 1 (for the release cycle f39):  https://badges.fedoraproject.org/tags/cookie/any
16:46:05 <pcreech_> zlopez++
16:46:06 <nirik> we have logs, but not sure how to get traffic stats
16:46:08 <dtometzki> thanks zlopez
16:46:41 <austinpowered> zlopez++
16:46:41 <zodbot> austinpowered: Karma for zlopez changed to 2 (for the release cycle f39):  https://badges.fedoraproject.org/tags/cookie/any
16:47:14 <nirik> ok, anything else on this, or shall we move to open floor?
16:47:20 <zlopez> nirik: I will try to ask flatpak folks, if they don't watch it
16:47:23 <dtometzki> zlopez++
16:47:23 <zodbot> dtometzki: Karma for zlopez changed to 3 (for the release cycle f39):  https://badges.fedoraproject.org/tags/cookie/any
16:48:09 <nirik> sounds good
16:48:15 <nirik> #topic Open Floor
16:48:40 <nirik> anyone have anything for open floor? questions, comments, favorite coin?
16:49:15 <pcreech> Favorite coin?  The ones in my pocket usually :D
16:50:06 <nirik> I like the susan b anthony 1$ us coin...
16:50:19 <nirik> it's almost the same size as a quarter and confuses everyone.
16:51:22 <nirik> anyhow, thanks for coming everyone!
16:51:27 <nirik> #endmeeting