16:00:14 <nirik> #startmeeting Infrastructure (2023-05-11)
16:00:14 <zodbot> Meeting started Thu May 11 16:00:14 2023 UTC.
16:00:14 <zodbot> This meeting is logged and archived in a public location.
16:00:14 <zodbot> The chair is nirik. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:00:14 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:14 <zodbot> The meeting name has been set to 'infrastructure_(2023-05-11)'
16:00:20 <nirik> #meetingname infrastructure
16:00:20 <zodbot> The meeting name has been set to 'infrastructure'
16:00:34 <nirik> #chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg
16:00:34 <zodbot> Current chairs: bodanel dtometzki jnsamyak lenkaseg nb nirik zlopez
16:00:38 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
16:00:43 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:00:47 <nirik> #info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
16:00:54 <nirik> #topic Ahoy!
16:00:59 <dtometzki> .hi
16:01:00 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <damian@riscv.tometzki.de>
16:01:01 <lenkaseg> .hi
16:01:03 <zodbot> lenkaseg: lenkaseg 'Lenka Segura' <lenka@sepu.cz>
16:01:04 <nirik> who all is around today?
16:01:16 <ashcrow> o/
16:01:17 <aheath1992> .hi
16:01:18 <zodbot> aheath1992: aheath1992 'Andrew Heath' <aheath1992@gmail.com>
16:01:58 <jnsamyak> .hi
16:01:59 <zodbot> jnsamyak: jnsamyak 'Samyak Jain' <samyak.jn11@gmail.com>
16:02:01 <nirik> I'll wait until 5after for crew to come aboard...
16:02:17 <nirik> stow your gear and be ready to make way...
16:02:42 <jnsamyak> Aye aye captain \0/
16:02:44 * nirik is going with a 🏴‍☠️ theme today
16:03:12 * aheath1992 grabs the rum
16:04:23 <nirik> #topic New crew introductions
16:04:39 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:04:44 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:04:54 <nirik> Any new hands in the crew today?
16:05:22 <saibug[m]> .hi seddik
16:05:25 <zodbot> saibug[m]: Sorry, but user 'saibug [m]' does not exist
16:06:32 <nirik> Alright, lets raise ⚓ and set ⛵ then!
16:06:44 <nirik> #next captian
16:06:55 <nirik> #topic Next Captain
16:07:01 <nirik> #info chair 2023-05-18 - mkonecny
16:07:05 <nirik> #info chair 2023-05-25 - dtometzki
16:07:10 <nirik> #info chair 2023-06-01 - ?
16:07:19 <lenkaseg> o/
16:07:21 <nirik> any of you up for sailing this ship on the 1st?
16:07:31 * lenkaseg volunteers
16:07:32 <saibug[m]> * .hi
16:07:46 <nirik> Cheers!
16:08:03 <nirik> #topic announcements and information
16:08:08 <nirik> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0730 UTC in #centos-meeting
16:08:12 <nirik> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:08:23 <nirik> Any other announcements to nail to the mainmast?
16:10:07 <nirik> #topic Oncall
16:10:08 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:10:08 <nirik> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:10:12 <nirik> #info darknao is on call from 2023-05-12 to 2023-05-18
16:10:17 <nirik> #info jednorozec is on call from 2023-05-19 to 2023-05-25
16:10:21 <nirik> #info ? is on call from 2023-05-26 to 2023-06-01
16:10:39 <nirik> anyone up to clime into the crows nest on the last week there?
16:11:25 <nirik> I guess we are well set for the next few weeks, so it doesn't matter too much
16:11:30 <nirik> #info Summary of last week: (from current oncall )
16:11:54 <nirik> Thats me! I didn't sight anything from the crows nest over the last week... no shouts of oncall...
16:12:02 <nirik> #topic Monitoring discussion [nirik]
16:12:16 <nirik> I suppose this one would be better for being the crows nest. Oh well. ;)
16:12:21 <nirik> #info https://nagios.fedoraproject.org/nagios
16:12:36 <nirik> A bit of cleanup here.
16:12:50 <nirik> I removed the dead server that was alerting ( vmhost-cc-rdu03)
16:13:10 <nirik> the copr folks cleaned up a few things
16:13:54 <nirik> We have still been seeing some fmn alerts (the new one). But less so.
16:14:28 <saibug[m]> I think there is a workaround..
16:14:36 <nirik> There's still a ticket on cleaning up some other ones...
16:14:46 <saibug[m]> aheath1992: is in it.. No?
16:14:58 <abompard> do the FMN alerts "resolve themselves" or do you usually do something?
16:15:05 <saibug[m]> s/in/on/
16:15:14 <nirik> There was some things fixed, but there's more...
16:15:22 <aheath1992> yep
16:15:26 <nirik> Aurélien B: they just resolve themselves...
16:16:19 <nirik> Looks like the last one warned, crtiticaled and then recovered about 50min later.
16:16:59 <abompard> intersting
16:17:18 <nirik> https://pagure.io/fedora-infrastructure/issue/11090 is that ticket on monitoring stuff.
16:17:25 <nirik> yeah, not sure whats going on. ;)
16:17:35 <nirik> anyhow, thats it on monitoring that I had...
16:19:31 <nirik> Anything else? or shall we move on to learning?
16:19:31 <lenkaseg> That's me!
16:19:32 <nirik> #topic Learning topic
16:19:32 <nirik> #info fas2discourse operator [lenkaseg] on 2023-05-11
16:19:32 <nirik> lenkaseg: take the wheel and guide us. :)
16:20:06 <lenkaseg> I'll be writing about fas2discourse operator
16:20:33 <lenkaseg> Last mini-initiative, that we finished recently with Saffronique
16:20:56 <lenkaseg> The idea came from mattdm
16:21:26 <lenkaseg> Fas2discourse operator serves to synchronize group membership from FAS (powered by FreeIPA) to Fedora Discussion (a Discourse instance).
16:21:54 <lenkaseg> I mean, Saffronique wrote it while I assisted :)
16:22:13 <lenkaseg> Ok, what does it do and why is it needed?
16:23:24 <lenkaseg> Users in Fedora discussion are added or removed according to their state in FAS (Fedora accounts system).
16:24:14 <lenkaseg> That means that every group in Fedora Discussion exists in FAS as well, under the same name, and every change made by the sponsors to the group in FAS will be automatically mirrored to Fedora Discussion.
16:24:46 <lenkaseg> In other words, FAS is the source of truth for Fedora Discussion group membership.
16:25:16 <lenkaseg> How did we solve it?
16:25:44 <lenkaseg> We're using an ansible operator-sdk, which runs an ansible playbook with several tasks.
16:26:23 <lenkaseg> In the first task, the operator retrieves secrets such as Discourse API key with hostname, and FASJSON hostname with principal from the private ansible repo and populates the variables in the playbook.
16:26:45 <lenkaseg> Second, fas2discourse operator handles the kerberos authentication to FASJSON (FAS API) via keytab file.
16:27:06 <lenkaseg> Third, the operator queries the Discourse API and retrieves the list of the groups and list of the users of each group.
16:27:43 <lenkaseg> Fourth, the operator queries the FASJSON with the Discourse group list and retrieves the membership of each group in FreeIPA.
16:28:10 <lenkaseg> Fifth, using set functions, the operator figures out two things:
16:28:25 <lenkaseg> 1. who is not in Discourse, but is in IPA group => and adds them
16:28:54 <lenkaseg> 2. who is in Discourse and not in IPA group => removes them
16:29:23 <lenkaseg> Important to note it's a one-way sync, so users added/removed by group admins in Discourse will be kicked out/added again with the next loop.
16:29:39 <lenkaseg> The loop runs every 20 mins.
16:30:13 <lenkaseg> (For now, let's see if running it more often will be more suitable)
16:30:25 <lenkaseg> If any tasks fail, the entire loop stops and retries.
16:30:53 <lenkaseg> The API queries and the group/user matching are written as python modules.
16:31:23 <lenkaseg> And that's how it works!
16:31:34 <lenkaseg> The code is here: https://pagure.io/cpe/fas2discourse
16:31:53 <lenkaseg> The SOPs are here: https://docs.fedoraproject.org/en-US/infra/ocp4/sop_fas2discourse_operator/
16:32:12 <lenkaseg> It has been successfully deployed to production last week and running in the openshift cluster.
16:33:12 <lenkaseg> That's all! Thanks for attention! Questions time!
16:33:18 <nirik> Pretty cool. Great work.
16:33:34 <abompard> lenkaseg++
16:33:34 <zodbot> abompard: Karma for lenkaseg changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:33:47 <lenkaseg> Thank you!
16:34:56 <nirik> I wonder how hard it would be to have it use fedora-messaging and trigger on changes to any groups it cares about, but that would add a lot of complexity...
16:35:47 <abompard> And it would remove users added in Discourse
16:37:38 <lenkaseg> nirik: that would be a question better answered by saffronique, you mean to track the changes in the FreeIPA groups from fedora-messaging?
16:37:40 <nirik> it does that already right? just on a 20min schedule instead of when fas changes
16:38:05 <nirik> yeah. Just a thought... it would only be an enhancement...
16:38:34 <lenkaseg> Yes, for now Discourse users have to wait max 20 mins to see the changes apply.
16:40:11 <lenkaseg> Aurélien B: yes, who gets added to a group in Discourse, but not in FAS, get kicked in 20 mns latest :)
16:40:59 <lenkaseg> I mean, in case that group is in FAS.
16:41:05 <abompard> So yeah, we need the regular run, but it could be speeded (sped?) up by a fedora-messaging listener
16:41:17 <lenkaseg> The operator does not remove or add groups. Only users.
16:41:43 <nirik> anyhow, I think it's all good for now...
16:41:48 <abompard> yeah
16:42:02 <nirik> and should be very useful. In fact I know mattdm is already using it for things.
16:42:13 <lenkaseg> Aurélien B: I'll pass that idea to saffronique! Thanks!
16:42:38 <abompard> nirik thought about it first! :-)
16:42:54 <lenkaseg> nirik: I'll pass that idea to saffronique! Thanks!
16:42:56 <lenkaseg> :D
16:43:15 <nirik> All the credit can be yours! :)
16:44:15 <abompard> s/credit/questions about how this could be implemented/
16:46:51 <nirik> 😀
16:46:52 <nirik> ok, any other questions for lenkaseg ?
16:47:18 <nirik> Thanks lenkaseg!
16:47:28 <nirik> #topic Open Waters
16:47:41 <nirik> Anyone have anything they would like to cast out on the open seas?
16:48:32 <smooge> thank you all for the 24x7x52week work you do
16:50:12 <nirik> Thanks everyone!
16:50:20 <nirik> fair winds and following seas to you all!
16:50:22 <nirik> #endmeeting