16:07:45 <nirik> #startmeeting Infrastructure (2021-12-09)
16:07:45 <zodbot> Meeting started Thu Dec  9 16:07:45 2021 UTC.
16:07:45 <zodbot> This meeting is logged and archived in a public location.
16:07:45 <zodbot> The chair is nirik. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:07:45 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:07:45 <zodbot> The meeting name has been set to 'infrastructure_(2021-12-09)'
16:07:45 <nirik> #meetingname infrastructure
16:07:45 <nirik> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak computerkid
16:07:45 <zodbot> The meeting name has been set to 'infrastructure'
16:07:45 <zodbot> Current chairs: bodanel computerkid dtometzki jnsamyak mobrien nirik pingou siddharthvipul zlopez
16:07:45 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
16:07:45 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:07:46 <nirik> #topic greetings!
16:08:02 <nirik> #topic New folks introductions
16:08:03 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:08:03 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:09:09 <nirik> anyone new today?
16:10:42 <darknao> .hi
16:10:43 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:10:44 <copperi[m]> guess not
16:10:55 <petebuffon> .hello petebuffon
16:10:56 <mkonecny> .hello
16:10:56 <zodbot> petebuffon: petebuffon 'Peter Buffon' <pabuffon@gmail.com>
16:10:58 <zodbot> mkonecny: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
16:11:14 <copperi[m]> .hello copperi
16:11:15 <zodbot> copperi[m]: copperi 'Jan Kuparinen' <copper_fin@hotmail.com>
16:11:27 <austinpowered> .hi
16:11:28 <zodbot> austinpowered: austinpowered 'T.C. Williams' <fedoraproject@wootenwilliams.com>
16:11:42 <mkonecny> .hello zlopez
16:11:43 <zodbot> mkonecny: zlopez 'Michal Konecny' <michal.konecny@psmail.xyz>
16:12:17 <pmoura> .hello phsmoura
16:12:17 <zodbot> pmoura: phsmoura 'Pedro Moura' <moura.pedro123@gmail.com>
16:12:17 <nirik> ok, looks like no new folks today...
16:12:35 <nirik> #topic Next chair
16:12:35 <nirik> #info magic eight ball says:
16:12:35 <nirik> #info chair 2021-12-09 - dtometzki
16:12:35 <nirik> #info chair 2021-12-16 - siddharthvipul1
16:13:03 <nirik> I am guessing no meeting on the 23rd and 30th?
16:13:37 <nirik> #info chair 2022-01-06 - ?
16:13:39 <mkonecny> But we should probably look at 2021-01-06
16:13:45 <nirik> who wants to start off the new year? ;)
16:14:11 <petebuffon> sure thing I can!
16:14:18 <nirik> cool. ;)
16:14:22 <nirik> thanks petebuffon
16:14:42 <nirik> #topic announcements and information
16:14:42 <nirik> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:14:42 <nirik> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:14:42 <nirik> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request
16:14:43 <nirik> #info infra weekly meeting cancelled on 23rd and 30th December
16:14:48 <nirik> any other announcements or info?
16:15:27 <mkonecny> #info The New Hotness 1.0.0 was released and deployed in production
16:16:29 <nirik> cool!
16:16:43 <nirik> #topic Oncall
16:16:43 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:16:43 <nirik> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:16:43 <nirik> ## .oncalltakeeu .oncalltakeus
16:16:43 <nirik> #info siddharthvipul on call from 2021-12-02 to 2021-12-09
16:16:46 <nirik> #info dtometzki on call from 2021-12-09 to 2021-12-16
16:16:48 <nirik> #info no one on call from 2021-12-17 to 2022-01-06
16:16:50 <nirik> #info ? on call from 2022-01-06 to 2022-01-13
16:17:03 <nirik> anyone want to take oncall from 01-06 to 01-13? I guess I can if no one else wants it
16:17:48 <darknao> i can take it
16:18:10 <nirik> ok, cool. thanks.
16:18:31 <nirik> #info Summary of last week: (from current oncall )
16:18:40 <nirik> I don't think siddharthvipul is here.
16:18:47 <VipulSiddharth[m> we got one ping on sunday early morning ping but nirik was around to answer that → copr 502 issue https://pagure.io/fedora-infrastructure/issue/10391
16:18:53 <nirik> oh, he is. ;) welcome
16:18:58 <VipulSiddharth[m> ^-^
16:19:11 <VipulSiddharth[m> so yeah, just 1 ping that I saw
16:19:16 <nirik> yeah, thats copr's backend dying on logrotate. ;( Hopefully fixed now.
16:19:42 <VipulSiddharth[m> 🤞
16:19:54 <nirik> #topic Monitoring discussion [nirik]
16:19:55 <nirik> #info https://nagios.fedoraproject.org/nagios
16:19:55 <nirik> #info Go over existing out items and fix
16:20:21 <nirik> some down aarch64 mustangs still... a down s390x builder (but I can fix that today).
16:20:53 <nirik> our outgoing email queue is having problems. gmail is throttling us... ;(
16:21:40 <nirik> about 190k emails trying to go out.
16:21:51 <nirik> other than that, the normal small stuff.
16:22:08 <mkonecny> Could we do anything about gmail?
16:22:17 <nirik> sadly, nothing I know of.
16:22:31 <nirik> we may have to grow the disk on bastion01 if it keeps up.
16:22:47 <nirik> I suppose we could drop emails, but some people may not want that.
16:23:18 <mkonecny> Lost emails are not great
16:23:52 <nirik> the other thing we could do is fix our notifications/hook to not send one email per commit, but a summary on merges.
16:24:38 <nirik> but I am not sure how easy that is to do.
16:25:24 <nirik> so, we were going to have a learning topic today, but turns out mobrien is in another meeting, so he wanted us to push to next week.
16:25:48 <nirik> we could do some backlog refinement, or just call it a short meeting?
16:25:48 <mkonecny> This would need a place to hold the information till we can handle it :-/
16:25:48 <mobrien> Actually my meeting just finished so I can do it
16:25:56 <nirik> ah ha!
16:26:05 <nirik> cool, take it away mobrien!
16:26:42 <mobrien> Ok let me know if any of my pasted sentences are too big and get swallowed by the matrix bridge
16:27:00 <nirik> #topic Learning topic
16:27:11 <mobrien> So My talk is on how Fedora does its work, feel free to interrupt with questions
16:27:24 <mobrien> There are three main categories of work in Fedora
16:27:32 <mobrien> * Day to day
16:27:38 <mobrien> * Mini initiatives
16:27:45 <mobrien> * Initiatives
16:27:55 <mobrien> Day to day includes tickets raised on the ticketing trackers which can be completed in a relatively short time frame. Anything from a few minutes up to a few days.
16:28:03 <mobrien> Also included in day to day may be general tasks like server upgrades, or reviewing pull requests etc..
16:28:57 <mobrien> When tickets are submitted on the fedora-infrahttps://pagure.io/fedora-infrastructure/issues or releng https://pagure.io/releng/issues trackers they are triaged in standup meetings mon-thurs twice daily.
16:29:03 <mobrien> s/infrahttps/infra https/
16:29:27 <mobrien> The tickets are evaluated as to how much work/trouble is involved to complete them.
16:29:50 <mobrien> Ideally all work should be ticketed but in the real world we sometimes forget
16:30:49 <mobrien> Tickets are triages with `gain` which is how much it effects the community and `trouble` which is the effort needed to resolve it
16:31:17 <Leo[m]> .hello leo
16:31:17 <mobrien> There are other tags like ops/dev or possibly categories like aws or bodhi etc..
16:31:18 <zodbot> Leo[m]: leo 'Leo Puvilland' <leo@craftcat.dev>
16:31:47 <mobrien> This helps when filtering what work needs to be done
16:32:22 <mobrien> Ideally high gain/low trouble tickets would be worked on first and high trouble/low gain last
16:32:43 <mobrien> this gives the best bang for your buck
16:32:49 <mobrien> If a ticket is deemed to large to be completed in a couple of days by a single person it will be tagged as a mini-initiative.
16:32:59 <mobrien> Mini-initiatives are medium to large bodies of work which would typically take more than a week to complete but less than a month of dedicated work.
16:33:36 <mobrien> These are evaluated by the Infra & Releng team(The red hat team who deal with Fedora infrastructure) and prioritised in our roughly monthly team planning meetings.
16:33:48 <mobrien> The highest priority mini initiative is then worked on as a second priority after day to day work.
16:34:01 <mobrien> As mini-initiative work is worked on as a second priority it can sometimes take a little longer to be completed.
16:34:44 <mobrien> Some examples of mini-initiatives can be found here https://pagure.io/fedora-infrastructure/boards/mini-initative
16:35:07 <mobrien> The third category is Initiatives. These are large bodies of work which are submitted here https://pagure.io/cpe/initiatives-proposal/issues for consideration.
16:35:22 <mobrien> The initiative proposals are reviewed by our product owner amoloney, if enough info is not provided or anything is unclear the PO will reach out to the requester for further information.
16:35:57 <mobrien> PO stands for Product Owner, just realised that may not be obvious
16:36:25 <mobrien> If the proposal is then deemed fit it will be passed to the ARC team a sub team of Infra & releng, this usually consists of myself and two others who will have the correct skill set to investigate.
16:36:55 <mobrien> Details of ARC are here https://fedora-arc.readthedocs.io/en/latest/index.html
16:37:07 <mobrien> The ARC team will do investigation into the technical feasibility of the proposal and also scope the requirements including how many people of what skill set are needed to work on it
16:38:12 <mobrien> After scoping, if deemed feasible the initiative along with a few others will be proposed to the stakeholders of the CPE team (another slighlty bigger Red Hat team) for work in the next quarter. The stakeholders will then prioritise these and the highest ones will be worked on.
16:38:30 <mobrien> The amount of initiatives worked in each quarter varies depending on resources but is usually 2 or 3.
16:38:58 <mobrien> The initiatives will then be worked on by a dedicated team of people from CPE and hopefully completed successfully and becomes part of Fedora Infrastructure
16:39:29 <mobrien> That was a big wall of text so I will let that sit for a minute and please feel free to ask any questions
16:41:29 * nirik nods.
16:41:56 <darknao> is Infra & Relang part of CPE team or that are two very distinct teams ?
16:42:17 <mobrien> Infra & Releng is a sub team of CPE
16:42:49 <mobrien> CPE stands for Community Platform Engineering and is responsible for Fedora & Centos
16:43:34 <mobrien> https://docs.fedoraproject.org/en-US/cpe/
16:45:12 <mobrien> There is are 40+ people in CPE and they are divided into a few sub teams, one of which is Infra & Releng
16:45:26 <VipulSiddharth[m> darknao: Infra and Releng is a sub team like other initiatives in CPE
16:45:44 <VipulSiddharth[m> where the main goal is keeping the lights on (in very simple words)
16:46:33 <nirik> but do note that community memebers are welcome to contribute wherever they like... with infra&releng or iniatives etc.
16:47:16 <mobrien> Yes, contribution anywhere is encouraged. We could always do with help
16:48:57 <mobrien> Some examples of Initiatives are things like noggin/AAA
16:49:16 <mobrien> That is probably the biggest one we have undertaken as a team
16:49:30 <mobrien> rpm autospec is another recent one
16:50:17 <mobrien> The ocp4 cluster for Fedora is another you may have heard of
16:50:45 <mobrien> We do some large chunks of tech debt as initiatives at times too like improvements to existing services if there is a lot of work needed
16:51:20 <VipulSiddharth[m> I like to think CPE is just a part of community, so anything CPE does, can be done by community member
16:51:20 <VipulSiddharth[m> Only separate things I can think might be some GDPR problems or internal Red Hat stuff (which is expected)
16:52:11 <mobrien> CPE is kind of like full time community members who can dedicate there full time
16:53:39 <mobrien> Ok if there's no more questions I guess we can move on to open floor
16:53:46 <nirik> thanks mobrien
16:53:51 <nirik> #topic Open Floor
16:54:16 <nirik> anything anyone has for open floor? thoughts? fav warm holiday drink?
16:54:41 <mobrien> I am partial to a hot port in this weather
16:54:49 <petebuffon> hot chocolate with a nip of whiskey
16:55:39 <mobrien> Baileys coffee is a festive favourite in a lot of Irish households too
16:56:30 <nirik> oh yeah, I should get some Baileys... great in coffee. ;)
16:56:52 <nirik> ok, thanks everyone!
16:56:55 <nirik> #endmeeting