15:01:18 <nirik> #startmeeting Infrastructure (2020-10-01)
15:01:18 <zodbot> Meeting started Thu Oct  1 15:01:18 2020 UTC.
15:01:18 <zodbot> This meeting is logged and archived in a public location.
15:01:18 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:18 <zodbot> The meeting name has been set to 'infrastructure_(2020-10-01)'
15:01:18 <nirik> #meetingname infrastructure
15:01:18 <zodbot> The meeting name has been set to 'infrastructure'
15:01:18 <nirik> #chair nirik pingou smooge cverna mizdebsk mkonecny abompard siddharthvipul mobrien
15:01:18 <zodbot> Current chairs: abompard cverna mizdebsk mkonecny mobrien nirik pingou siddharthvipul smooge
15:01:18 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
15:01:19 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
15:01:20 <nirik> #topic aloha
15:01:25 <siddharthvipul> hellooo
15:01:31 <mobrien> .hello2
15:01:32 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com>
15:01:32 <siddharthvipul> .hello siddharthvipul1
15:01:34 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
15:02:38 <pingou> ó/
15:02:40 <coderdude> hello everyone .
15:02:40 <coderdude> came from the hacktoberfest meet :)
15:02:48 <siddharthvipul> coderdude, welcome :)
15:03:05 <nirik> morning everyone
15:03:18 <mkonecny> .hello zlopez
15:03:19 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@psmail.xyz>
15:03:46 * nirik will wait a min or two more for folks to wander in
15:05:45 <snehalbaghel> o/
15:06:15 <nirik> ok, lets dive on in
15:06:28 <nirik> #topic New folks introductions
15:06:28 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:06:28 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:06:38 <nirik> any new folks today that would like to introduce themselves?
15:06:53 <siddharthvipul> coderdude, snehalbaghel I see 2 new names
15:06:54 <coderdude> Me
15:07:00 <siddharthvipul> would you like to give a short introduction?
15:07:23 <coderdude> Hello. I am Adwait Thattey
15:07:23 <coderdude> Currently studying in IIIT Sri City, India
15:07:42 <coderdude> I have contributed to one project in the past, a couple of PRs
15:07:53 <coderdude> Here to explore more projects and learn :)
15:07:57 <nirik> welcome coderdude!
15:08:12 <mkonecny> welcome on board coderdude
15:08:12 <snehalbaghel> Hi I'm Snehal. I came from the hacktoberfest meeting. I'm currently working on Fedor Barges and was just curious to see what goes around here after Vipul's talk
15:08:27 <siddharthvipul> :}
15:08:42 <mobrien> welcom snehalbaghel and coderdude
15:08:47 <nirik> cool. welcome snehalbaghel also
15:09:01 <mobrien> I hope siddharthvipul was telling you all good things :)
15:09:13 <siddharthvipul> mobrien, >:) you think so?
15:09:20 <snehalbaghel> Haha that's why we're here
15:09:20 <mkonecny> Welcome both on the board of this ship
15:09:41 <nirik> shall we sail on?
15:09:47 <nirik> #topic Next chair
15:09:47 <nirik> #info magic eight ball says:
15:09:47 <nirik> #info 2020-10-01 - nirik
15:09:47 <nirik> #info 2020-10-08 - mobrien
15:09:48 <nirik> #info 2020-10-15 - siddharthvipul
15:10:06 <mobrien> +1 for next week
15:10:08 <nirik> I guess we have the next 2... do we want someone for 2020-10-22?
15:10:39 <nirik> probibly 2 is enough ahead. ;)
15:10:53 <nirik> #topic announcements and information
15:10:53 <nirik> #info CPE Sustaining EU-hours team has a Monday through Friday 30 minute meeting going through tickets at 0830 UTC in #centos-meeting
15:10:53 <nirik> #info CPE Sustaining NA-hours team has a Monday through Friday 30 minute meeting going through tickets at 1800 UTC in #fedora-admin
15:10:53 <nirik> #info Datacenter move is over, but some items still need to be done: see https://fedoraproject.org/wiki/Infrastructure/2020-post-datacenter-move-known-issues
15:10:55 <nirik> #info CPE at Nest blog post is out https://communityblog.fedoraproject.org/cpe-at-nest-2020/
15:10:57 <nirik> #info F33 Beta freeze is over
15:10:59 <nirik> #info F33 Final freeze is starting on 2020-10-06
15:11:02 <nirik> any other info or announcements ?
15:11:16 <nirik> I guess the pagure upgrade finished...
15:11:48 <mobrien> yes, it was succesfull thanks to pingou and misc
15:12:07 <nirik> #info pagure now on python3/rhel8
15:12:11 <nirik> great work everyone!
15:12:22 <nirik> oh, there's an old ticket about selinux on pagure...
15:12:52 <pingou> and mobrien :)
15:12:53 <nirik> ah, thats for pkgs01...
15:13:08 <nirik> https://pagure.io/fedora-infrastructure/issue/7941
15:13:22 <nirik> anyhow, any other announcements? I guess hacktoberfest started...
15:14:41 <mkonecny> Yes, today
15:15:16 <nirik> #topic Oncall
15:15:16 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:15:16 <nirik> #info nirik is oncall for 2020-09-24 to 2020-10-01
15:15:16 <nirik> #info siddharthvipul is oncall for 2020-10-01 to 2020-10-08
15:15:16 <nirik> #info ? is oncall for 2020-10-09 to 2020-10-15
15:15:27 <nirik> anyone want oncall for that last block? ;)
15:15:36 <nirik> #info Summary of last week: (from current oncall )
15:15:58 <nirik> Not much to report... just one or two pings and easy to direct people to the place to get them addressed.
15:16:18 <nirik> siddharthvipul: you want to take it over now?
15:16:43 <siddharthvipul> yess
15:16:55 <siddharthvipul> .oncalltakeeu
15:16:57 <zodbot> siddharthvipul: Kneel before zod!
15:17:02 <siddharthvipul> zodbot, \o/
15:17:03 <mobrien> I can take that last oncall block if nobody else wants it
15:17:12 <nirik> mobrien: thanks!
15:17:32 <nirik> #topic Monitoring discussion [nirik]
15:17:32 <nirik> #info https://nagios.fedoraproject.org/nagios
15:17:32 <nirik> #info Go over existing out items and fix
15:17:35 <nirik> lets see...
15:17:58 <nirik> now that we are out of freeze we could fix up some of these nagging ones.
15:18:18 <nirik> proxy39 has a bunch of alerts, but I think thats being rebuilt?
15:18:34 <mobrien> I have some of the nagios checks updated to python3 I'll be pushing them tomorrow hopefully
15:18:37 <nirik> pagure01 has some, but it's no longer in service. ;)
15:18:49 <mobrien> proxy39 playbook is running at the moment
15:19:22 <nirik> cool. Yeah, perhaps tomorrow we can spend some time fixing things... but it's all pretty minor
15:19:49 <nirik> That takes us to the reafs of learning...
15:19:51 <nirik> #topic Learning topic
15:19:52 <nirik> #info mirrorlist [nirik] for 2020-10-01
15:19:52 <smooge> ... comes back and sees he missed a meeting
15:20:01 <nirik> smooge: the best kind of meeting. ;)
15:20:04 <pingou> I've removed pagure01 from the pagure group, but the host is still up and disk still present
15:20:23 <pingou> but running the pagure playbook shouldn't touch it
15:20:54 <nirik> fedoraproject has a network of volenteer mirrors that mirror our content. They pull from our download servers and are registered in our mirrormanager application.
15:21:37 <nirik> On each of our proxies we run a small rust application called mirrorlist-server.
15:22:06 <nirik> each hour mirrormanager collects any changes to repos and pushes out new data to those mirrorlist-servers.
15:22:24 <nirik> users then in turn hit those (by default) at mirrors.fedoraproject.org
15:22:47 <nirik> the application then produces a metalink and hands it back to the users dnf or yum
15:23:14 <nirik> a metalink is a cool file format, it has a list of places that have a file and the checksums of it so the client can know they have the exact right one.
15:23:49 <nirik> so users dnf/yum parse the metalink and then go to mirrors listed in it until they find one with a valid repomd.xml file, then they download the packages based on that.
15:24:40 <nirik> There's a bit more complexity as the metalink also lists the last 3 repomd.xml checksums... ie, any of the last 3 are valid... this is prevent everyone from hitting our master mirrors right after it updates
15:24:46 <nirik> (since it would be the only up to date thing)
15:25:15 <nirik> also in mirrormanager sites can list things that they want to always go to their mirror... ie, the metalink is just their mirror for those clients.
15:25:30 <nirik> They can list that by network block, ASN, etc.
15:25:53 <nirik> on our end we only see the mirrors/metalink part, the rest goes directly to mirrors in our network...
15:26:07 <nirik> so thats why we can't say 'package X was downloaded Y times'. we don't know.
15:26:30 <nirik> This is probibly our most critical service. If it's down, people tell us really quickly.
15:26:40 <nirik> Luckily it's in general pretty solid...
15:26:50 <nirik> Any questions or places I can clarify?
15:27:02 <mobrien> I have one
15:27:18 <mobrien> Whats the difference between a tier1 mirror and the rest?
15:28:29 <nirik> ah yeah... so tier1 mirrors are ones that pull directly from us. The rest of the mirrors are supposed to pull from their nearest tier1 mirror.
15:28:46 <mobrien> oh right ok
15:30:04 <nirik> it's sort of a CDN, but not... ;)
15:30:09 <nirik> any other questions?
15:31:37 <mkonecny> What about the ostree images, are they using the same mirrors?
15:31:56 <nirik> mkonecny: nope. ostree we found was difficult for our mirror network... tons of small files...
15:32:14 <nirik> so, it's hosted at our main datacenter and has amazon cloudfront in front of it.
15:32:52 <mkonecny> nirik: Ok, thanks for answer
15:33:18 <nirik> Hopefully it's pretty fast. :) and many thanks to amazon for providing that, it would be difficult on our mirror network
15:33:48 <mkonecny> I can confirm, that it's fast most of the time :-)
15:33:54 <nirik> #topic Open Floor
15:34:08 <nirik> So, we need more learning topics and teachers...
15:34:52 <nirik> if anyone has topics or wants to talk on anything, please add it on the board.
15:35:13 <mobrien> I'd be willing to talk on anything just not sure I have anything people want to hear :)
15:35:59 <mobrien> packager workflow may be a good one but not sure who could talk on it
15:36:07 <mkonecny> I could talk about release-monitoring.org
15:36:50 <mkonecny> The packager workflow would be a nice topic
15:36:58 <nirik> I can talk on packager workflow if folks like...
15:37:09 <nirik> mkonecny: a release monitoring one would be nice
15:37:16 <mobrien> +1 for both of those talks
15:38:50 <nirik> cool. thanks.
15:39:01 <nirik> any other items for open floor?
15:39:29 <mkonecny> It will be also nice to have talk about other apps that are running in infra like koji or bodhi
15:40:00 <nirik> yeah, I could do a koji one, or mboddu perhaps... not sure who would best for bodhi
15:40:33 * mboddu reading back, multiple things going on
15:40:40 <nirik> no worries.
15:40:42 <mobrien> maybe cverna might do a guest appearance for bodhi
15:41:38 * mboddu got some time now, as the process that I am running failed and its someone else job to fix it :D
15:41:39 <nirik> yeah, we should try and get more of our developers up on it... I am sure cverna doesn't want to own it forever. :) But would be good to teach about it
15:41:57 <mkonecny> fedora messaging itself could be also a nice topic to learn about
15:42:11 <mkonecny> Or noggin
15:42:19 <nirik> +1 to both, yeah.
15:42:41 <nirik> perhaps we should fire a list thread and get people lined up for talks?
15:42:41 <pingou> pagure, mdapi, toddlers :)
15:42:47 <pingou> there is no shortage of apps to discuss :)
15:42:47 <nirik> (or is that... types)
15:43:12 <mboddu> I can talk about fedora release engineering, how the package builds gets built and distributed in different life cycles of fedora
15:43:13 <mkonecny> pingou: You are right, it will be nice to have learning session about each of them :-)
15:43:26 <mkonecny> mboddu: +1
15:43:49 <mboddu> Just let me know when I can talk about it
15:43:54 * mboddu jumping again :)
15:43:57 <nirik> mboddu: sign up on the board...
15:44:12 <mboddu> Will do
15:45:59 <nirik> ok, if nothing more will close in a min
15:47:20 <nirik> Thanks everyone!
15:47:22 <nirik> #endmeeting