15:00:04 <mkonecny> #startmeeting Infrastructure (2020-07-16)
15:00:04 <zodbot> Meeting started Thu Jul 16 15:00:04 2020 UTC.
15:00:04 <zodbot> This meeting is logged and archived in a public location.
15:00:04 <zodbot> The chair is mkonecny. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:04 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:04 <zodbot> The meeting name has been set to 'infrastructure_(2020-07-16)'
15:00:04 <mkonecny> #meetingname infrastructure
15:00:04 <mkonecny> #chair nirik pingou smooge cverna mizdebsk mkonecny abompard siddharthvipul mobrien
15:00:04 <zodbot> The meeting name has been set to 'infrastructure'
15:00:04 <zodbot> Current chairs: abompard cverna mizdebsk mkonecny mobrien nirik pingou siddharthvipul smooge
15:00:04 <mkonecny> #info Agenda is at: https://board.net/p/fedora-infra
15:00:05 <mkonecny> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
15:00:06 <mkonecny> #topic aloha
15:00:18 <siddharthvipul> .hello siddharthvipul1
15:00:19 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
15:00:32 <seddik> hello team :)
15:00:34 <mkonecny> Welcome everyone on today's show
15:00:47 <siddharthvipul> seddik: hey \o
15:00:50 <mkonecny> We have a few topics that will blow your mind
15:01:00 <siddharthvipul> mkonecny: not again :P
15:01:09 <nirik> morning
15:01:10 <mkonecny> (Mind blowing is not guaranteed )
15:01:16 * pingou in a meeting :(
15:01:25 <nirik> pingou: we all are... ;)
15:01:28 * nirik runs
15:01:35 <siddharthvipul> bwahaha
15:02:13 * mkonecny Looking at nirik with smile on face :-)
15:02:32 <mkonecny> Ok, let's start with it
15:02:37 <mkonecny> #topic New folks introductions
15:02:37 <mkonecny> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:02:37 <mkonecny> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:02:49 <mkonecny> Anybody new wants to say a hello
15:02:57 <mkonecny> ?
15:02:58 <seddik> Meeeeeeeeeeeeee
15:03:10 <mkonecny> Welcome on board seddik
15:03:18 <siddharthvipul> seddik: hehe, please do :) welcome
15:03:19 <seddik> i have sent an email for introduce  my self :)
15:03:32 <mkonecny> seddik: The microphone is yours
15:03:34 <nirik> welcome seddik
15:04:13 <pingou> nirik: ok, in two :D
15:05:00 <mkonecny> seddik: Do you have any questions? This is a good time to ask any
15:05:33 <seddik> i didn't see any exyfix issue
15:06:09 <nirik> we should try and find/mark some more...
15:06:20 <siddharthvipul> seddik: we haven't updated them for a while, also a lot of infra bugs are not really easy fixes (since privileges) but do go through the list and tell us if you like to work on something
15:06:32 <siddharthvipul> nirik: +1
15:06:40 <mboddu> gchang: You can introduce yourself here
15:06:50 <siddharthvipul> gchang: \o hey there
15:07:01 <seddik> siddharthvipul ok i see :)
15:08:03 <gchang> Hi everyone! I'm an incoming college freshman that's excited to contribute!
15:08:16 <siddharthvipul> gchang: welcome to the Fedora Project :)
15:08:18 <mkonecny> Welcome to the show gchang
15:08:45 <gchang> Thanks :)
15:09:27 <nirik> welcom gchang
15:09:28 <mboddu> Welcome to Fedora project gchang :)
15:09:49 <mkonecny> Time is running and the show must go on
15:09:51 <mkonecny> #topic Next chair
15:09:51 <mkonecny> #info magic eight ball says:
15:09:51 <mkonecny> #info 2020-07-16 - mkonecny
15:09:51 <mkonecny> #info 2020-07-23 - siddharthvipul
15:09:51 <mkonecny> #info 2020-07-30 - ?
15:10:01 <mboddu> It would be nice to add some easy fix to help seddik and gchang to go over them
15:10:09 <mboddu> easy fix tickets*
15:10:23 <mkonecny> Any volunteer for 2020-07-30?
15:10:25 <siddharthvipul> mboddu: not just them, me too ;) :P
15:10:46 <mboddu> mkonecny: I can help if no one is
15:10:49 <Tayeh> .fas
15:10:49 <zodbot> Tayeh: (fas <query>) -- Search the Fedora Account System usernames, full names, and email addresses for a match.
15:10:59 <Tayeh> .fas mohamed94
15:10:59 <zodbot> Tayeh: mohamed94 'Mohammed Tayeh' <m.tayeh94@gmail.com>
15:11:08 <mkonecny> mboddu: The show is yours :-)
15:11:21 <mkonecny> #info 2020-07-30 - mboddu
15:11:21 <mboddu> mkonecny: Thanks :)
15:11:42 <mkonecny> #topic announcements and information
15:11:42 <mkonecny> #info CPE Sustaining EU-hours team has a Monday through Friday 30 minute meeting going through tickets at 0830 UTC in #centos-meeting
15:11:43 <mkonecny> #info CPE Sustaining NA-hours team has a Monday through Friday 30 minute meeting going through tickets at 1800 UTC in #fedora-admin
15:11:43 <mkonecny> #info Fedora Infrastructure will be moving in 2020-06 from its Phoenix Az datacenter to one near Herndon Va. A lot of planning will be involved on this. Please watch out for announcements on changes.
15:11:44 <mkonecny> #info Fedora Communishift move has started but will take longer than expected. Current estimate for bringing back into production is TBD
15:12:01 <mkonecny> Anything else to announce, something spicy perhaps ;-)
15:12:26 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/2020-post-datacenter-move-known-issues has a list of known items from post datacenter move
15:12:35 <seddik> mkonecny volunteer  ? about what ?
15:12:45 <siddharthvipul> seddik: for chairing the meeting
15:12:56 <mkonecny> seddik: ^^
15:13:16 <mkonecny> nirik: Added to agenda
15:13:17 <siddharthvipul> seddik: we got mboddu for 30th, we will get back for next sets of meetings on next thursday (in case you want to volunteer)
15:14:14 <mboddu> siddharthvipul: I have never done it, so I am looking at your insights :)
15:14:28 <siddharthvipul> mboddu: we got your back :P
15:14:36 <mkonecny> mboddu: ^^
15:14:45 <mkonecny> To next topic than
15:14:46 <mkonecny> #topic Oncall
15:14:46 <mkonecny> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:14:46 <mkonecny> #info mkonecny is oncall for 2020-07-09 -> 2020-07-16
15:14:46 <mkonecny> #info siddharthvipul  is oncall for 2020-07-16 -> 2020-07-23
15:14:47 <mkonecny> #info ???  is oncall for 2020-07-23 -> 2020-07-30
15:14:53 <mboddu> Thanks siddharthvipul and mkonecny
15:15:10 <mkonecny> Any oncall volunteer for 2020-07-23 -> 2020-07-30
15:15:13 <mkonecny> ?
15:15:20 <mboddu> Sure, why not :D
15:15:25 <mboddu> mkonecny: ^
15:15:26 <siddharthvipul> mboddu is on fire
15:15:31 <siddharthvipul> thank you :")
15:15:33 <mkonecny> Ok, it's yours
15:15:46 <mkonecny> #info mboddu is oncall for 2020-07-23 -> 2020-07-30
15:15:48 <siddharthvipul> .oncalltakeeu
15:15:48 <zodbot> siddharthvipul: Kneel before zod!
15:15:53 <siddharthvipul> \o/
15:16:14 <mkonecny> #info Summary of last week: (from current oncall )
15:16:20 <mboddu> Oh we have mass rebuild scheduled for next wed, Jul 22, fun...
15:16:40 * mboddu should send an announcement
15:16:42 <mkonecny> Ok, so there was pagure.io outage on Friday
15:16:42 <siddharthvipul> mboddu: sleepless nights coming?
15:16:53 <mkonecny> And no ping beside that
15:17:11 <siddharthvipul> i saw that robosignatory was out last night
15:17:14 <mboddu> mkonecny: On that note, robosig died sometime last night, I rebooted it
15:17:22 <mboddu> Sorry, restarted it*
15:17:31 <mkonecny> mboddu: Hm, I didn't got any ping about that
15:17:40 <nirik> we need to ping all the changes that asked for mass rebuild and make sure they are ready. ;(
15:17:52 <mboddu> nirik: Ack
15:17:54 <nirik> mboddu: I also restarted it before I went to bed. ;)
15:18:04 <nirik> so, I guess it died multiple times?
15:18:19 <siddharthvipul> nirik: my night, your day, so not sure :P
15:18:20 <mboddu> nirik: Ohhh....
15:18:32 <siddharthvipul> oh nevermind
15:18:55 <mboddu> mkonecny: It was a ticket in fedora-infrastructure and got pinged in #fedora-admin
15:19:25 <mkonecny> No oncall ping then
15:19:41 <mkonecny> Or it was in time I wasn't around
15:19:50 <mboddu> mkonecny: Nope, just wanted to add to pagure.io being down, robosig also went down
15:20:46 <mkonecny> There was problem creating a ticket in fedora-infrastructure when pagure.io went down :-)
15:21:05 <seddik> i'm not famillar with IRC , is there way to save configuration (channles etc..) ?
15:21:12 <mkonecny> The show will not finish itself, let's move to next topic
15:21:15 <mkonecny> #topic Monitoring discussion [nirik]
15:21:15 <mkonecny> #info https://nagios.fedoraproject.org/nagios
15:21:15 <mkonecny> #info Go over existing out items and fix
15:21:29 <nirik> seddik: depends on client, usually they have a config file or the like?
15:21:34 <nirik> lets see
15:21:48 <seddik> i use the webChat
15:22:02 <seddik> webchat.freenode.net
15:22:23 <nirik> we have 4 aarch64 buildjw's down... but we don't yet have access to the pdu's so we can power cycle them. Thats still in progress...
15:22:29 <seddik> and hexchat on my fedora workstation =D =D
15:23:07 <nirik> the usual swap is low on a bunch of virthosts... (I guess we should reopen that rhel8 bug)
15:23:24 <nirik> a few fedmsg alerts... some to remove, some to clean up
15:24:07 <nirik> and some misc checks that just aren't working... possibly due to python2->python3
15:24:31 <nirik> so, stuff to clean up, but nothing urgent.
15:25:23 <nirik> so, thats it, move on I think...
15:25:31 <mkonecny> thanks nirik
15:25:34 <mkonecny> #topic Data-Center Move update
15:25:43 <mkonecny> Do we have any update here
15:25:45 <mkonecny> ?
15:26:15 <nirik> just the link I provided eariler. Bringing staging and misc machines back up...
15:26:28 <nirik> hopefully next week or so we will start on bringing staging back
15:26:42 <seddik> nirik what about  ALERTS "Swap-Is-Low
15:26:58 <mkonecny> nirik: That's good to hear \o/
15:27:08 <nirik> [08:23:07] <nirik> the usual swap is low on a bunch of virthosts... (I guess we should reopen that rhel8 bug)
15:27:18 <nirik> it's a rhel8 kernel bug... we thought it was fixed, but seems not
15:27:28 <seddik> hum
15:28:38 <nirik> ha, except 08... which seems to have no swap for some reason.
15:28:57 <nirik> anyhow, nothing too urgent, they are all operating fine.
15:29:46 <seddik> @niri
15:29:47 <seddik> Thank you, that's clear.
15:30:01 <seddik> nirik
15:30:25 <mkonecny> So this brings us to the end of this show
15:30:26 <mkonecny> #topic Open Floor
15:30:42 <Tayeh> hi everyone
15:30:51 <mkonecny> hi Tayeh
15:31:00 <Tayeh> i'm reading a lot of docs https://fedora-infra-docs.readthedocs.io/en/latest/sysadmin-guide/index.html
15:31:18 <seddik> hi Tayeh
15:31:31 <Tayeh> and i'm learn ansible now because not work with before
15:32:04 <seddik> ansible == "Swiss knife" ;D
15:32:04 <Tayeh> is there any thing can i help with it now, or still i need to learn more ?
15:32:19 <mkonecny> I'm still finding something new in Ansible every time I work with it
15:32:45 <mkonecny> Yesterday, I learned about strategies
15:32:50 <nirik> yeah, ansible is tons of fun. ;)
15:32:55 <Tayeh> yes
15:32:56 <mboddu> Same here, I got to know there is a dconf module in ansible
15:33:06 <siddharthvipul> some would say glorified bash :P
15:33:11 <siddharthvipul> that's not me though ^ XD
15:33:39 <mboddu> Now I can setup up workstation more easily in the way I want it to (problems/fun with living on rawhide) :D
15:33:52 <Tayeh> all of my work on my company done with bash, this is the first time i learn ansible
15:33:58 <nirik> yeah, setting up your own laptop/machine is a great place to start.
15:34:01 <siddharthvipul> mboddu: that's what got  me started, then it was a lot of automation in internship
15:34:15 <siddharthvipul> nirik: do you have something like that?
15:34:19 <Tayeh> yeah
15:34:48 <nirik> siddharthvipul: I do, but its kinda out of date at this point. I need to do a fresh round of it soon... perhaps redo the laptop with btrfs...
15:35:00 <mboddu> siddharthvipul: I need to update my playbook though
15:35:30 <siddharthvipul> :)
15:35:32 <mboddu> nirik: Uh oh I forgot about btrfs, I need to update my playbook for that as well
15:35:51 <siddharthvipul> mboddu: I will use it with f33 after 5 months :P
15:35:51 <Tayeh> i'm worked as system admin and i have a gull oVirt env. when i need to lean anything i make some VM's and work on it
15:36:18 <Tayeh> *Full
15:36:25 <mkonecny> Tayeh: That's impressive
15:36:28 <siddharthvipul> that's nice
15:36:40 <nirik> I recall in the past people were calling these personal workstation setups lightsabers... ;) since jedi have to make one as part of their training. :)
15:36:52 <mboddu> I forgot about the upgrade, with the upgrade does it switch to btrfs?
15:37:15 <mboddu> nirik: Haha, I am naming my role as lightsabers now :D
15:37:22 <mboddu> ansible role*
15:37:38 <siddharthvipul> mboddu: isn't that what ralph's was called?
15:37:59 <mkonecny> siddharthvipul: He was called a jedi
15:38:08 <mboddu> I dont know
15:38:28 <siddharthvipul> here we go https://github.com/ralphbean/lightsaber
15:38:55 <nirik> yeah, thats the one I was thinking of.
15:39:15 <Tayeh> nice
15:39:59 <siddharthvipul> a lot of people took inspiration from there.. Justin made swiss knife, sayan made elderwand etc
15:40:31 <siddharthvipul> is this the first open floor we are using as open floor :P
15:40:58 <nirik> heh. I'm glad we are... nice discussion.
15:41:20 <Tayeh> :D
15:41:49 <mkonecny> siddharthvipul: I made the magic land of release-monitoring.org :-D
15:42:21 <siddharthvipul> mkonecny: and you did a noble thing sir
15:42:59 * mkonecny wears a pointy hat
15:44:22 <mkonecny> If there is nothing else to talk about, let's end this show
15:45:19 <Tayeh> who can give some simple task to work together to help me learn more with Fedora infrastructure
15:45:46 <seddik> Tayeh a lot of infrastructure tasks are made with ansible
15:46:04 <nirik> we normally have a 'easyfix' tag to mark such tasks... but I think we don't have any right now
15:46:21 <nirik> so, look at tickets, listen in channel for interesting things and offer to help on them.
15:46:35 <Tayeh> ok
15:47:43 <seddik> nirik we use postfix on proxy13 ?
15:48:49 <nirik> yep. we use postfix everywhere.
15:49:31 <seddik> i'm asking this question , tuesday i have saw mail in queue on this server
15:50:05 <seddik> there wasn't much
15:51:27 <nirik> usually that happens if it's been off line and there's emails to root about something...
15:52:27 <seddik> i'm agree with you
15:52:27 <mkonecny> Because we are almost at the ending time I will close the meeting in five minutes
15:56:50 <mkonecny> Thanks to everyone who attended the show today. I hope you enjoyed it and would come next time :-)
15:56:53 <mkonecny> #endmeeting