16:00:01 <nirik> #startmeeting Infrastructure (2021-05-20)
16:00:01 <zodbot> Meeting started Thu May 20 16:00:01 2021 UTC.
16:00:01 <zodbot> This meeting is logged and archived in a public location.
16:00:01 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:01 <zodbot> The meeting name has been set to 'infrastructure_(2021-05-20)'
16:00:01 <nirik> #meetingname infrastructure
16:00:01 <nirik> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak
16:00:01 <zodbot> The meeting name has been set to 'infrastructure'
16:00:01 <zodbot> Current chairs: bodanel dtometzki jnsamyak mobrien nirik pingou siddharthvipul zlopez
16:00:01 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
16:00:02 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:00:03 <nirik> #topic ahoy!
16:00:15 <siddharthvipul> nirik: Good morning :D
16:00:19 <dtometzki> .hello dtometzki
16:00:20 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <linux@tometzki.de>
16:00:26 <dtometzki> hi
16:00:29 <siddharthvipul> .hello siddharthvipul1
16:00:30 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
16:00:38 <darknao> .hi
16:00:39 <zodbot> darknao: darknao 'Francois Andrieu' <naolwen@gmail.com>
16:00:40 <nirik> ahoy everyone! welcome aboard
16:00:54 <Zlopez[m]> .hello zlopez
16:00:55 <zodbot> Zlopez[m]: zlopez 'Michal Konečný' <michal.konecny@psmail.xyz>
16:01:01 <nirik> #topic New folks introductions
16:01:01 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:01:01 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:01:27 <nirik> Any new crew today? lets wait a few in case.
16:01:32 <nils> .hello nphilipp
16:01:33 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com>
16:02:06 <cyberworm54> .hello cyberworm54
16:02:07 <zodbot> cyberworm54: cyberworm54 'None' <cyberworm54@dc317.org>
16:02:55 <bodanel> .hello bodanel
16:02:56 <zodbot> bodanel: bodanel 'Bogdan Benea' <benea_bogdan@yahoo.com>
16:03:43 <nirik> ok, I guess we will sail on then...
16:03:54 <nirik> #topic Next chair
16:03:54 <nirik> #info magic eight ball says:
16:03:54 <nirik> #info chair 2021-05-27 - dtometzki
16:03:54 <nirik> #info chair 2021-06-03 - siddharthvipul
16:03:55 <nirik> #info chair 2021-06-10 - ?
16:04:09 <nirik> anyone want to take the wheel for the 06-10 meeting?
16:04:12 <bodanel> I can do it
16:04:55 <nirik> excellent. thanks!
16:05:16 <nirik> #topic announcements and information
16:05:16 <nirik> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:05:16 <nirik> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:05:16 <nirik> #info work is being done on getting COPR proper powerPC systems
16:05:16 <nirik> #info CPE is hiring! https://communityblog.fedoraproject.org/community-platform-engineering-is-hiring/
16:05:19 <nirik> #info nest with Fedora CFP is open! https://communityblog.fedoraproject.org/announcing-dates-cfp-for-nest-with-fedora/ (Aug 5th-8th)
16:05:22 <nirik> #info mobrien is away 2021-05-10 to 2021-06-20
16:05:31 <nirik> Anyone have more announcements to shout out over the foredeck?
16:06:30 <nirik> ok then...
16:06:33 <nirik> #topic Oncall
16:06:33 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:06:33 <nirik> #info darknao is on call for 2021-05-13 to 2021-05-20
16:06:33 <nirik> #info bodanel is on call for 2021-05-20 to 2021-05-27
16:06:34 <nirik> #info siddharthvipul  is on call for 2021-05-27 to 2021-06-03
16:06:35 <nirik> #info ? is on call for 2021-06-03 to 2021-06-10
16:06:43 <nirik> anyone like to take oncall for 06-03 to 06-10?
16:06:46 <siddharthvipul> who is oncall starting this week?
16:07:00 <dtometzki[m]> i can do it
16:07:00 <siddharthvipul> let me check board
16:07:00 <bodanel> me
16:07:17 <jnsamyak> .hello jnsamyak
16:07:18 <zodbot> jnsamyak: jnsamyak 'Samyak Jain' <samyak.jn11@gmail.com>
16:07:22 <nirik> yeah, it's bodanel. :)
16:07:31 <nirik> darknao: anything to report from the last week?
16:07:39 <nirik> #info Summary of last week: (from current oncall )
16:07:44 <darknao> a few pings
16:07:51 <darknao> # 05.14: pagure down, restarted by smoogen
16:07:59 <darknao> # 05.18: timeouts issue on kojihub, was investigated by nirik at that time
16:08:12 <darknao> # 05.19: reposync broken on freeipa repo between pagure & github (taken care by pingou)
16:08:35 <nirik> awesome. Thanks much for routing all those.
16:08:55 <Zlopez[m]> Nice sumarry
16:09:01 <siddharthvipul> darknao++
16:09:01 <zodbot> siddharthvipul: Karma for darknao changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:09:07 * nirik should try and keep track like that too.
16:09:19 <darknao> I didn't did much
16:09:24 <siddharthvipul> nirik: I thought same too.. next week :D
16:09:39 <siddharthvipul> darknao: you did what was expected and made a nice note of things.. that's great :D
16:09:51 <nirik> yeah.
16:10:01 <nirik> #topic Monitoring discussion [nirik]
16:10:01 <nirik> #info https://nagios.fedoraproject.org/nagios
16:10:01 <nirik> #info Go over existing out items and fix
16:10:14 <nirik> so pretty much its the same as last week... however...
16:10:30 <t0xic0der> .hello t0xic0der
16:10:30 <zodbot> t0xic0der: t0xic0der 'Akashdeep Dhar' <akashdeep.dhar@gmail.com>
16:10:41 <nirik> if people are looking for things to do, there's a few things alerting that they could try and fix:
16:11:03 <nirik> busgateway01.iad2.fedoraproject.org
16:11:03 <nirik> 
16:11:03 <nirik> 
16:11:03 <nirik> Check datanommer for recent faf messages
16:11:03 <nirik> 
16:11:04 <nirik> This service has 1 comment associated with it	This service problem has been acknowledged
16:11:06 <nirik> CRITICAL 	05-20-2021 16:05:11 	48d 20h 20m 5s 	3/3 	CRIT: datanommer has not seen a 'faf' message in 339 days, 21 hours, 20 minutes, 42 seconds
16:11:09 <nirik> that could just be removed.
16:11:42 <nirik> mbs-backend01.iad2.fedoraproject.org Check fedmsg consumers and producers hub This service has 1 comment associated with it	This service problem has been acknowledged	CRITICAL 	05-20-2021 16:07:59 	14d 22h 32m 24s 	3/3 	ERROR: MBSProducer not found among installed plugins
16:11:53 <nirik> that one too. mbs was upgraded and no longer has that producer
16:12:39 <nirik> pagure02.fedoraproject.org Systemd Unitt This service has 1 comment associated with it	This service problem has been acknowledged CRITICAL 	05-20-2021 16:06:57 	55d 19h 2m 16s 	3/3 	CHECK_NRPE STATE CRITICAL: Socket timeout after 30 seconds.
16:12:45 <nirik> that one has a ticket associated with it:
16:13:06 <nirik> https://pagure.io/fedora-infrastructure/issue/9461
16:13:35 <nirik> seddik was working on it, but they haven't been around much, so if someone else wanted to take it over that would be great. ;)
16:13:52 <nirik> all those could be Pull requests to fix the issue/remove the thing in anisble. :)
16:14:27 <nirik> any questions on that?
16:15:25 <dtometzki[m]> we will try
16:15:32 <darknao> I can take a look too
16:16:06 <nirik> awesome. thanks
16:16:27 <nirik> So, we don't have a learning topic that I see... perhaps we could brainstorm/setup some more.
16:16:38 <nirik> #topic learning topics needed
16:16:52 <nirik> what would folks like to present or hear about?
16:17:04 <bodanel> provisioning new vm process ?
16:17:25 <nirik> bodanel: sure! Thats a good one...
16:17:28 <bodanel> would be nice to hear how everything is tight together
16:17:53 <jnsamyak> Also, one question from last topic, how can we test changes made to ansible repository! 😬
16:18:15 <bodanel> you can run dry run the ansible
16:18:28 <bodanel> with your branch
16:18:30 <dtometzki[m]> how ?
16:19:07 <nirik> not too easily. ;) There is --syntax-check for base syntax checking... and then we often run things against staging first... with -l staging
16:19:08 <bodanel> ansible has the -C option which basically will show you what will change but will NOT change anything
16:19:14 <dtometzki[m]> local or where ?
16:19:33 <jnsamyak> Can you elaborate once please, I just took help of bot for now if the everything was succeeding or not
16:19:38 <nirik> yeah, there is --check and --diff, but rbac-playbook doesn't support that I don't think (because it would/could leak private info)
16:21:25 <jnsamyak> Oh okay, thanks for the answer, I'll give both things a try, just to be double sure
16:21:34 <nirik> we don't really have a great story for testing/checking stuff. ;(
16:21:53 <Zlopez[m]> I think in past we talked about pull request test run, but I'm not sure if we got to any resolution about this
16:22:09 <nirik> we do run some tests on PR's via zuul...
16:22:12 <t0xic0der> @bodanel: The -C flag does not work with all modules, does it? Depending upon commands available at the control machine and on the remote machine...
16:22:16 <nirik> but it's basic syntax / linting
16:23:04 <jnsamyak> Is there any work around to it for future, that we can integrate for enabling testing more precisely?
16:23:23 <nirik> there is molecule, but setting that up could be a ton of work.
16:23:29 <jnsamyak> <nirik "we do run some tests on PR's via"> Yes zuul bot for the worth zuul++ :")
16:23:29 <zodbot> jnsamyak: Karma for zuul changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:24:20 <bodanel> @t0xic0der I think so but from personal experience it will do the job in most cases
16:24:49 <nirik> Any other learning topics anyone wants to speak on? :)
16:25:03 <darknao> I have one question (completly unrelated), but how is planned openshift upgrade ?
16:25:18 <darknao> I mean, ocp3.11 upgrade
16:25:50 <darknao> the version we are using is almost a year old, and I was wondering if we are doing any minor upgrade usually
16:26:00 <Zlopez[m]> nirik: molecule is really nice tool
16:26:00 <nirik> darknao: you mean 3 -> 4? or 3.11.N -> 3.11.N+1
16:26:21 <darknao> -> 3.11.xxx
16:26:22 <Zlopez[m]> But yeah this would be big amount of work
16:26:25 <nirik> The plan is to move to 4. We have some hardware to do so...
16:26:44 <nirik> for 3.x minor upgrades, just when desired I have been doing them (via openshift-ansible)
16:26:55 <nirik> I can do a update cycle on them soon.
16:27:07 <dtometzki[m]> Zlopez[m] for what is molecule ?
16:27:54 <bodanel> can do a bunch of tests, never used personally but other colleagues like the tool also
16:27:58 <Zlopez[m]> It allows you to write unit tests for ansible scripts
16:28:17 <Zlopez[m]> We used it in mbbox project
16:28:26 <nirik> it creates containers, runs the ansible in the container and checks it against assertions you defined.
16:29:20 <Zlopez[m]> MBBox is basically a kubernetes operator, basically a bunch of ansible roles https://github.com/fedora-infra/mbbox
16:30:40 <nirik> #topic Open Floor
16:30:53 <darknao> is there anything we can do to help on the ocp4 upgrade topic ? or is anything already being taken care of ?
16:30:57 <nirik> anyone have any last discussion/thoughts before we pull into port?
16:31:30 <bodanel> I would still love to hear about the process for spining up new VM's
16:31:31 <nirik> darknao: might be at some point. I am meeting with networking later today to get the hardware on the right vlans, etc... then we just need to install it and test things. :)
16:31:46 <bodanel> technical curiosity
16:31:52 <cyberworm54> are the meeting going to take place here  (freenode)for the time being? Just want to make sure.
16:32:22 <nirik> cyberworm54: for now. Likely if we move, everything will move and leave behind redirects to the new place.
16:32:54 <cyberworm54> I see, thanks!
16:33:17 <darknao> nirik: cool, let me know if you need some help :)
16:33:24 <nirik> darknao: will do!
16:33:43 <nirik> If nothing else will close out in a minute...
16:34:36 <jnsamyak> Just to mention, I would like to thank bodanel for helping me out to start with one of the issues for ansible :")
16:34:51 <nirik> bodanel++
16:34:51 <zodbot> nirik: Karma for bodanel changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:35:08 <jnsamyak> bodanel+++
16:35:44 <bodanel> jnsamyak, thks for working on it
16:35:53 <bodanel> jnsamya++
16:35:58 <bodanel> jnsamyak++
16:36:01 <zodbot> bodanel: Karma for jnsamyak changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:36:06 <nirik> #endmeeting