16:00:01 <nirik> #startmeeting Infrastructure (2021-05-20) 16:00:01 <zodbot> Meeting started Thu May 20 16:00:01 2021 UTC. 16:00:01 <zodbot> This meeting is logged and archived in a public location. 16:00:01 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:01 <zodbot> The meeting name has been set to 'infrastructure_(2021-05-20)' 16:00:01 <nirik> #meetingname infrastructure 16:00:01 <nirik> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak 16:00:01 <zodbot> The meeting name has been set to 'infrastructure' 16:00:01 <zodbot> Current chairs: bodanel dtometzki jnsamyak mobrien nirik pingou siddharthvipul zlopez 16:00:01 <nirik> #info Agenda is at: https://board.net/p/fedora-infra 16:00:02 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/ 16:00:03 <nirik> #topic ahoy! 16:00:15 <siddharthvipul> nirik: Good morning :D 16:00:19 <dtometzki> .hello dtometzki 16:00:20 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <linux@tometzki.de> 16:00:26 <dtometzki> hi 16:00:29 <siddharthvipul> .hello siddharthvipul1 16:00:30 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com> 16:00:38 <darknao> .hi 16:00:39 <zodbot> darknao: darknao 'Francois Andrieu' <naolwen@gmail.com> 16:00:40 <nirik> ahoy everyone! welcome aboard 16:00:54 <Zlopez[m]> .hello zlopez 16:00:55 <zodbot> Zlopez[m]: zlopez 'Michal Konečný' <michal.konecny@psmail.xyz> 16:01:01 <nirik> #topic New folks introductions 16:01:01 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves 16:01:01 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted 16:01:27 <nirik> Any new crew today? lets wait a few in case. 16:01:32 <nils> .hello nphilipp 16:01:33 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com> 16:02:06 <cyberworm54> .hello cyberworm54 16:02:07 <zodbot> cyberworm54: cyberworm54 'None' <cyberworm54@dc317.org> 16:02:55 <bodanel> .hello bodanel 16:02:56 <zodbot> bodanel: bodanel 'Bogdan Benea' <benea_bogdan@yahoo.com> 16:03:43 <nirik> ok, I guess we will sail on then... 16:03:54 <nirik> #topic Next chair 16:03:54 <nirik> #info magic eight ball says: 16:03:54 <nirik> #info chair 2021-05-27 - dtometzki 16:03:54 <nirik> #info chair 2021-06-03 - siddharthvipul 16:03:55 <nirik> #info chair 2021-06-10 - ? 16:04:09 <nirik> anyone want to take the wheel for the 06-10 meeting? 16:04:12 <bodanel> I can do it 16:04:55 <nirik> excellent. thanks! 16:05:16 <nirik> #topic announcements and information 16:05:16 <nirik> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting 16:05:16 <nirik> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3 16:05:16 <nirik> #info work is being done on getting COPR proper powerPC systems 16:05:16 <nirik> #info CPE is hiring! https://communityblog.fedoraproject.org/community-platform-engineering-is-hiring/ 16:05:19 <nirik> #info nest with Fedora CFP is open! https://communityblog.fedoraproject.org/announcing-dates-cfp-for-nest-with-fedora/ (Aug 5th-8th) 16:05:22 <nirik> #info mobrien is away 2021-05-10 to 2021-06-20 16:05:31 <nirik> Anyone have more announcements to shout out over the foredeck? 16:06:30 <nirik> ok then... 16:06:33 <nirik> #topic Oncall 16:06:33 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall 16:06:33 <nirik> #info darknao is on call for 2021-05-13 to 2021-05-20 16:06:33 <nirik> #info bodanel is on call for 2021-05-20 to 2021-05-27 16:06:34 <nirik> #info siddharthvipul is on call for 2021-05-27 to 2021-06-03 16:06:35 <nirik> #info ? is on call for 2021-06-03 to 2021-06-10 16:06:43 <nirik> anyone like to take oncall for 06-03 to 06-10? 16:06:46 <siddharthvipul> who is oncall starting this week? 16:07:00 <dtometzki[m]> i can do it 16:07:00 <siddharthvipul> let me check board 16:07:00 <bodanel> me 16:07:17 <jnsamyak> .hello jnsamyak 16:07:18 <zodbot> jnsamyak: jnsamyak 'Samyak Jain' <samyak.jn11@gmail.com> 16:07:22 <nirik> yeah, it's bodanel. :) 16:07:31 <nirik> darknao: anything to report from the last week? 16:07:39 <nirik> #info Summary of last week: (from current oncall ) 16:07:44 <darknao> a few pings 16:07:51 <darknao> # 05.14: pagure down, restarted by smoogen 16:07:59 <darknao> # 05.18: timeouts issue on kojihub, was investigated by nirik at that time 16:08:12 <darknao> # 05.19: reposync broken on freeipa repo between pagure & github (taken care by pingou) 16:08:35 <nirik> awesome. Thanks much for routing all those. 16:08:55 <Zlopez[m]> Nice sumarry 16:09:01 <siddharthvipul> darknao++ 16:09:01 <zodbot> siddharthvipul: Karma for darknao changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 16:09:07 * nirik should try and keep track like that too. 16:09:19 <darknao> I didn't did much 16:09:24 <siddharthvipul> nirik: I thought same too.. next week :D 16:09:39 <siddharthvipul> darknao: you did what was expected and made a nice note of things.. that's great :D 16:09:51 <nirik> yeah. 16:10:01 <nirik> #topic Monitoring discussion [nirik] 16:10:01 <nirik> #info https://nagios.fedoraproject.org/nagios 16:10:01 <nirik> #info Go over existing out items and fix 16:10:14 <nirik> so pretty much its the same as last week... however... 16:10:30 <t0xic0der> .hello t0xic0der 16:10:30 <zodbot> t0xic0der: t0xic0der 'Akashdeep Dhar' <akashdeep.dhar@gmail.com> 16:10:41 <nirik> if people are looking for things to do, there's a few things alerting that they could try and fix: 16:11:03 <nirik> busgateway01.iad2.fedoraproject.org 16:11:03 <nirik> 16:11:03 <nirik> 16:11:03 <nirik> Check datanommer for recent faf messages 16:11:03 <nirik> 16:11:04 <nirik> This service has 1 comment associated with it This service problem has been acknowledged 16:11:06 <nirik> CRITICAL 05-20-2021 16:05:11 48d 20h 20m 5s 3/3 CRIT: datanommer has not seen a 'faf' message in 339 days, 21 hours, 20 minutes, 42 seconds 16:11:09 <nirik> that could just be removed. 16:11:42 <nirik> mbs-backend01.iad2.fedoraproject.org Check fedmsg consumers and producers hub This service has 1 comment associated with it This service problem has been acknowledged CRITICAL 05-20-2021 16:07:59 14d 22h 32m 24s 3/3 ERROR: MBSProducer not found among installed plugins 16:11:53 <nirik> that one too. mbs was upgraded and no longer has that producer 16:12:39 <nirik> pagure02.fedoraproject.org Systemd Unitt This service has 1 comment associated with it This service problem has been acknowledged CRITICAL 05-20-2021 16:06:57 55d 19h 2m 16s 3/3 CHECK_NRPE STATE CRITICAL: Socket timeout after 30 seconds. 16:12:45 <nirik> that one has a ticket associated with it: 16:13:06 <nirik> https://pagure.io/fedora-infrastructure/issue/9461 16:13:35 <nirik> seddik was working on it, but they haven't been around much, so if someone else wanted to take it over that would be great. ;) 16:13:52 <nirik> all those could be Pull requests to fix the issue/remove the thing in anisble. :) 16:14:27 <nirik> any questions on that? 16:15:25 <dtometzki[m]> we will try 16:15:32 <darknao> I can take a look too 16:16:06 <nirik> awesome. thanks 16:16:27 <nirik> So, we don't have a learning topic that I see... perhaps we could brainstorm/setup some more. 16:16:38 <nirik> #topic learning topics needed 16:16:52 <nirik> what would folks like to present or hear about? 16:17:04 <bodanel> provisioning new vm process ? 16:17:25 <nirik> bodanel: sure! Thats a good one... 16:17:28 <bodanel> would be nice to hear how everything is tight together 16:17:53 <jnsamyak> Also, one question from last topic, how can we test changes made to ansible repository! 😬 16:18:15 <bodanel> you can run dry run the ansible 16:18:28 <bodanel> with your branch 16:18:30 <dtometzki[m]> how ? 16:19:07 <nirik> not too easily. ;) There is --syntax-check for base syntax checking... and then we often run things against staging first... with -l staging 16:19:08 <bodanel> ansible has the -C option which basically will show you what will change but will NOT change anything 16:19:14 <dtometzki[m]> local or where ? 16:19:33 <jnsamyak> Can you elaborate once please, I just took help of bot for now if the everything was succeeding or not 16:19:38 <nirik> yeah, there is --check and --diff, but rbac-playbook doesn't support that I don't think (because it would/could leak private info) 16:21:25 <jnsamyak> Oh okay, thanks for the answer, I'll give both things a try, just to be double sure 16:21:34 <nirik> we don't really have a great story for testing/checking stuff. ;( 16:21:53 <Zlopez[m]> I think in past we talked about pull request test run, but I'm not sure if we got to any resolution about this 16:22:09 <nirik> we do run some tests on PR's via zuul... 16:22:12 <t0xic0der> @bodanel: The -C flag does not work with all modules, does it? Depending upon commands available at the control machine and on the remote machine... 16:22:16 <nirik> but it's basic syntax / linting 16:23:04 <jnsamyak> Is there any work around to it for future, that we can integrate for enabling testing more precisely? 16:23:23 <nirik> there is molecule, but setting that up could be a ton of work. 16:23:29 <jnsamyak> <nirik "we do run some tests on PR's via"> Yes zuul bot for the worth zuul++ :") 16:23:29 <zodbot> jnsamyak: Karma for zuul changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 16:24:20 <bodanel> @t0xic0der I think so but from personal experience it will do the job in most cases 16:24:49 <nirik> Any other learning topics anyone wants to speak on? :) 16:25:03 <darknao> I have one question (completly unrelated), but how is planned openshift upgrade ? 16:25:18 <darknao> I mean, ocp3.11 upgrade 16:25:50 <darknao> the version we are using is almost a year old, and I was wondering if we are doing any minor upgrade usually 16:26:00 <Zlopez[m]> nirik: molecule is really nice tool 16:26:00 <nirik> darknao: you mean 3 -> 4? or 3.11.N -> 3.11.N+1 16:26:21 <darknao> -> 3.11.xxx 16:26:22 <Zlopez[m]> But yeah this would be big amount of work 16:26:25 <nirik> The plan is to move to 4. We have some hardware to do so... 16:26:44 <nirik> for 3.x minor upgrades, just when desired I have been doing them (via openshift-ansible) 16:26:55 <nirik> I can do a update cycle on them soon. 16:27:07 <dtometzki[m]> Zlopez[m] for what is molecule ? 16:27:54 <bodanel> can do a bunch of tests, never used personally but other colleagues like the tool also 16:27:58 <Zlopez[m]> It allows you to write unit tests for ansible scripts 16:28:17 <Zlopez[m]> We used it in mbbox project 16:28:26 <nirik> it creates containers, runs the ansible in the container and checks it against assertions you defined. 16:29:20 <Zlopez[m]> MBBox is basically a kubernetes operator, basically a bunch of ansible roles https://github.com/fedora-infra/mbbox 16:30:40 <nirik> #topic Open Floor 16:30:53 <darknao> is there anything we can do to help on the ocp4 upgrade topic ? or is anything already being taken care of ? 16:30:57 <nirik> anyone have any last discussion/thoughts before we pull into port? 16:31:30 <bodanel> I would still love to hear about the process for spining up new VM's 16:31:31 <nirik> darknao: might be at some point. I am meeting with networking later today to get the hardware on the right vlans, etc... then we just need to install it and test things. :) 16:31:46 <bodanel> technical curiosity 16:31:52 <cyberworm54> are the meeting going to take place here (freenode)for the time being? Just want to make sure. 16:32:22 <nirik> cyberworm54: for now. Likely if we move, everything will move and leave behind redirects to the new place. 16:32:54 <cyberworm54> I see, thanks! 16:33:17 <darknao> nirik: cool, let me know if you need some help :) 16:33:24 <nirik> darknao: will do! 16:33:43 <nirik> If nothing else will close out in a minute... 16:34:36 <jnsamyak> Just to mention, I would like to thank bodanel for helping me out to start with one of the issues for ansible :") 16:34:51 <nirik> bodanel++ 16:34:51 <zodbot> nirik: Karma for bodanel changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 16:35:08 <jnsamyak> bodanel+++ 16:35:44 <bodanel> jnsamyak, thks for working on it 16:35:53 <bodanel> jnsamya++ 16:35:58 <bodanel> jnsamyak++ 16:36:01 <zodbot> bodanel: Karma for jnsamyak changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 16:36:06 <nirik> #endmeeting