15:00:02 <pingou> #startmeeting Infrastructure (2019-10-17) 15:00:02 <zodbot> Meeting started Thu Oct 17 15:00:02 2019 UTC. 15:00:02 <zodbot> This meeting is logged and archived in a public location. 15:00:02 <zodbot> The chair is pingou. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:02 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:02 <zodbot> The meeting name has been set to 'infrastructure_(2019-10-17)' 15:00:05 <pingou> #meetingname infrastructure 15:00:05 <zodbot> The meeting name has been set to 'infrastructure' 15:00:07 <pingou> #topic aloha 15:00:09 <pingou> #chair nirik pingou relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs 15:00:09 <zodbot> Current chairs: abompard bowlofeggs cverna mizdebsk mkonecny nirik pingou relrod smooge tflink 15:00:23 <bowlofeggs> .hello2 15:00:24 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com> 15:00:33 <nirik> morning 15:00:37 <cverna> morning 15:01:05 <marc84> morning everyone 15:01:10 <relrod> morning 15:01:12 <pyratebeard> hi all 15:02:14 <dustymabe> .hello2 15:02:15 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com> 15:02:17 <mkonecny> .hello zlopez 15:02:18 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu> 15:02:24 <abompard> .hello2 15:02:27 <zodbot> abompard: abompard 'Aurelien Bompard' <aurelien@bompard.org> 15:02:30 <pingou> I'll wait another 30s and we'll get started 15:03:01 <pingou> #topic Next chair 15:03:12 <pingou> I've been wondering if we should tie this to the oncall 15:03:24 <pingou> and make the oncall be the chair at the end of their week 15:03:59 <pingou> would this make things simpler or would this make it so that it comes down to the same people too frequently? 15:04:05 <dustymabe> :) - insult to injury? 15:04:25 <dustymabe> how about next weeks oncall is this weeks chair ? 15:04:27 <nirik> on the one hand they would know a lot about what happened that last week 15:04:33 <nirik> on the other hand... more work for them 15:04:46 <cverna> the pool of people going oncall is quite limited, I think more people could run the meeting 15:04:53 <nirik> cverna: +1 15:04:56 <cverna> for example apprentice could run the meeting 15:05:01 <pingou> fair 15:05:06 <dustymabe> cverna++ 15:05:10 <pingou> so do we have any volunteer for next week? :) 15:05:14 <dustymabe> that's actually a great idea for an apprentice 15:05:26 <cverna> yes :-) 15:05:29 <bgilbert> .hello2 15:05:30 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net> 15:05:36 * nirik will not be here for next weeks meeting. 15:06:18 <bowlofeggs> dustymabe should host next week 15:06:41 <pingou> and you the week after? 15:07:10 * dustymabe mostly lurks - but I can run the meeting - not sure I can sign up for action items to follow up though 15:07:11 * pingou happy to have two weeks planned 15:07:12 * cverna can take next week 15:07:16 <dustymabe> but I'm good at assigning action items :) 15:07:38 <pingou> so cverna or dustymabe ? :) 15:07:58 <dustymabe> how about cverna and then dustymabe 15:08:05 <pingou> deal! 15:08:05 <cverna> me me me :) 15:08:20 * dustymabe always refers to himself with both first and last name 15:08:28 <pingou> #info magic eight ball says: cverna on October 24th then dustymabe on October 31st 15:08:35 <pingou> #topic New folks introductions 15:08:38 <pingou> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves 15:08:40 <pingou> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted 15:08:44 <pingou> do we have any new folks joining us today? 15:09:17 <austinpowered> I guess I would still be considered new 15:09:56 <dustymabe> austinpowered++ 15:09:56 <zodbot> dustymabe: Karma for austinpowered changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:10:14 <pingou> did you introduce yourself to a meeting already? (sorry if you did and I don't remember :( ) 15:10:52 <austinpowered> I did not. I stepped afk for a moment at the start. Back now...; 15:11:19 <nirik> welcome 15:11:23 <pingou> welcome then :) 15:11:27 <joalon> Welcome! 15:11:39 <cverna> welcome back :-P 15:11:46 <pingou> do you want to introduce yourself quickly here or on the list? Are you more interested in sys-admin or dev? 15:12:50 <austinpowered> sys-admin - I've been trying to get my humble home lab to be infrastructure as code with ansible 15:13:23 <pingou> sounds nice, are you already in the apprentice group? 15:14:12 <austinpowered> been pouring through the fedora-admin/ansible files 15:14:21 <austinpowered> yes ot apprentive group 15:14:26 <pingou> cool :) 15:14:34 <austinpowered> yes to apprentice group 15:14:37 <pingou> let us know if we can help you get started on anything 15:14:47 <pingou> or if you're blocked 15:16:02 <austinpowered> just drinking from the fire hose - could probably use some guidance - I'll ask in fedoara-admin after this meeting 15:16:09 <pingou> sounds good :) 15:16:19 <pingou> so let's move to announcements 15:16:20 <pingou> #topic announcements and information 15:16:26 <pingou> #info We are looking for people to maintain Fedocal - mkonecny 15:16:29 <pingou> #info CPE Team - Post Flock report was published (https://communityblog.fedoraproject.org/cpe-team-at-flock-post-flock/) - mkonecny 15:16:31 <pingou> #info f31 final freeze now in effect 15:16:33 <pingou> #info kevin/nirik on PTO from 2019-10-23 to 2019-10-27 15:16:35 <pingou> #info robosignatory running with fedora-messaging deployed in stg and prod. 15:16:37 <pingou> #info koji updated for https://docs.pagure.org/koji/CVE-2019-17109/ 15:16:39 <pingou> #info work ongoing to test fasClient changes to make it work on f31 and el8 15:16:41 <pingou> #info work ongoing to deprecate ModernPaste pastebin - new fpaste client ready for testing: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/GCHZLJSBWJIQERU524P67OMVWZTIOMN4/ - report issues to Ankur/FranciscoD 15:16:43 <pingou> anything else we'd like to add ? 15:16:57 <pingou> #info multi-packages gating has started in staging 15:17:19 <pingou> #info single-package gating working end to end in staging 15:17:20 <mkonecny> #info Fedocal already found someone, I just forgot to update meeting 15:17:30 <nirik> yeah, great news. ;) 15:17:50 <pingou> mkonecny: removing it from the board then :) 15:18:12 <pingou> any more new info? 15:18:15 <mkonecny> I will update it 15:18:21 <pingou> mkonecny: just did ;-0) 15:18:23 <relrod> #info fasClient for f31 and el8 are actually in the infra repo now, but relrod forgot to update the meeting document to state that above :) 15:18:37 <nirik> awesome. :) Thanks for working on that... 15:18:48 <cverna> relrod++ 15:18:48 <nirik> relrod: I guess go ahead and close that ticket? 15:18:48 <pingou> removed 15:18:50 <pingou> relrod++ 15:18:54 <relrod> nirik: ah yeah 15:19:23 <nirik> also f32 I think exists... we have one rawhide machine. ;) 15:19:45 <relrod> yeah there's an f32 build in the infra repo as well 15:20:21 <nirik> ah, stg... thats why I didn't see it. ;) 15:20:41 <relrod> oh I can fix that 15:21:04 <pingou> next topic? 15:21:46 <pingou> #topic Oncall 15:21:49 <pingou> #info https://fedoraproject.org/wiki/Infrastructure/Oncall 15:21:51 <pingou> #info pingou is on call 2019-10-10->2019-10-17 15:21:53 <pingou> #info smooge is on call 2019-10-17->2019-10-24 15:21:55 <pingou> #info relrod is on call 2019-10-24->2019-10-31 15:22:00 <pingou> do we have anyone for the first week of November? 15:22:12 <nirik> I can do that one... 15:22:35 <nirik> unless soemone else wants to. ;) 15:22:39 <pingou> anyone? 15:23:44 <nirik> looks like me. ;) 15:23:45 <pingou> nirik: sorry looks like you're the lucky winner 15:23:48 <pingou> #info nirik is on call 2019-10-31->2019-11-07 15:23:51 <pingou> #info Summary of last week: (from pingou ) 15:23:54 <pingou> #info No pings to me so quiet week here 15:24:12 <pingou> I'm not sure if everything is fine, people are using tickets for everything or by-passing the oncall mechanism 15:25:06 <pingou> ok moving on 15:25:09 <pingou> #topic Monitoring discussion 15:25:11 <pingou> #info https://nagios.fedoraproject.org/nagios 15:25:36 <nirik> lets see... 15:25:42 <nirik> most are known 15:25:50 <mkonecny> pingou: There is still freeze going on 15:26:08 <pingou> mkonecny: likely helps 15:26:08 <nirik> notifs-backend01 is getting a full disk. :( I looked and it seems like it's journal logs of all the stuff it's sending? 15:26:17 <nirik> not sure what to do about that 15:26:52 <pingou> we can likely trim them down 15:27:55 <nirik> Filesystem Size Used Avail Use% Mounted on 15:27:55 <nirik> /dev/mapper/GuestVolGroup00-root 15G 14G 827M 95% / 15:28:14 <nirik> there's 863MB of /var/log/ 15:28:36 <pingou> less than 1G out of 14 though 15:28:42 <pingou> but still pretty big 15:28:47 <nirik> we could just make it bigger, but I'd hate to reinstall it... 15:29:00 <nirik> I guess we could just resize it up... 15:29:13 <marc84> nirik +1 15:29:44 <nirik> I mean I'd love to actually just replace it with a new implementation, but thats going to take a while 15:30:02 <pingou> the disk may be full before that 15:30:03 <nirik> anyhow, thats the only non known one I see... so we can move on. 15:30:11 <nirik> likely. ;) 15:30:19 <nirik> I can send in a freeze break to resize it... 15:30:33 <pingou> let's move to tickets 15:30:36 <pingou> #topic Tickets discussion 15:30:38 <pingou> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket 15:30:40 <pingou> #info no tickets marked for the meeting 15:30:42 <pingou> #topic Tickets discussion 15:30:44 <pingou> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket 15:30:46 <pingou> #info no tickets marked for the meeting 15:30:48 <pingou> oups 15:30:50 <pingou> #topic Tickets discussion 15:30:52 <pingou> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket 15:30:54 <pingou> #info no tickets marked for the meeting 15:30:56 <pingou> raah 15:30:58 <pingou> #info Fedora CoreOS Team requests 15:30:59 <pingou> #link https://hackmd.io/5uB7hOJKSjGUt65iLgPnbA#Existing-requests-for-Fedora-Infra 15:31:02 <pingou> there! 15:31:04 * dustymabe waves 15:31:08 * bgilbert waves 15:31:15 <dustymabe> this is a running list I'm keeping 15:31:21 <dustymabe> we have a few new tickets since last time 15:31:27 <pingou> AWS IAM permissions: https://pagure.io/fedora-infrastructure/issue/8142 15:31:36 <pingou> coreos apps in communishift: https://pagure.io/fedora-infrastructure/issue/8218 15:31:49 <pingou> communishift /dev/kvm: https://pagure.io/fedora-infrastructure/issue/8208 15:32:03 <pingou> GCP access: https://pagure.io/fedora-infrastructure/issue/8281 15:32:10 <pingou> new AWS region: https://pagure.io/fedora-infrastructure/issue/8299 15:32:14 <dustymabe> thanks pingou 15:32:29 <dustymabe> i don't know where to begin :) 15:32:37 <dustymabe> one at a time I guess 15:32:48 * nirik knows of most of these... but I have had higher prio stuff. 15:32:53 <nirik> so, not sure what to say either. 15:33:02 <dustymabe> [AWS IAM permissions] - this is something kevin has been working on, but is not high priority right now 15:33:05 <dustymabe> we're not blocked there 15:33:10 <pingou> dustymabe: could you order the hackmd by priority? 15:33:17 <nirik> perhaps... yeah, that ^ 15:33:35 <dustymabe> [coreos apps in communishift] we mostly just need some docs on how to get users added to a project 15:33:42 <dustymabe> pingou: yes I can for next time 15:33:47 <pingou> thanks 15:34:14 <dustymabe> [communishift /dev/kvm] this would be really nice to have so we can start using communishift for some more advanced testing/development projects 15:34:37 <dustymabe> [GCP access] this one is new - can we get some eyes on it? 15:35:00 <dustymabe> [new AWS region] this one is new too - who do we need to talk to to get other regions enabled ? 15:35:21 <bgilbert> * assuming Infra is comfortable doing so 15:35:42 * nirik hasn't seen that one, filed last night after I went to sleep I guess? 15:36:09 <bgilbert> the last one? yeah 15:36:24 <dustymabe> nirik: not expecting any answers today - just want to raise them as something we can look at sooner than later 15:36:39 <nirik> sure, happy to. 15:36:39 <bgilbert> +1 15:37:02 <nirik> off hand, I don't see any reason not to enable that opt in... seems fine to me. I was unaware it existed. ;) 15:38:09 <pingou> should we move to the backlog? 15:38:28 <dustymabe> nirik: regarding GCP - is there one person in particular we can work with? 15:39:01 <nirik> I thought we were waiting on something there, let me look at the ticket 15:39:19 <nirik> oh, question on credits... 15:39:43 <nirik> but I guess that doesn't block the account setup... 15:39:54 <bgilbert> right. we're still waiting for them to get back to us 15:40:00 <dustymabe> yes so it looks like either credits or a credit card ? 15:40:12 <bgilbert> but the amount of money will be small at first, in the worst case 15:40:35 <dustymabe> bgilbert: do you know if they require a card even if we have credits ? 15:40:45 <bgilbert> I don't know 15:40:49 <nirik> I'll talk to the holder of the purse(s) and see how we want to set things up. 15:40:57 <bgilbert> +1 15:40:58 <smooge> my understanding is that they will require a card or a PO 15:41:10 <dustymabe> nirik: +1 - loop me in on that if you need any help 15:41:49 <smooge> this is to make sure they are covered in case you put something which burns through the credits etc 15:41:58 <dustymabe> smooge: yeah, makes sense 15:42:16 <nirik> sure, can do 15:43:26 <pingou> next topic? 15:43:39 <nirik> topic ++ 15:43:41 <cverna> +1 15:43:46 <pingou> #topic backlog discussion 15:43:49 <pingou> #info go over our backlog and discuss and determine priority 15:43:51 <pingou> #link https://pagure.io/fedora-infrastructure/issue/8178 15:43:53 <pingou> #info topic: provision new aarch64 builders 15:43:55 <pingou> #link https://pagure.io/fedora-infrastructure/issue/8157 15:43:57 <pingou> #info topic: ansible: enable ansible-report as a hook 15:43:59 <pingou> so these two have been there for a few weeks now 15:44:09 <pingou> are there any we want to add/remove? 15:44:21 <cverna> I have not add a chance to work on the hook :-( 15:44:44 <cverna> s/add/had 15:45:24 <nirik> on the aarch64 stuff smooge has been poking at it. 15:45:38 <nirik> there's some progress, but we were also stalled by a dnf bug. 15:46:00 <nirik> (armv7 guests wouldn't work / couldn't install or upgrade anything) 15:46:31 <cverna> oh that's a dnf issue ? I guess that the cause for the container to fail building on armv7 15:46:36 <nirik> but yeah, we are doing poorly at finishing backlogs 15:47:05 <nirik> cverna: I don't think it's the same issue... this only affects newer hardware. The older should keep working I think... 15:47:16 <cverna> ah ok :-) 15:48:04 <cverna> yes not sure how to make the backlog item more visible, so we can focus more on them 15:48:29 <nirik> I think more scheduling might help 15:48:32 <pingou> and it's only a part of it, little projects such as gating rawhide are taking a bit of our time :) 15:48:45 <nirik> "I am going to work on X at this time on this channel, join if you want to follow/help" 15:49:07 <nirik> not to mention unplanned work like this koji db stuff. ;( 15:49:17 <pingou> yup 15:49:34 <pingou> I don't think there is anything else to do there 15:49:38 <pingou> open-floor? 15:49:46 <cverna> happy to try what nirik said "I am going to work ....", do we use the infra list to announce that 15:49:55 <pingou> infra list +1 15:50:02 <dustymabe> +1 15:50:10 <nirik> cverna: +1... I was trying to use ticket, but they might not be as visible. 15:50:15 <nirik> I suppose both would be best... 15:50:16 <dustymabe> even if you get nothing back I can promise it's useful for transparency 15:50:39 <cverna> ok so let's try that :-) 15:51:02 <cverna> and we can do both list and tickets 15:51:52 <nirik> sounds good 15:52:06 <austinpowered> the ticket is where progress is documented and issues discussed. The list should be used to draw attention to the ticket. 15:52:25 <austinpowered> But I'm the new guy.... 15:53:09 <austinpowered> I would expect to find everything about the issue in the ticket 15:53:17 <cverna> yes +1 15:53:23 <nirik> yep. 15:53:29 <dustymabe> For open floor I have two items - let me know when I should go 15:54:28 <pingou> let's move to open-floor 15:54:31 <pingou> #topic Open Floor 15:54:32 <pingou> dustymabe: go for it! 15:55:11 <dustymabe> First off: 15:55:31 <dustymabe> This meeting has been amazing. I'm so happy to see so many people here and joining in and making all of our processes and work items more transparent 15:55:41 <dustymabe> It feels like a nice thriving community 15:55:46 <dustymabe> kudos to everyone involved!@ 15:56:03 * dustymabe will do 2nd item in 20 seconds 15:56:10 <mkonecny> \me dancing on the Open floor 15:56:15 <pingou> thanks for the kind words :) 15:56:26 * cverna joins the dance :-) 15:56:38 <nirik> awwww...thanks. 15:56:57 <dustymabe> Second: I have a ticket: [method for copying in ostree content to our ostree repos](https://pagure.io/releng/issue/8811) 15:57:04 <dustymabe> I have a request for security audit in that ticket. nirik and mboddu and I did half of the work for this a few fridays ago and I asked if I should re-use the ticket or open a new one for a security audit request. Was told to use the same ticket. Can we get a security audit scheduled? 15:57:31 <pingou> we'll need to ping puiterwijk on this 15:57:34 <dustymabe> alternatively, do I need to open a new ticket? 15:57:56 <nirik> I am not sure if puiterwijk still wants to help with these or if we need a new process... 15:58:08 <nirik> but we should definitely find out asap. 15:58:13 <pingou> last I've heard he was still ok to be our security officer 15:58:31 <dustymabe> probably should be at least a 2 or 3 people who can perform that task 15:58:32 <pingou> this should be a good occasion to verify that he is and has the time :) 15:59:12 <dustymabe> can someone take an action item to find out what the new security audit process is ? 15:59:29 <mkonecny> pingou: Speaking of the time, time for this meeting is running out 15:59:33 <nirik> in the ideal world... 15:59:34 <pingou> yup 15:59:44 <nirik> dustymabe: I can take it. 15:59:50 <dustymabe> nirik: thanks! 15:59:57 <dustymabe> nirik: I'll update the ticket 16:00:07 <pingou> nirik: let's raise this on the infra list, we can work on this together 16:00:25 * nirik adds a task 16:00:45 <nirik> pingou: sure. I was going to ask in private first to avoid putting anyone on the spot... 16:00:53 <pingou> +1 16:01:59 <nirik> then we can go from there. will cc you 16:02:05 <dustymabe> thanks all 16:02:16 <cverna> Thanks all 16:02:19 <pingou> let's call it a meeting then :) 16:02:20 <pingou> #endmeeting