15:00:02 <pingou> #startmeeting Infrastructure (2019-10-17)
15:00:02 <zodbot> Meeting started Thu Oct 17 15:00:02 2019 UTC.
15:00:02 <zodbot> This meeting is logged and archived in a public location.
15:00:02 <zodbot> The chair is pingou. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:02 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:02 <zodbot> The meeting name has been set to 'infrastructure_(2019-10-17)'
15:00:05 <pingou> #meetingname infrastructure
15:00:05 <zodbot> The meeting name has been set to 'infrastructure'
15:00:07 <pingou> #topic aloha
15:00:09 <pingou> #chair nirik pingou relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs
15:00:09 <zodbot> Current chairs: abompard bowlofeggs cverna mizdebsk mkonecny nirik pingou relrod smooge tflink
15:00:23 <bowlofeggs> .hello2
15:00:24 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:00:33 <nirik> morning
15:00:37 <cverna> morning
15:01:05 <marc84> morning everyone
15:01:10 <relrod> morning
15:01:12 <pyratebeard> hi all
15:02:14 <dustymabe> .hello2
15:02:15 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
15:02:17 <mkonecny> .hello zlopez
15:02:18 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu>
15:02:24 <abompard> .hello2
15:02:27 <zodbot> abompard: abompard 'Aurelien Bompard' <aurelien@bompard.org>
15:02:30 <pingou> I'll wait another 30s and we'll get started
15:03:01 <pingou> #topic Next chair
15:03:12 <pingou> I've been wondering if we should tie this to the oncall
15:03:24 <pingou> and make the oncall be the chair at the end of their week
15:03:59 <pingou> would this make things simpler or would this make it so that it comes down to the same people too frequently?
15:04:05 <dustymabe> :) - insult to injury?
15:04:25 <dustymabe> how about next weeks oncall is this weeks chair ?
15:04:27 <nirik> on the one hand they would know a lot about what happened that last week
15:04:33 <nirik> on the other hand... more work for them
15:04:46 <cverna> the pool of people going oncall is quite limited, I think more people could run the meeting
15:04:53 <nirik> cverna: +1
15:04:56 <cverna> for example apprentice could run the meeting
15:05:01 <pingou> fair
15:05:06 <dustymabe> cverna++
15:05:10 <pingou> so do we have any volunteer for next week? :)
15:05:14 <dustymabe> that's actually a great idea for an apprentice
15:05:26 <cverna> yes :-)
15:05:29 <bgilbert> .hello2
15:05:30 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
15:05:36 * nirik will not be here for next weeks meeting.
15:06:18 <bowlofeggs> dustymabe should host next week
15:06:41 <pingou> and you the week after?
15:07:10 * dustymabe mostly lurks - but I can run the meeting - not sure I can sign up for action items to follow up though
15:07:11 * pingou happy to have two weeks planned
15:07:12 * cverna can take next week
15:07:16 <dustymabe> but I'm good at assigning action items :)
15:07:38 <pingou> so cverna or dustymabe ? :)
15:07:58 <dustymabe> how about cverna and then dustymabe
15:08:05 <pingou> deal!
15:08:05 <cverna> me me me :)
15:08:20 * dustymabe always refers to himself with both first and last name
15:08:28 <pingou> #info magic eight ball says: cverna on October 24th then dustymabe on October 31st
15:08:35 <pingou> #topic New folks introductions
15:08:38 <pingou> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:08:40 <pingou> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:08:44 <pingou> do we have any new folks joining us today?
15:09:17 <austinpowered> I guess I would still be considered new
15:09:56 <dustymabe> austinpowered++
15:09:56 <zodbot> dustymabe: Karma for austinpowered changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:10:14 <pingou> did you introduce yourself to a meeting already? (sorry if you did and I don't remember :( )
15:10:52 <austinpowered> I did not. I stepped afk for a moment at the start. Back now...;
15:11:19 <nirik> welcome
15:11:23 <pingou> welcome then :)
15:11:27 <joalon> Welcome!
15:11:39 <cverna> welcome back :-P
15:11:46 <pingou> do you want to introduce yourself quickly here or on the list? Are you more interested in sys-admin or dev?
15:12:50 <austinpowered> sys-admin - I've been trying to get my humble home lab to be infrastructure as code with ansible
15:13:23 <pingou> sounds nice, are you already in the apprentice group?
15:14:12 <austinpowered> been pouring through the fedora-admin/ansible files
15:14:21 <austinpowered> yes ot apprentive group
15:14:26 <pingou> cool :)
15:14:34 <austinpowered> yes to apprentice group
15:14:37 <pingou> let us know if we can help you get started on anything
15:14:47 <pingou> or if you're blocked
15:16:02 <austinpowered> just drinking from the fire hose - could probably use some guidance - I'll ask in fedoara-admin after this meeting
15:16:09 <pingou> sounds good :)
15:16:19 <pingou> so let's move to announcements
15:16:20 <pingou> #topic announcements and information
15:16:26 <pingou> #info We are looking for people to maintain Fedocal - mkonecny
15:16:29 <pingou> #info CPE Team - Post Flock report was published (https://communityblog.fedoraproject.org/cpe-team-at-flock-post-flock/) - mkonecny
15:16:31 <pingou> #info f31 final freeze now in effect
15:16:33 <pingou> #info kevin/nirik on PTO from 2019-10-23 to 2019-10-27
15:16:35 <pingou> #info robosignatory running with fedora-messaging deployed in stg and prod.
15:16:37 <pingou> #info koji updated for https://docs.pagure.org/koji/CVE-2019-17109/
15:16:39 <pingou> #info work ongoing to test fasClient changes to make it work on f31 and el8
15:16:41 <pingou> #info work ongoing to deprecate ModernPaste pastebin - new fpaste client ready for testing: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/GCHZLJSBWJIQERU524P67OMVWZTIOMN4/ - report issues to Ankur/FranciscoD
15:16:43 <pingou> anything else we'd like to add ?
15:16:57 <pingou> #info multi-packages gating has started in staging
15:17:19 <pingou> #info single-package gating working end to end in staging
15:17:20 <mkonecny> #info Fedocal already found someone, I just forgot to update meeting
15:17:30 <nirik> yeah, great news. ;)
15:17:50 <pingou> mkonecny: removing it from the board then :)
15:18:12 <pingou> any more new info?
15:18:15 <mkonecny> I will update it
15:18:21 <pingou> mkonecny: just did ;-0)
15:18:23 <relrod> #info fasClient for f31 and el8 are actually in the infra repo now, but relrod forgot to update the meeting document to state that above :)
15:18:37 <nirik> awesome. :) Thanks for working on that...
15:18:48 <cverna> relrod++
15:18:48 <nirik> relrod: I guess go ahead and close that ticket?
15:18:48 <pingou> removed
15:18:50 <pingou> relrod++
15:18:54 <relrod> nirik: ah yeah
15:19:23 <nirik> also f32 I think exists... we have one rawhide machine. ;)
15:19:45 <relrod> yeah there's an f32 build in the infra repo as well
15:20:21 <nirik> ah, stg... thats why I didn't see it. ;)
15:20:41 <relrod> oh I can fix that
15:21:04 <pingou> next topic?
15:21:46 <pingou> #topic Oncall
15:21:49 <pingou> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:21:51 <pingou> #info pingou is on call 2019-10-10->2019-10-17
15:21:53 <pingou> #info smooge is on call 2019-10-17->2019-10-24
15:21:55 <pingou> #info relrod is on call 2019-10-24->2019-10-31
15:22:00 <pingou> do we have anyone for the first week of November?
15:22:12 <nirik> I can do that one...
15:22:35 <nirik> unless soemone else wants to. ;)
15:22:39 <pingou> anyone?
15:23:44 <nirik> looks like me. ;)
15:23:45 <pingou> nirik: sorry looks like you're the lucky winner
15:23:48 <pingou> #info nirik is on call 2019-10-31->2019-11-07
15:23:51 <pingou> #info Summary of last week: (from pingou )
15:23:54 <pingou> #info No pings to me so quiet week here
15:24:12 <pingou> I'm not sure if everything is fine, people are using tickets for everything or by-passing the oncall mechanism
15:25:06 <pingou> ok moving on
15:25:09 <pingou> #topic Monitoring discussion
15:25:11 <pingou> #info https://nagios.fedoraproject.org/nagios
15:25:36 <nirik> lets see...
15:25:42 <nirik> most are known
15:25:50 <mkonecny> pingou: There is still freeze going on
15:26:08 <pingou> mkonecny: likely helps
15:26:08 <nirik> notifs-backend01 is getting a full disk. :( I looked and it seems like it's journal logs of all the stuff it's sending?
15:26:17 <nirik> not sure what to do about that
15:26:52 <pingou> we can likely trim them down
15:27:55 <nirik> Filesystem                        Size  Used Avail Use% Mounted on
15:27:55 <nirik> /dev/mapper/GuestVolGroup00-root   15G   14G  827M  95% /
15:28:14 <nirik> there's 863MB of /var/log/
15:28:36 <pingou> less than 1G out of 14 though
15:28:42 <pingou> but still pretty big
15:28:47 <nirik> we could just make it bigger, but I'd hate to reinstall it...
15:29:00 <nirik> I guess we could just resize it up...
15:29:13 <marc84> nirik +1
15:29:44 <nirik> I mean I'd love to actually just replace it with a new implementation, but thats going to take a while
15:30:02 <pingou> the disk may be full before that
15:30:03 <nirik> anyhow, thats the only non known one I see... so we can move on.
15:30:11 <nirik> likely. ;)
15:30:19 <nirik> I can send in a freeze break to resize it...
15:30:33 <pingou> let's move to tickets
15:30:36 <pingou> #topic Tickets discussion
15:30:38 <pingou> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:30:40 <pingou> #info no tickets marked for the meeting
15:30:42 <pingou> #topic Tickets discussion
15:30:44 <pingou> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:30:46 <pingou> #info no tickets marked for the meeting
15:30:48 <pingou> oups
15:30:50 <pingou> #topic Tickets discussion
15:30:52 <pingou> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:30:54 <pingou> #info no tickets marked for the meeting
15:30:56 <pingou> raah
15:30:58 <pingou> #info Fedora CoreOS Team requests
15:30:59 <pingou> #link https://hackmd.io/5uB7hOJKSjGUt65iLgPnbA#Existing-requests-for-Fedora-Infra
15:31:02 <pingou> there!
15:31:04 * dustymabe waves
15:31:08 * bgilbert waves
15:31:15 <dustymabe> this is a running list I'm keeping
15:31:21 <dustymabe> we have a few new tickets since last time
15:31:27 <pingou> AWS IAM permissions: https://pagure.io/fedora-infrastructure/issue/8142
15:31:36 <pingou> coreos apps in communishift: https://pagure.io/fedora-infrastructure/issue/8218
15:31:49 <pingou> communishift /dev/kvm: https://pagure.io/fedora-infrastructure/issue/8208
15:32:03 <pingou> GCP access: https://pagure.io/fedora-infrastructure/issue/8281
15:32:10 <pingou> new AWS region: https://pagure.io/fedora-infrastructure/issue/8299
15:32:14 <dustymabe> thanks pingou
15:32:29 <dustymabe> i don't know where to begin :)
15:32:37 <dustymabe> one at a time I guess
15:32:48 * nirik knows of most of these... but I have had higher prio stuff.
15:32:53 <nirik> so, not sure what to say either.
15:33:02 <dustymabe> [AWS IAM permissions] - this is something kevin has been working on, but is not high priority right now
15:33:05 <dustymabe> we're not blocked there
15:33:10 <pingou> dustymabe: could you order the hackmd by priority?
15:33:17 <nirik> perhaps... yeah, that ^
15:33:35 <dustymabe> [coreos apps in communishift] we mostly just need some docs on how to get users added to a project
15:33:42 <dustymabe> pingou: yes I can for next time
15:33:47 <pingou> thanks
15:34:14 <dustymabe> [communishift /dev/kvm] this would be really nice to have so we can start using communishift for some more advanced testing/development projects
15:34:37 <dustymabe> [GCP access] this one is new - can we get some eyes on it?
15:35:00 <dustymabe> [new AWS region] this one is new too - who do we need to talk to to get other regions enabled ?
15:35:21 <bgilbert> * assuming Infra is comfortable doing so
15:35:42 * nirik hasn't seen that one, filed last night after I went to sleep I guess?
15:36:09 <bgilbert> the last one?  yeah
15:36:24 <dustymabe> nirik: not expecting any answers today - just want to raise them as something we can look at sooner than later
15:36:39 <nirik> sure, happy to.
15:36:39 <bgilbert> +1
15:37:02 <nirik> off hand, I don't see any reason not to enable that opt in... seems fine to me. I was unaware it existed. ;)
15:38:09 <pingou> should we move to the backlog?
15:38:28 <dustymabe> nirik: regarding GCP - is there one person in particular we can work with?
15:39:01 <nirik> I thought we were waiting on something there, let me look at the ticket
15:39:19 <nirik> oh, question on credits...
15:39:43 <nirik> but I guess that doesn't block the account setup...
15:39:54 <bgilbert> right.  we're still waiting for them to get back to us
15:40:00 <dustymabe> yes so it looks like either credits or a credit card ?
15:40:12 <bgilbert> but the amount of money will be small at first, in the worst case
15:40:35 <dustymabe> bgilbert: do you know if they require a card even if we have credits ?
15:40:45 <bgilbert> I don't know
15:40:49 <nirik> I'll talk to the holder of the purse(s) and see how we want to set things up.
15:40:57 <bgilbert> +1
15:40:58 <smooge> my understanding is that they will require a card or a PO
15:41:10 <dustymabe> nirik: +1 - loop me in on that if you need any help
15:41:49 <smooge> this is to make sure they are covered in case you put something which burns through the credits etc
15:41:58 <dustymabe> smooge: yeah, makes sense
15:42:16 <nirik> sure, can do
15:43:26 <pingou> next topic?
15:43:39 <nirik> topic ++
15:43:41 <cverna> +1
15:43:46 <pingou> #topic backlog discussion
15:43:49 <pingou> #info go over our backlog and discuss and determine priority
15:43:51 <pingou> #link https://pagure.io/fedora-infrastructure/issue/8178
15:43:53 <pingou> #info topic: provision new aarch64 builders
15:43:55 <pingou> #link https://pagure.io/fedora-infrastructure/issue/8157
15:43:57 <pingou> #info topic: ansible: enable ansible-report as a hook
15:43:59 <pingou> so these two have been there for a few weeks now
15:44:09 <pingou> are there any we want to add/remove?
15:44:21 <cverna> I have not add a chance to work on the hook :-(
15:44:44 <cverna> s/add/had
15:45:24 <nirik> on the aarch64 stuff smooge has been poking at it.
15:45:38 <nirik> there's some progress, but we were also stalled by a dnf bug.
15:46:00 <nirik> (armv7 guests wouldn't work / couldn't install or upgrade anything)
15:46:31 <cverna> oh that's a dnf issue ? I guess that the cause for the container to fail building on armv7
15:46:36 <nirik> but yeah, we are doing poorly at finishing backlogs
15:47:05 <nirik> cverna: I don't think it's the same issue... this only affects newer hardware. The older should keep working I think...
15:47:16 <cverna> ah ok :-)
15:48:04 <cverna> yes not sure how to make the backlog item more visible, so we can focus more on them
15:48:29 <nirik> I think more scheduling might help
15:48:32 <pingou> and it's only a part of it, little projects such as gating rawhide are taking a bit of our time :)
15:48:45 <nirik> "I am going to work on X at this time on this channel, join if you want to follow/help"
15:49:07 <nirik> not to mention unplanned work like this koji db stuff. ;(
15:49:17 <pingou> yup
15:49:34 <pingou> I don't think there is anything else to do there
15:49:38 <pingou> open-floor?
15:49:46 <cverna> happy to try what nirik said "I am going to work ....", do we use the infra list to announce that
15:49:55 <pingou> infra list +1
15:50:02 <dustymabe> +1
15:50:10 <nirik> cverna: +1... I was trying to use ticket, but they might not be as visible.
15:50:15 <nirik> I suppose both would be best...
15:50:16 <dustymabe> even if you get nothing back I can promise it's useful for transparency
15:50:39 <cverna> ok so let's try that :-)
15:51:02 <cverna> and we can do both list and tickets
15:51:52 <nirik> sounds good
15:52:06 <austinpowered> the ticket is where progress is documented and issues discussed. The list should be used to draw attention to the ticket.
15:52:25 <austinpowered> But I'm the new guy....
15:53:09 <austinpowered> I would expect to find everything about the issue in the ticket
15:53:17 <cverna> yes +1
15:53:23 <nirik> yep.
15:53:29 <dustymabe> For open floor I have two items - let me know when I should go
15:54:28 <pingou> let's move to open-floor
15:54:31 <pingou> #topic Open Floor
15:54:32 <pingou> dustymabe: go for it!
15:55:11 <dustymabe> First off:
15:55:31 <dustymabe> This meeting has been amazing. I'm so happy to see so many people here and joining in and making all of our processes and work items more transparent
15:55:41 <dustymabe> It feels like a nice thriving community
15:55:46 <dustymabe> kudos to everyone involved!@
15:56:03 * dustymabe will do 2nd item in 20 seconds
15:56:10 <mkonecny> \me dancing on the Open floor
15:56:15 <pingou> thanks for the kind words :)
15:56:26 * cverna joins the dance :-)
15:56:38 <nirik> awwww...thanks.
15:56:57 <dustymabe> Second: I have a ticket: [method for copying in ostree content to our ostree repos](https://pagure.io/releng/issue/8811)
15:57:04 <dustymabe> I have a request for security audit in that ticket. nirik and mboddu and I did half of the work for this a few fridays ago and I asked if I should re-use the ticket or open a new one for a security audit request. Was told to use the same ticket. Can we get a security audit scheduled?
15:57:31 <pingou> we'll need to ping puiterwijk on this
15:57:34 <dustymabe> alternatively, do I need to open a new ticket?
15:57:56 <nirik> I am not sure if puiterwijk still wants to help with these or if we need a new process...
15:58:08 <nirik> but we should definitely find out asap.
15:58:13 <pingou> last I've heard he was still ok to be our security officer
15:58:31 <dustymabe> probably should be at least a 2 or 3 people who can perform that task
15:58:32 <pingou> this should be a good occasion to verify that he is and has the time :)
15:59:12 <dustymabe> can someone take an action item to find out what the new security audit process is ?
15:59:29 <mkonecny> pingou: Speaking of the time, time for this meeting is running out
15:59:33 <nirik> in the ideal world...
15:59:34 <pingou> yup
15:59:44 <nirik> dustymabe: I can take it.
15:59:50 <dustymabe> nirik: thanks!
15:59:57 <dustymabe> nirik: I'll update the ticket
16:00:07 <pingou> nirik: let's raise this on the infra list, we can work on this together
16:00:25 * nirik adds a task
16:00:45 <nirik> pingou: sure. I was going to ask in private first to avoid putting anyone on the spot...
16:00:53 <pingou> +1
16:01:59 <nirik> then we can go from there. will cc you
16:02:05 <dustymabe> thanks all
16:02:16 <cverna> Thanks all
16:02:19 <pingou> let's call it a meeting then :)
16:02:20 <pingou> #endmeeting