14:00:00 <relrod> #startmeeting Infrastructure (2018-05-17)
14:00:00 <zodbot> Meeting started Thu May 17 14:00:00 2018 UTC.
14:00:00 <zodbot> This meeting is logged and archived in a public location.
14:00:00 <zodbot> The chair is relrod. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:00:00 <zodbot> The meeting name has been set to 'infrastructure_(2018-05-17)'
14:00:00 <relrod> #meetingname infrastructure
14:00:00 <zodbot> The meeting name has been set to 'infrastructure'
14:00:00 <relrod> #topic aloha
14:00:00 <relrod> #chair smooge relrod nirik pingou puiterwijk tflink
14:00:00 <zodbot> Current chairs: nirik pingou puiterwijk relrod smooge tflink
14:00:10 <puiterwijk> .hello2
14:00:11 <zodbot> puiterwijk: puiterwijk 'Patrick "マルタインアンドレアス" Uiterwijk' <puiterwijk@redhat.com>
14:00:12 <tflink> .hello2
14:00:12 <nirik> good morning everyone.
14:00:14 <zodbot> tflink: tflink 'Tim Flink' <tflink@redhat.com>
14:00:34 * nirik isn't one of the cool people who can use hello2. ;)
14:00:37 <nirik> .hello kevin
14:00:38 <relrod> Morning all, rise and shine, it's meeting time ;)
14:00:38 <zodbot> nirik: kevin 'Kevin Fenzi' <kevin@scrye.com>
14:00:40 <abompard> .hello2
14:00:40 <zodbot> abompard: abompard 'Aurelien Bompard' <aurelien@bompard.org>
14:01:05 <pingou> .hell pingou
14:01:07 <ksinny> .hello sinnykumari
14:01:08 <pingou> .hello pingou
14:01:08 <zodbot> ksinny: sinnykumari 'Sinny Kumari' <ksinny@gmail.com>
14:01:11 <zodbot> pingou: pingou 'Pierre-YvesChibon' <pingou@pingoured.fr>
14:01:14 <pingou> I guess that typo says it all
14:01:20 <relrod> pingou: ;)
14:01:24 <bowlofeggs> .hello2
14:01:25 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
14:01:37 <smooge> hello
14:01:47 <bowlofeggs> hello my infrastructure accomplices
14:02:27 <relrod> #topic New folks introductions
14:02:28 <relrod> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
14:02:48 <suanand> hi
14:02:49 <relrod> Any newcomers who would like to introduce themselves and give a quick introduction, as people filter in?
14:02:52 <nirik> whats the collective noun for an infrastructure? a datacenter of...
14:03:05 <pingou> computers?
14:03:15 <abompard> infrastuff
14:03:23 <suanand> hi, i am sundeep from i18n-devel team
14:03:26 <nirik> network? meh... sorry, derailing, more coffee. ;)
14:03:34 <nirik> welcome suanand
14:03:41 <suanand> thanks nirik
14:03:47 <bowlofeggs> greeting suanand :)
14:03:51 <bowlofeggs> (s)
14:04:04 <relrod> welcome suanand
14:04:14 <pingou> o/ suanand
14:04:14 <suanand> thanks bowlofeggs, relrod
14:04:23 <suanand> hi pingou
14:05:40 <relrod> suanand: do hang out in #fedora-admin and ask questions :)
14:06:01 <suanand> sure will do, thanks relrod :)
14:06:52 <relrod> Okay moving on to announcements
14:07:03 <relrod> #topic announcements and information
14:07:03 <relrod> #info relrod moving vh01 and vh03 hosts to new boxes
14:07:03 <relrod> #info relrod PTO 9 Jun - 19 Jun
14:07:03 <relrod> #info Pagure updated
14:07:04 <relrod> #info armv7 Builders downgraded to fedora 27 - kevin
14:07:20 <bowlofeggs> #info bowlofeggs is working on bodhi-3.8.0 beta, but isn't quite deployed yet
14:07:31 <puiterwijk> #info hopefully metalink is now more resilient to bad pkls by just not serving them to users
14:07:49 <bowlofeggs> https://github.com/fedora-infra/bodhi/blob/develop/docs/user/release_notes.rst
14:08:30 <relrod> Anyone have questions/comments, or other announcements to add?
14:09:24 <relrod> okay
14:09:27 <relrod> #topic Oncall reminder and discussion - kevin
14:09:35 * relrod passes nirik the talking stick
14:10:09 <nirik> I just wanted to note the post to the infra list... and discussion after...
14:10:22 <nirik> if anyone has questions or comments.
14:10:40 <nirik> Hopefully it will help us out
14:11:21 <nirik> There was talk about having people shadow the oncall or be mentored by them...
14:11:38 <bowlofeggs> oncall++
14:11:49 <ksinny> +1
14:11:52 <nirik> IMHO folks are welcome to ask questions or follow anything we are doing in public channels. ;) If people are busy they might wait to respond.
14:12:37 <nirik> Thats all unless others have more to discuss.
14:12:40 <relrod> nirik: yeah. I guess one question I had is who all should be in the oncall rotation? Is it just -main? Or are others supposed to be able to do it as well in the future, since a lot of it is triaging?
14:13:09 <bowlofeggs> i would think they don't need to be in main, as a non main member
14:13:23 <nirik> yeah, good question... I'm not sure. I think perhaps we could have others do it... if they have been around for a while...
14:13:25 <bowlofeggs> as long as they have enough familiarity to know whether an issue is worth interrupting someone over
14:13:32 <nirik> yeah.
14:14:00 <nirik> perhaps be at least in sysadmin-noc or sysadmin-vetren?
14:14:21 <sapo> I was the one who came up with the idea.
14:14:52 <pingou> nirik: +1 I think they would need some kind of access to be able to quickly take a look if they so desire
14:14:53 <sapo> I understand that mentoring is not comfortable during outages.
14:15:34 <sapo> What about some kind of classroom like mentoring during silent periods?
14:15:58 <relrod> sapo: we've done things like that before...apprentice days
14:16:00 <sapo> I mean describing recurring activities...
14:16:07 <relrod> they usually have less-than-great turnout
14:16:13 <nirik> To me mentoring means a longer term one on one teaching relationship... I think we susbscribe to a more group model where we want everyone to help people, but they need to ask questions and get involved...
14:16:21 <sapo> OK. I didn't know that.
14:16:33 <nirik> yeah, we could do another one of those.
14:16:50 <nirik> and/or also we have a 'learn about' session in the end of the meeting...
14:17:09 <nirik> thats a good way to learn about stuff, but often we don't have anyone lined up.
14:17:35 * relrod nods
14:17:52 <pingou> should we return the question?
14:18:04 <pingou> instead of a learn about, do a "ask me anything" ?
14:18:41 <nirik> well, that gets back to forcing people to ask when they don't know what to ask. ;)
14:18:50 <relrod> pingou: well we do have apprentice office hours at the end too...
14:19:12 <pingou> nirik: fair
14:19:24 <pingou> relrod: yeah I was also wondering if it doesn't overlap is the open-floor
14:21:00 <nirik> There's lots of things we could do... it's hard to know which ones will work best
14:21:08 * relrod nods
14:21:16 <relrod> okay any futher questions/discussion on this?
14:21:19 <nirik> probibly depends on the folks who are apprentices at the time too...
14:21:50 <relrod> If people come up with ideas later, please add them to the mailing list thread
14:22:02 <puiterwijk> h
14:22:22 <puiterwijk> (ignore that)
14:22:24 <relrod> puiterwijk: oh really now :P
14:22:40 <relrod> okay
14:22:41 <relrod> #topic Tickets discussion
14:22:42 <relrod> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=2
14:22:51 <relrod> .ticket 3983
14:22:55 <zodbot> relrod: Issue #3983: fedmsg enablement - replace the updates sync on releng04 - fedora-infrastructure - Pagure - https://pagure.io/fedora-infrastructure/issue/3983
14:22:56 <nirik> So, I added our oldest ticket here. ;)
14:23:22 <puiterwijk> Hey, that script got replaced
14:23:23 <nirik> I was wondering if perhaps we could just do this in meeting or soon. ;) We just need a fedmsg-hub listener right?
14:23:27 <bowlofeggs> haha
14:23:33 <nirik> puiterwijk: but it is still not fedmsg based. :(
14:23:43 <bowlofeggs> we might need a bodhi fedmsg to be sent too, not sure
14:23:46 <puiterwijk> Ah, right
14:23:46 <nirik> and releng04 doesn't exist anymore and...
14:23:55 <puiterwijk> bowlofeggs: nah, that's there.
14:24:21 <puiterwijk> nirik: so, I'm honestly not entirely sure what the advantage of fedmsg-based would be over this, except for that we then need to implement more locking.
14:24:23 <bowlofeggs> oh yeah, sync.wait gets sent
14:24:41 <puiterwijk> Right now, it just runs every 5 minutes, and if there's nothing to do, it's like 20 IOPS and done. Less then a second
14:24:46 <nirik> puiterwijk: well, right now it sends an email every 5min when one is running
14:24:49 <bowlofeggs> it does introduce one more way the system can fail
14:24:52 <bowlofeggs> fedmsgs can get lost
14:24:58 <nirik> so I guess we could change the locking
14:24:59 <puiterwijk> nirik: huh, it shouldn't email if nothing hchanged
14:25:00 <bowlofeggs> cron polling is more reliable, but less efficient
14:25:01 <puiterwijk> That's a bug
14:25:08 <puiterwijk> Ooooh, right. The locking outputs
14:25:11 <nirik> it's the lock wrapper
14:25:16 <puiterwijk> Yeah, we can silence the lock wrapper
14:26:07 <nirik> would it be ok to just run it every minute then?
14:26:13 <nirik> if it only takes a second or two
14:26:20 <puiterwijk> Yep.
14:26:29 <puiterwijk> That was one of my goals, just never got there
14:27:15 <relrod> is doing that cron silence + change an easyfix that an apprentice can do?
14:27:29 <puiterwijk> I'd say yes. Should be pretty simple
14:27:50 <nirik> just silence the existing one? well...
14:27:59 <nirik> then we don't get errors, but I guess thats ok
14:28:23 <puiterwijk> I think that it still checks the process, so if the process that locked it died, it'll say so and still run the tool
14:28:31 <nirik> I can just do this... I want to get rid of it. ;)
14:28:37 <nirik> yeah, it does.
14:29:06 <puiterwijk> So then the warning for "lock is taken and process is running" is just for detecting if one particular run takes several hours or the like
14:29:39 <nirik> right.
14:29:52 <nirik> and if it dies with the lock, it will see the pid doesn't exist and unlock
14:30:10 <puiterwijk> Yep
14:30:36 <puiterwijk> Done
14:30:38 <nirik> anyhow, I'll just do this. :) we can move on... unless there's other tickets anyone wants to discuss
14:30:44 <nirik> or puiterwijk can. ;)
14:31:06 <relrod> .ticket 6241 is still listed as a meeting ticket, but I think that's just from a few weeks ago?
14:31:06 <zodbot> relrod: Error: "6241 is still listed as a meeting ticket, but I think that's just from a few weeks ago?" is not a valid integer.
14:31:11 <relrod> ...
14:32:07 <nirik> it takes the entire argument. ;)
14:32:13 <nirik> .ticket 6241
14:32:15 <zodbot> nirik: Issue #6241: How to orphan in EPEL only ? - fedora-infrastructure - Pagure - https://pagure.io/fedora-infrastructure/issue/6241
14:32:27 * cverna is a bit late but here :)
14:32:28 <nirik> oh yeah, I keep not doing this. :) I will change the tags
14:32:56 <relrod> ok
14:33:14 <relrod> Any other tickets for discussion?
14:33:47 <suanand> actually need some help - to speed up transtats deployment
14:33:53 <suanand> .ticket 6459
14:33:55 <zodbot> suanand: Issue #6459: [RFR] transtats - fedora-infrastructure - Pagure - https://pagure.io/fedora-infrastructure/issue/6459
14:34:05 <suanand> should we take at end of this meeting?
14:34:58 <nirik> I think now is ok... puiterwijk was last looking at this...
14:35:05 <nirik> where is the status now?
14:35:06 <suanand> puiterwijk, what should be the next step? how can I help?
14:35:10 * nirik looks at the ticket
14:35:47 <puiterwijk> Someone needs to fix up the openshift templates and work with suanand through the RFR. I have not had time to do so, and have tried various times to hand it over to someone, without luck
14:36:55 <nirik> I'm not sure I am up to speed on openshift enough... I could try, but not sure how far I would get
14:37:07 <nirik> relrod: would you be willing to give it a go? you did a few apps?
14:37:09 <puiterwijk> Well, the openshift part should be pretty much done
14:37:21 <suanand> (not sure) if I re-attach objects (with Dockerfile)
14:37:34 <suanand> which is currently for s2i
14:37:40 <puiterwijk> And it's down to fixing the last environment var things, and then going through RFR (getting people on the books for maintaining it, writing SOPs, etc)
14:37:42 <nirik> oh, I didn't realize that... ok, so it's just a matter of tweaking it to get it working?
14:37:48 <puiterwijk> Yes
14:37:52 <suanand> yes
14:37:57 <puiterwijk> And then the RFR process stuffs
14:38:13 <puiterwijk> It should all be in ansible as well
14:38:33 <relrod> I can try to work with suanand on it...but I'm not 100% up to speed on openshift stuff either, so depending on what kinds of tweaks I might not be useful..
14:38:39 <suanand> maybe puiterwijk, you get some time (not very urgent tough)
14:38:54 <suanand> (though)
14:39:00 <puiterwijk> relrod: you can always ask me on the openshift stuff. That should be like one or two variable names.
14:39:56 <puiterwijk> Also, suanand knows how the openshift stuff works (he got the initial thing in his openshift), so you can ask him.
14:40:55 <relrod> okay
14:42:14 <relrod> alright, anything else here?
14:42:23 <suanand> i am good..
14:42:31 <relrod> #topic Apprentice Open office minutes
14:42:32 <relrod> #info A time where apprentices may ask for help or look at problems.
14:42:56 <relrod> Any apprentices have questions, want to show off what they've been doing, want to check in and say hi?
14:43:27 <nirik> I wanted to add another way apprentices could help if they are interested in packaging...
14:43:47 <nirik> .tiny https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&classification=Fedora&email1=infra-sig&emailassigned_to1=1&emailcc1=1&emailtype1=substring&list_id=8012091&order=bug_id&product=Fedora&query_based_on=&query_format=advanced
14:43:48 <zodbot> nirik: http://tinyurl.com/y8pn93p6
14:44:18 <nirik> thats a list of all the 'infra-sig' bugzilla bugs. :) Patches against those packages to fix those things on src.fedoraproject.org welcome
14:44:19 <creaked> I am interested in helping packaging
14:44:21 * puiterwijk waits for .tiny to become da.gd
14:44:34 <relrod> puiterwijk: hahah
14:44:40 <puiterwijk> :-)
14:44:44 <nirik> creaked: cool. :) Take a look at those and ask if you have questions on the fixing workflow.
14:44:46 <x3mboy> Hello!
14:44:52 <x3mboy> .hello2
14:44:53 <zodbot> x3mboy: x3mboy 'Eduard Lucena' <eduardlucena@gmail.com>
14:44:59 <bowlofeggs> and tiny should do https:// ☺
14:45:14 <relrod> puiterwijk: That was my first thought as soon as I saw tinyurl.com, so you're not wrong. :P
14:45:29 <creaked> nirik: Ok will do
14:45:36 <puiterwijk> relrod: I knew it would be your first thought, which is why I wondered how long it'd take to get that fixed :D
14:45:42 <relrod> puiterwijk: and I have good justification, da.gd is open source ;)
14:47:19 <relrod> okay
14:47:24 <puiterwijk> nirik: oh, and I think nobody tried their hand at the "prize question" yet, where the price was a cookie, right? :)
14:47:43 <relrod> #topic open floor
14:47:56 <nirik> puiterwijk: indeed. want to ask that again here?
14:48:01 <puiterwijk> Sure!
14:48:21 <puiterwijk> So, for any apprentices, for the price of a cookie: what's the difference between "service httpd restart" and "systemctl restart httpd"?
14:49:04 <relrod> apprentices: Make him give you stroopwafel instead of a cookie, it's way better ;)
14:49:28 <smooge> Well for a stroopwafel I can answer one of them
14:49:29 <bowlofeggs> haha
14:49:51 <puiterwijk> Oh yeah, these cookies are exchangable when we meet in person for a stroopwafel :)  (only apprentices eligible)
14:50:22 <puiterwijk> (also, conditions apply, like a max etc. :) )
14:52:11 <bowlofeggs> <crickets>
14:52:29 <puiterwijk> Well, if anyone wants to answer it later, just come by on #fedora-admin, and make sure to refer to this meeting. :)
14:53:00 <nirik> I wonder... could we line up a learn about for next week?
14:54:48 <relrod> I'd love to see an IPA one (sorry puiterwijk) but not sure if a meeting is enough time to really learn about it.
14:55:55 <mizdebsk> i'd like openshift one, as our way of deploying apps there is not very typical
14:56:03 <puiterwijk> relrod: that's fine, feel free to ask me if you think I can give a let's learn you're interested in. I'm not sure IPA is all that interesting as it stands now, but sure.
14:56:27 <puiterwijk> mizdebsk: right... I did an ad-hoc "learn about" at the infra hackfest, but that's a longer session then just a 10-minute thing...
14:56:32 <puiterwijk> Maybe we should schedule something else for that
14:57:21 <puiterwijk> mizdebsk: however, I think the opinionated roles should help making things significantly simpler there.
14:59:12 <relrod> okay have about a minute left before top of the hour. Anything else to discuss quickly?
14:59:23 <bowlofeggs> the openshift thing could be a good flock talk :)
14:59:31 <puiterwijk> bowlofeggs++
14:59:32 <bowlofeggs> and those get recorded, i think?
14:59:36 <puiterwijk> Yeah, they do
14:59:37 <relrod> usually yeah
14:59:41 <puiterwijk> I can totally submit one there
14:59:45 <relrod> \o/
14:59:57 <relrod> okay, thanks all for coming!
15:00:00 <relrod> #endmeeting