18:00:31 <nirik> #startmeeting Infrastructure (2016-06-16)
18:00:31 <zodbot> Meeting started Thu Jun 16 18:00:31 2016 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:31 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:31 <zodbot> The meeting name has been set to 'infrastructure_(2016-06-16)'
18:00:31 <nirik> #meetingname infrastructure
18:00:31 <nirik> #topic aloha
18:00:31 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson
18:00:31 <zodbot> The meeting name has been set to 'infrastructure'
18:00:31 <zodbot> Current chairs: abadger1999 dgilmore lmacken nirik pbrobinson pingou puiterwijk relrod smooge threebean
18:00:31 <nirik> #topic New folks introductions / Apprentice feedback
18:01:00 * threebean waves
18:01:05 <puiterwijk> Hello
18:01:18 * aikidouke here
18:01:24 * skrzepto is here
18:01:29 * pcreech here
18:01:33 * subho here
18:01:39 * lousab is here
18:01:43 <lousab> hello
18:02:10 <smooge> hello
18:02:11 <athos> hello :)
18:02:31 <nirik> hello all.
18:02:41 <nirik> any new folks like to give a short one line introduction?
18:03:47 <nirik> ok, if no, will go on to status/info
18:03:57 <nirik> #topic announcements and information
18:03:57 <nirik> #info Still in F24 final freeze - everyone
18:03:58 <nirik> #info spam attacks continue - patrick/smooge/kevin
18:03:58 <nirik> #info (re)setup proxy07 on a cloud node at bodhost, seems working so far - kevin
18:03:58 <nirik> #info New 10G switches and new bladecenters arriving in phx2 to be setup in coming weeks - kevin/smooge/patrick
18:04:09 <nirik> any other status/info folks would like to note? or discuss from above?
18:05:06 <smooge> spam
18:05:33 <nirik> yeah, the spam... continues. ;)
18:05:40 * pingou here (late)
18:07:33 * nirik is a bit distracted, still in the go/no-go meeting.
18:07:45 <smooge> alright will run
18:07:53 <smooge> ok we are in final freeze
18:07:54 <nirik> I had one discussion item
18:07:57 <smooge> which may or may not occur
18:08:18 <nirik> #topic Some upcoming releng changes that may affect infra:
18:08:18 <nirik> Review F25 change that may impact Infra: Automated Docker Releases
18:08:18 <nirik> Review F25 change that may impact Infra: Fedora Docker Registry
18:08:42 <nirik> .ticket 5356
18:08:43 <zodbot> nirik: #5356 (Review F25 change that may impact Infra: Automated Docker Releases) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/5356
18:08:44 <nirik> .ticket 5357
18:08:46 <zodbot> nirik: #5357 (Review F25 change that may impact Infra: Fedora Docker Registry) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/5357
18:09:08 <nirik> I looked at these and added feedback, but if everyone else could take a look and do so too that would be great.
18:09:47 <smooge> ok those will require new services and also impcact deliverables so pleas edo
18:10:01 <smooge> nad my fingers aren't typing well in colocation :)
18:10:31 <smooge> does anyone have anything to say right now on those two tickets?
18:10:46 <smooge> I expect we will talk more about them in the upcoming months
18:10:50 <nirik> yeah.
18:11:22 <smooge> ok will move onto next topic.
18:11:25 <smooge> #topic Apprentice office hours
18:11:49 * bwood09 back from hiatus and ready to start working
18:11:51 <smooge> I first want to say ++ to linuxmodder for helping various apprentices in #fedora-admin
18:11:58 <pingou> linuxmodder++
18:11:58 <zodbot> pingou: Karma for linuxmodder changed to 18 (for the f23 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:12:05 <smooge> linuxmodder++
18:12:11 <vivek_> linuxmodder++
18:12:11 <zodbot> vivek_: Karma for linuxmodder changed to 19 (for the f23 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:12:14 <smooge> I think I gave my cookies alreayd
18:12:32 <athos> linuxmodder++
18:12:32 <zodbot> athos: Karma for linuxmodder changed to 20 (for the f23 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
18:12:41 <smooge> I have seen him helping a lot of people while I have been stuck dealing with SPAM issues.
18:12:50 <smooge> Hi bwood09
18:12:54 <smooge> welcome back
18:12:57 <bwood09> Thanks ^_^
18:13:11 <smooge> are there any questions at the moment?
18:13:34 <athos> yes, I didn't even know he was on fi-apprentice since he's been that active in the channel helping ppl out
18:13:46 <lousab> smooge i have a question on nagios ansible
18:15:05 <smooge> hi lousab
18:15:08 <lousab> i've seen on batcave under ansible/roles i think and i didn't understood if there's the yalm file or not or if i have to build for #5337
18:15:10 <smooge> what is the question
18:15:10 <aikidouke> and i have something unrelated to lousab
18:15:29 <smooge> .ticket 5337
18:15:30 <zodbot> smooge: #5337 (add monitoring for fedoramagazine) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/5337
18:15:56 <bwood09> and I was thinking about picking up ticket 4973 if nobody's done anything with it
18:16:00 <bwood09> .ticket 4973
18:16:01 <zodbot> bwood09: #4973 (add nagios check for mailman01's REST interface.) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/4973
18:16:12 <smooge> ha everyone wants to do nagios :).
18:16:20 <lousab> :)
18:16:24 <smooge> and I am about to change out how nagios is done
18:16:26 <nirik> nagios doesn't use anything close to as sane as a yaml file. ;)
18:17:07 <smooge> so nagios is mostly done through a file format that is one of the many meh formats. Why is it like that? meh
18:17:42 <lousab> nirik: i normally configure nagios with kickstart if i have to install it and thruk and monarch plugins....i never used ansible..i'm learning it :(
18:17:48 * jflory7 is here
18:17:52 <smooge> We haven't been consistent on how we implement them across the system so they end up being very fragiole
18:19:28 <nirik> yeah, the best way I think to see is pick a similar host to the one you are adding and 'git grep' it and look at where it's defined in nagios
18:19:34 <bwood09> Is there anything we can do to streamline that process? What changes are we looking at for nagios?
18:19:40 <nirik> There's also some old commits likely where machines were added... look in git log
18:20:16 <lousab> nirik: ok thanks
18:20:33 <nirik> the plan is for smooge to reimplement it so ansible just adds the needed stuff when we add a host.
18:20:35 <lousab> aikidouke: i'm new here :) i'm an apprentice
18:20:37 <nirik> but thats not done yet. ;)
18:20:44 <bwood09> hm, that sounds like a cool idea
18:20:49 <aikidouke> maybe this needs to start in releng or something, but how long do we think it might be before we need an app/tracker for snap packages or flatpacks
18:21:04 <bwood09> I need to get my Ansible infra rebuilt and refresh my memory on how to use it
18:21:14 <bwood09> I think the last time I was here we might have been migrating over to it
18:21:31 <nirik> aikidouke: I think thats a ways out. :)
18:21:47 <smooge> lousab, so what I think we are looking for right now would be a nagios file just like you would use on your other places htat would monitor a website
18:22:22 <aikidouke> :) ack - was typing that out waiting to hit enter, guess im not good at waiting
18:22:39 <aikidouke> and sorry lousab: did not mean to step on your toes
18:22:40 <smooge> aikidouke, releng would be better
18:22:47 <aikidouke> thanks
18:23:08 <nirik> aikidouke: neither snappy nor flatpack is what I would consider ready for widespread use, despite PR people saying so. ;)
18:23:27 <lousab> aikidouke: no problems :)
18:23:37 <smooge> bwood09, my plan is to make a set of templates for hosts like we have in the ./roles/nagios/client/templates/check_fedmsg_gateway_proc.cfg.j2 but for services and hosts
18:23:41 <aikidouke> :)
18:24:04 <lousab> smooge: ok right
18:24:12 <smooge> so lousab and bwood09 what I would like is a file that meets that
18:24:15 <bwood09> smooge, that makes sense
18:24:50 <smooge> so when looking at say fedoramagazine we want to be able to change that out to be fedoraproject or ask.fedoraproject or ...
18:25:05 <bwood09> I'll need to get back in and look at that file
18:25:19 <smooge> those files use a {{{ }}} variable substitution
18:25:59 <smooge> so write the configs with sya fedoramagazine in mind but then look at 'this could be a var' in comments
18:26:37 <smooge> that way the work is being done once and can be 'redone' when time is ready
18:27:02 <smooge> for general help on what oges in the files.. I have to work on that this weekend
18:27:11 <smooge> does that help?
18:27:23 <bwood09> makes sense to me
18:27:56 <lousab> smooge: ok i'll find you because i'm starting to use ansible now so solving this ticket means undestarnd for me also how ansible works
18:28:04 <smooge> cool
18:28:17 <bwood09> same here
18:28:25 <smooge> any other questions?
18:28:46 <bwood09> How much incident management stuff do we do? A lot of my recent professional experience is in that area
18:29:27 <smooge> we don't have incidents. we have unicorns and butterflies
18:29:44 <bwood09> lol I can see that ;)
18:30:07 <athos> lol
18:30:36 <pingou> and http://media-cache-ak0.pinimg.com/736x/bf/3f/4c/bf3f4c4e4cbc909f957f939bb6bc7cc6.jpg
18:30:41 <aikidouke> if a butterfly flaps its wings in the phx2 data center....
18:30:51 <smooge> puiterwijk, is our security lead.
18:30:56 <pingou> aikidouke: we're fine, it's the other end of the world that's not :)
18:31:07 <aikidouke> lol
18:31:29 <smooge> puiterwijk is in another meeting I think so I would aim a question towards his email to find out
18:31:53 <pingou> truth is we have some but not that many
18:32:01 <pingou> (luckily)
18:32:26 <bwood09> I work at Navy Federal Credit Union now and omg so many actual incidents it's crazy
18:32:54 <pingou> somehow, I can imagine that :)
18:32:57 <aikidouke> would it help to review our, what are they called, security guidelines? who to report X to, etc
18:33:03 <puiterwijk> We have some documents of how to manage incidents that I'm updating and writing some more. But I try to make sure we have as few incidents as possible. If you have any specific questions, please let me know
18:33:18 <nirik> we have CSI for that process currently...
18:33:25 <bwood09> puiterwijk, if there's anything I can do to help out, I'd be happy to
18:33:25 <aikidouke> thanks nirik
18:33:35 <puiterwijk> nirik: right. That CSI is what I'm reading through and updating
18:33:36 <nirik> https://docs.fedoraproject.org/en-US/Community_Services_Infrastructure/1/html/Security_Policy/index.html
18:33:42 <nirik> puiterwijk: excellent.
18:33:59 <puiterwijk> I'm also writing a bunch of documents for myself and others of what to do for specific roles/services.
18:34:11 <aikidouke> there are probably a few hosts that need CSI variables updated/added still
18:34:19 <aikidouke> that is a good place to start
18:34:20 <nirik> yes, there's a number of them. ;)
18:34:24 <nirik> patches welcome.
18:34:35 <puiterwijk> Yep, absolutely. If anyone sees any open or outdated CSI variables, fixes are welcome
18:34:58 <athos> 145 groups w/o csi variables
18:35:10 <lousab> aikidouke: yes but i didn't really understod how to..i mean i've undestood iptables roles but how can we start?
18:35:18 <athos> I am trying to understand how to codument those and will probably start shipping patches this weekend
18:35:37 <aikidouke> good q lousab
18:35:53 <nirik> theres a section on them on the apprentice page... and also some previous patches on the list right now.
18:35:59 <athos> though I will need to get the feeling on how to set security categories
18:36:04 <pingou> athos: cool, thanks
18:36:05 <aikidouke> so - updating CSI vars takes a bit of research
18:36:21 <aikidouke> reading through the security doc nirik linked will help
18:36:29 <athos> oh, I did
18:36:39 <aikidouke> if you are unsure, make an educated guess, then ask
18:36:48 <athos> ack
18:37:26 <aikidouke> you can get a feel for the categories, by looking at existing configurations
18:37:39 <athos> I am trying to figure out if I can write some scripts to set the relationships
18:37:48 <aikidouke> for example - if mirrormanager went down - that would be bad....
18:38:19 <pingou> for everyone :)
18:38:39 <aikidouke> walk through each of the tasks for one particular service - see what it depends on
18:38:45 <pingou> it's one of these apps where nagios is slower than our users :)
18:39:06 <aikidouke> grep through everything else to see what depends on the service/host you are looking at
18:39:14 <aikidouke> :)
18:39:20 <athos> thx :)
18:40:00 <aikidouke> dont be afraid to make a mistake - I've made them and noone shows up at your house to take away your birthday
18:40:24 <bwood09> I would hope the birthday police don't show up
18:40:24 <lousab> nirik: could you please tell me exactly what section? of course when you have time...:)
18:40:33 <bwood09> although not aging would be cool
18:40:34 <smooge> For the most part, the worst you are going to get is a hohoho email from me
18:40:44 <aikidouke> :)
18:40:54 <nirik> https://fedoraproject.org/wiki/Infrastructure_Apprentice#Longer_term_quests
18:40:55 <athos> please take away 8 or 9 of my bdays!
18:41:01 <pingou> bwood09: the problem is: you get older but w/o the cakes
18:41:04 <aikidouke> haha...same here
18:41:06 <bwood09> ooooooooh
18:41:11 <pingou> (even if the cake is a lie)
18:41:15 <aikidouke> lol
18:41:28 <bwood09> it wouldn't be a triumph anymore
18:41:31 <aikidouke> im getting us off track...sorry
18:41:32 <bwood09> no huge success
18:41:36 <bwood09> lol yeah me too sorry
18:43:16 <smooge> ok back on topic
18:43:20 <smooge> any other questions?
18:43:44 <smooge> I will move to open floor
18:43:48 <smooge> #topic Open Floor
18:44:15 <nirik> FYI, the go/no-go meeting just ended a few minutes ago... and we are go for f24 release next tuesday. ;)
18:44:22 <smooge> yay
18:44:23 <puiterwijk> \o/
18:44:25 <pingou> \ó/
18:44:30 <athos> awesome :)
18:44:44 <bwood09> great :D
18:44:51 <smooge> so get your f24 ++ cookies while you can
18:44:53 * pingou got to step away
18:45:03 <pingou> thanks for chairing smooge and nirik :)
18:45:15 <smooge> ok pingou night
18:45:27 <smooge> ok I think we are done for today anyway
18:45:31 <smooge> going once
18:45:33 <lousab> :)
18:45:35 <smooge> going twice
18:45:36 <puiterwijk> I would like to propose replacing the word "spam" with "fish" from now on. My blood triggers anytime I read the word "spam" now, so I want to change it slightly :)
18:45:59 <bwood09> I had a <triggering intensifies> gif somewhere... lol
18:45:59 <puiterwijk> </sorta-kidding>
18:46:00 <pingou> puiterwijk: and vice-versa?
18:46:05 <smooge> canned spiced ham?
18:46:06 <puiterwijk> pingou: sure
18:46:19 <pingou> puiterwijk: so are dolphin spam or mamals?
18:46:35 <smooge> #endmeeting