18:00:00 <nirik> #startmeeting Infrastructure (2016-06-23)
18:00:00 <zodbot> Meeting started Thu Jun 23 18:00:00 2016 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:00 <zodbot> The meeting name has been set to 'infrastructure_(2016-06-23)'
18:00:01 <nirik> #meetingname infrastructure
18:00:01 <zodbot> The meeting name has been set to 'infrastructure'
18:00:01 <nirik> #topic aloha
18:00:01 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson
18:00:01 <zodbot> Current chairs: abadger1999 dgilmore lmacken nirik pbrobinson pingou puiterwijk relrod smooge threebean
18:00:01 <nirik> #topic New folks introductions / Apprentice feedback
18:00:04 * pingou here
18:00:15 <smooge> here
18:00:16 <jflory7> .hello jflory7
18:00:17 <zodbot> jflory7: jflory7 'Justin W. Flory' <me@justinwflory.com>
18:00:28 <puiterwijk> Hello
18:00:30 * lousab here hi all :)
18:00:30 <aikidouke> .hello
18:00:30 <zodbot> aikidouke: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
18:00:53 <aikidouke> ermm
18:00:58 * aikidouke here
18:01:06 <nirik> morning everyone. ;)
18:01:14 <pingou> cverna: just in time for the meeting :)
18:01:14 <clime> Here. Hello, I am a fresh infrastracture apprentice. I work in Red Hat - Czech Republic and I am part of COPR team under Miroslav Suchy.
18:01:18 <pcreech> !here
18:01:22 <pingou> hi clime, welcome
18:01:27 <nirik> do we have any new folks who would like to give a short introduction of themselves?
18:01:29 <clime> pingou: thx :)
18:01:31 * cverna is here
18:01:32 <nirik> welcome clime. :)
18:01:33 <cverna> I now :)
18:01:38 <clime> nirik: thx!
18:01:43 <jflory7> Welcome clime!
18:01:52 <clime> jflory: thx :)
18:02:08 <kzuraw> Hello, I'm first time here. I'm python developer who recently which to fedora and decided to do some more than just using the system.
18:02:12 * pbrobinson is sort of here, at least for a little while
18:02:17 <mirek-hm> I will put eye on clime and I will guide him through apprentice time to some higher level
18:02:32 <nikobasi> Hello i am a new folk i want to get involved with linux administration and programming projects
18:02:50 <devyani7> .hello devyani7
18:02:51 <zodbot> devyani7: devyani7 'Devyani Kota' <devyanikota@gmail.com>
18:02:52 <jflory7> kzuraw, nikobasi: Hi there, welcome to both of you too :)
18:02:57 <aikidouke> wow lots of new folks...welcome one and all
18:03:00 <jflory7> devyani7: o/
18:03:15 <nirik> kzuraw / clime / nikobasi: welcome all. Are you all interested more in sysadmin tasks or application development, or both?
18:04:01 <clime> nirik: as for me, both probably...whatever is more needed.
18:04:10 <nikobasi> i am interested mostly in sysadmin tasks but i want to particapate in programming also
18:04:16 <kzuraw> I will start from more app development stuff
18:04:31 <nirik> both are always needed. ;) application devel is probibly easier to get started as there are lots more easyfix tasks...
18:04:43 <clime> oki, I'll start there then.
18:05:19 <nirik> great. If anyone would like to join our apprentice program ( https://fedoraproject.org/wiki/Infrastructure_Apprentice ) see me after the meeting in #fedora-admin and we can get you added and such.
18:06:08 <tammy_> Hello everyone
18:06:11 <clime> be there.
18:06:19 <pingou> hi tammy_
18:06:19 <nirik> morning tammy_
18:06:29 <nirik> ok, any other new folks?
18:07:17 <tammy_> I have joined the apprentice program but had the hardest time accessing things
18:07:37 <nirik> sorry about that. :(
18:07:53 <nirik> if you have some time later in the meeting in the apprentice section or after we can try and assist?
18:08:05 <tammy_> I guess I am slow :)
18:08:25 <lousab> tammy_ don't worry me too i'm slow ;)
18:08:54 <tammy_> OK
18:09:17 <nirik> no worries, it's pretty complex and sometimes people who have it setup don't explain things too well since we are too close to it.
18:09:27 <nirik> but we can get you setup, I am sure of it.
18:10:39 <nirik> ok, shall we move on to status/info?
18:10:43 <pingou> +1
18:10:49 <nirik> #topic announcements and information
18:10:49 <nirik> #info Fedora 24 is released! Hurray! - everyone
18:10:49 <nirik> #info release day went pretty smoothly - everyone
18:10:49 <nirik> #info ansible upgraded to 2.1.1 rc1 on batcave - kevin
18:10:49 <nirik> #info pkgdb notifications moved from scm-commits to new scm-notifications list - kevin
18:10:50 <nirik> #info spam incursions picked up again after 4 day lull
18:10:57 <nirik> anything else this week to note?
18:11:00 <pingou> #info FMN rewrite done, stress-testing it
18:11:11 <pingou> I'm running the 3rd of 4 tests, should be done tomorrow
18:11:15 <pingou> then write a blog post
18:11:18 <nirik> pingou: great. ;) how much faster is it?
18:11:31 <pingou> nirik: it does 15K messages in ~4h
18:11:41 <pingou> I approximate 15K is about a mass-branching
18:12:13 <nirik> ok. I have no idea how that compares to the current one. ;)
18:12:35 <pingou> puiterwijk: was saying it takes us more than a day to recover from mass-branching
18:13:05 <nirik> nice. a good bit faster then. ;)
18:13:06 <pingou> on a quad-core, with 4 workers and 1 backends, I get ~0.8msg/second
18:13:20 <pingou> so ~2sec/msg
18:13:25 <nirik> where is the bottleneck, cpu?
18:13:33 <pingou> in prod looking at the logs we're at 6 to 12sec/msg
18:13:37 <pingou> yup
18:13:44 <tammy_> Does any testing need to be done after the release
18:13:51 <pingou> a little bit of io (accessing the cache & co) but mostly CPU
18:14:05 <tammy_> Just to make sure everything is working fine
18:14:10 <pingou> tammy_: oh yes, we will need to thoroughly test in stg
18:14:20 <nirik> tammy_: well, not of the release, but for updates and next release and rawhide... thats all handled by the qa folks in #fedora-qa
18:14:28 <nirik> oh, sorry, misunderstood. ;)
18:14:58 <tammy_> OK
18:15:10 <nirik> so, yeah, we will want to test it a bunch in staging.
18:15:21 <nirik> pingou: can you mail the list when it's in stg?
18:15:32 <pingou> nirik: the backend machine will likely need to be a little more beefy
18:15:45 <tammy_> I like to see the list
18:15:46 <nirik> we can do that. I think we have cpus for it
18:16:31 <tammy_> Yeah
18:16:39 <nirik> ok, anything else on that pingou ?
18:17:08 <pingou> my test machine is a quad-core(i5 760) with 16G of ram
18:17:19 <pingou> all the procs are used (make sense) and I'm consuming 9G of ram
18:17:31 <pingou> (that's with 6 workers and 1 backend)
18:17:59 <pingou> <eom>
18:18:14 <nirik> Right now notifs-backend01 is 6gb mem and 4 cpus.
18:18:28 <nirik> we can bump that pretty easily.
18:18:31 <pingou> cool
18:19:03 <nirik> say 8 and 24gb?
18:19:10 <pingou> sounds good
18:19:20 <pingou> 16G or 20G would probably be sufficient
18:19:24 <pingou> but if we have spare :]
18:19:37 <nirik> ok, on to discussion items.
18:19:40 <nirik> #topic items to get done before f25 alpha? - kevin
18:19:58 <nirik> so, with Fedora 24 out the door now.... we have about a month and a half until f25 begins
18:20:03 <pingou> \ó/
18:20:13 <nirik> what all do we need to / want to get done? I think pingou had a list of things on the list.
18:20:24 <pingou> so we want FAS3 in stg
18:20:40 <pingou> the sql migration script from FAS2 to FAS3 has been merged
18:20:52 <pingou> rerold is working on packaging (first scratch build successfull)
18:21:06 <nirik> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/JT6T6YCY53EH3TQRZURBWQ4GTT4E2FYP/
18:21:11 <pingou> SmootherFrOgZ: needs to finish the backward compat code so that login works from FAS2 db :)
18:21:19 <tammy_> Is this the list
18:21:31 <nirik> I think it's: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/JT6T6YCY53EH3TQRZURBWQ4GTT4E2FYP/
18:21:35 <pingou> and skrzepto and I should start looking at unit-tests (but that's imho not a blocker for stg)
18:21:35 <puiterwijk> I want to get the new Ipsilon release out and at least in staging, ideally production
18:21:44 <pingou> puiterwijk +1
18:21:50 <jflory7> Does any plan for Hubs fit into this time frame?
18:21:53 <nirik> pingou: so, just stg for fas3? prod after f25 is done?
18:22:00 <puiterwijk> Also, lots of Basset improvements in the next week or two
18:22:06 <pingou> nirik: sounds good
18:22:17 <pingou> on ipsilon we need oidc
18:22:24 <pingou> but also changes wrt who is what in which group
18:22:30 <pingou> (user, sponsor, admin)
18:22:52 <nirik> jflory7: according to the list, a demo by flock (early aug)
18:22:57 <pingou> jflory7: hubs is planned as a demo for flock, nothing in stg before then
18:23:03 <pingou> jinx :)
18:23:04 * jflory7 nods
18:24:04 <nirik> I have a few things: want to get zodbot moved to new supybot.
18:24:23 <nirik> want to try and get builders all moved to f24 (but need to coordinate with releng on that)
18:25:02 <nirik> puiterwijk: where do we stand on adding ppc to cloud and upgrading cloud ?
18:25:12 <cverna> nirik: anything blocking the new supybot ?
18:25:13 <nirik> and what happened to the mediawiki update?
18:25:34 <nirik> cverna: I don't think so, we could use a bit of testing on our staging one... "ursabot"
18:26:09 <puiterwijk> nirik: I've progressed on that today (cloud) and should be able to add the first ppc tomorrow, as everything's installed on it and it should Just Work(R). Then I just need to add the images to openstack and hope it works
18:26:50 <puiterwijk> upgrade is also moving along, albeit slower than hoped for because of other projects. Should happen before F25 alpha, but isn't linked to the freezes
18:27:13 <puiterwijk> mediawiki update I'm going to plan for somewhere next week.
18:27:17 <nirik> puiterwijk: (and mirek-hm / clime ): I added a copr bug for armv7 builders as we have hardware for a long time now. But if we can upgrade the cloud and get ironic working on them that might be easier than special copr code.
18:27:35 <mirek-hm> nirik: I noticed
18:27:47 <puiterwijk> nirik: yeah, I saw it. I'm going to vote for ironic, since I had that work earlier in the week.
18:27:58 <puiterwijk> (it broke since, and I need to see why... but it worked for two deployments)
18:28:41 <nirik> well, thats something. :)
18:29:31 <nirik> oh, not sure if I will get to it before f25alpha, but I am going to start looking into migrating infrastructure trac to pagure.
18:29:43 * pingou looks at cverna
18:29:51 <pingou> "no pressure"
18:30:01 <cverna> the import script is much better now :)
18:30:04 <nirik> has there been more work on the importer? ;) It was pretty far along..
18:30:08 <nirik> excellent.
18:30:14 <pingou> cverna: did you see puiterwijk 's one?
18:30:16 <cverna> still need the support for attachment
18:30:49 <cverna> yeah I ll get some of puiterwijk work :)
18:31:08 <cverna> I will have some time this week end
18:31:16 <cverna> to make some progress
18:31:30 <nirik> ok, anyone have anything else they would like to try and get done before flock/f25 freeze?
18:31:54 * nirik notes to apprentices to look for these projects, hopefully they will have some easyfixes to help work on if you like.
18:32:13 <smooge> f25 alpha seems so fricking close
18:32:24 <pingou> kinda does yes
18:32:27 <nirik> time flies. ;)
18:32:42 <cverna> no time to get bored :)
18:32:45 <nirik> pretty much there's flock and then f25alpha... not much time after
18:33:07 <nirik> FYI for anyone wondering: https://flocktofedora.org/
18:33:36 * cverna is looking forward for flock :)
18:33:56 <nirik> #topic Apprentice Office Hours
18:33:57 <smooge> will wave to you all from the other side of the oceans
18:34:04 <tammy_> Thank you nirik
18:34:19 <nirik> smooge: sorry you aren't going to make it. ;( and adamw either. :(
18:34:27 <pingou> smooge: :(
18:34:34 <adamw> we'll have our own flock
18:34:36 <adamw> with beer and hookers
18:34:47 <pcreech> screw the hookers!
18:34:56 <pcreech> er...
18:34:56 <nirik> anyhow, any apprentices with questions or comments or want some pointers? tammy_: you want to try and sort out your access? or is another time good?
18:35:08 * adamw hopes it's obvious that was a futurama quote.
18:35:28 <tammy_> I need pointers but I am not in front of my computer
18:35:45 <pcreech> I've been getting back into apprenticing, currently looking for some app-dev to hitch my talents to
18:35:53 <tammy_> My question is how to access ssh to bastion batcave I am confused
18:36:13 <tammy_> The instructions are not clear on the wiki page
18:36:35 <tammy_> Or if I am unable to access that what else can be done
18:36:39 <aikidouke> can anyone think of a use case for deploying openshift in our infra?
18:36:39 <nirik> tammy_: The ssh config in the howto sets things up to _transparently_ ssh via bastion to batcave. If you have that config, you just ssh batcave01.phx2.fedoraproject.org from your machine and ssh takes care of the tunneling via bastion
18:37:04 <tammy_> I have windows
18:37:12 <nirik> pcreech: looked at easyfixes? also there's a lot of work ongoing in hubs... #fedora-hubs
18:37:32 <tammy_> I will look into that
18:37:37 <lousab> tammy_ have you tried to setup putty? i've an how to for that if you want
18:37:48 <lousab> i mean on windows
18:37:51 <nirik> aikidouke: there's a group of folks setting up openshift on top of our cloud... misc is leading that. I don't know the status off hand.
18:37:52 <tammy_> Yes
18:37:56 <mirek-hm> tammy_: this is my part of ssh config http://paste.fedoraproject.org/383816/7044146/
18:38:10 <mirek-hm> then I just ssh to bastion and it works
18:38:11 <aikidouke> thx nirik
18:38:12 <nirik> tammy_: I have no windows, nor have ever used it, so I am not sure I can help you there, but perhaps others can
18:38:26 <tammy_> OK I will try
18:38:36 <pcreech> nirik: Cool, will take a look
18:38:47 * aikidouke thinks nirik's house must get awfully hot in the summer
18:39:02 <nirik> lousab: perhaps you could add some putty info to our ssh howto? or a pointer ?
18:39:10 <nirik> aikidouke: I have A/C. ;)
18:39:14 <tammy_> Yes
18:39:15 <aikidouke> ;)
18:39:17 <mirek-hm> tammy_: in worst case install fedora in VM and ssh from there
18:39:33 <lousab> nirik: yes i could of course and also give a hand to tammy_
18:39:42 <clime> nirik: so can I try to fix some issue from here: https://fedorahosted.org/fedora-infrastructure/report/1 ? :)
18:39:46 <tammy_> I do have vm
18:40:10 <nikobasi> use virtualbox
18:40:13 <tammy_> And cam install fedora let me try that
18:40:22 <tammy_> I have VMware
18:40:27 <nirik> clime: sure, or https://fedorahosted.org/fedora-infrastructure/report/14 (thats the ones marked easyfix)
18:40:35 <lousab> tammy_ if you have problems feel free to ask me ;)
18:40:57 <tammy_> OK
18:41:06 <lousab> nirik: just i'd like to understand wich kind of permission i have to modify ssh how to
18:41:11 <clime> nirik: oh oki, i'll look for some nice one to solve.
18:41:34 <tammy_> The same here what permissions do I have
18:42:05 <nirik> lousab: you should be able to commit changes, sure. On batcave01: 'git clone /git/infra-docs' make your changes and 'git commit' and 'git push' them.
18:42:25 <nirik> all apprentices can update infra docs
18:42:43 <aikidouke> tammy_ are you saying you have a fedora vm running and cant ssh into batcave?
18:42:51 <tammy_> OK I will try that this weekend
18:49:23 <kzuraw> @nirik, ok, to whom I need to speak to get access to apprentice group?
18:49:25 <nirik> yeah, just basically 'anything related to the group/meeting someone wanted to talk about, but wasn't able to add to the agenda'
18:49:49 <nirik> kzuraw: we can set you up in #fedora-admin after the meeting. ;)
18:50:03 <clime> nirik: i'll join too.
18:50:28 <nirik> cool.
18:50:51 <nirik> ok, lets go ahead and end the meeting here, but it will be a bit before zodbot catches up I bet. ;)
18:50:56 <nirik> thanks for coming everyone!
18:50:58 <tammy_> OK
18:51:01 <tammy_> Thanks
18:51:10 <clime> THX.
18:51:13 <nirik> do continue in #fedora-admin, #fedora-apps and #fedora-noc
18:51:16 <nirik> #endmeeting