17:03:30 <nirik> #startmeeting Infrastructure Pre FAD meetup (2014-12-01)
17:03:30 <zodbot> Meeting started Mon Dec  1 17:03:30 2014 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:03:30 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:03:30 <nirik> #meetingname infrastructure-pre-fad-meetup
17:03:30 <nirik> #topic intro
17:03:30 <zodbot> The meeting name has been set to 'infrastructure-pre-fad-meetup'
17:03:41 <nirik> pingou: want to call in any others interested? or will it just be us?
17:03:42 <pingou> who is there?
17:04:08 <pingou> nirik: maybe puiterwijk and threebean might be interested to
17:05:07 <smooge> good day
17:05:21 <nirik> hey folks.
17:05:27 <pingou> o/
17:05:34 * stickster in another meeting but lurks o/
17:06:40 <nirik> so, we just wanted to meet up and discuss where we are with mm3 and see what areas we should be ready to ask questions of mdomsch....
17:06:48 <pingou> mm2 ;-)
17:06:58 <nirik> sorry, right. need more coffee. ;)
17:07:07 <nirik> #topic Current MM2 status
17:07:20 <nirik> pingou: perhaps you could go over whats implemented so far in your mm2?
17:07:37 <pingou> so I believe most the of the UI (if not all) has been implemented: dev instance is at: http://209.132.184.188/mirrormanager/
17:07:55 <pingou> I have started to port some of the scripts used in the backend
17:07:57 <nirik> so, thats the ui for mirror admins right?
17:08:01 <pingou> yes
17:08:21 <pingou> the mirrorlist itself does not change (that's the one serving the content to yum iiuc)
17:08:49 <nirik> yeah, it's pretty simple, but not sure if we could do anything to improve it...
17:09:14 <pingou> nirik: one thing I did, it making the pages listing all the mirrors there dynamic instead of static
17:09:15 <nirik> we also need: part that detects repos and adds them to db
17:09:28 <nirik> and: crawler that checks mirrors for freshness
17:09:41 <pingou> (static == we had cron generating these pages before)
17:09:45 <nirik> and thing that generates the pkl that mirrorlists use
17:09:56 <pingou> nirik: those are all the backend script
17:10:02 <pingou> I have converted few of them
17:10:08 <pingou> the ones making the pickle I think
17:10:11 <pingou> the crawler
17:10:20 <stickster> And for some of those, probably some differences for e.g. atomic/ostree
17:10:23 <pingou> not sure about the one adding the repos
17:10:27 * nirik nods.
17:10:33 <stickster> although I think oddshocks and lmacken have been looking at those
17:10:46 <pingou> stickster: that's the part where I really don't know how we should do things
17:11:29 <lmacken> https://fedorahosted.org/fedora-infrastructure/ticket/4585#propertyform
17:11:30 <pingou> basically, I would first like to go through the process with matt, then check with him what does what (and thus if everything is converted or if we missed some things)
17:11:36 <stickster> I certainly don't :-) But David & Luke should be deeply enough into how ostree works that they can make some architectural suggestions
17:11:45 <pingou> http://ambre.pingoured.fr/public/mirrormanager.png this is a scheme of the DB
17:11:47 <nirik> #info currently UI frontend is implemented in mm2
17:11:54 <nirik> #info some backend scripts are ported.
17:12:08 <pingou> lmacken: looks simple enough :)
17:12:19 <nirik> #info gap areas: crawler, update repos, make pkl mirrorlist files.
17:12:27 <lmacken> pingou: yeah, completely untested, of course :)
17:12:28 <pingou> one thing I would like to check with Matt, lmacken and Ralph is if we can optimize the DB a little
17:12:29 <smooge> mm2 or mm3. I am a bit confused..
17:12:36 <pingou> smooge: MM2 MirrorManager2
17:12:54 <pingou> lmacken: one table there contains almost 1M records
17:12:55 <smooge> ok nm
17:13:04 <lmacken> pingou: fun.
17:13:04 <nirik> Also, if we want to redo architecture this would be a good time to do so...
17:13:11 <pingou> which sounds like too much for want we would like in there
17:13:16 <nirik> like how the crawlers work, or if there's a better way to do mirrorlists
17:13:21 <pingou> nirik: +1
17:13:25 * lmacken excited to write unit tests for mm2
17:13:31 <lmacken> pingou: do they exist now?
17:13:41 <pingou> lmacken: apparently not yet :)
17:14:02 <nirik> also, if we could fedmsg it... so it can know when updates/composes happen based on that instead of crawling dirs... or perhaps crawl dirs once a day as a backup check.
17:14:04 <lmacken> cool. Writing tests & benchmarks will be fun.
17:14:19 <pingou> #info TODO: write unit-tests
17:14:31 <lmacken> fedmsg integration, optionally client-side too?
17:14:51 <pingou> lmacken: client side becomes tricky wrt ssl certs
17:14:51 * lmacken not sure what code actually runs on the mirrors themselves
17:15:09 <lmacken> pingou: ah, yeah
17:15:10 <nirik> we have a client side script written by adrian...
17:15:10 <pingou> nirik: there was also the question of 'emergency' updates
17:15:32 <nirik> we just need to make sure it's really pushed for mirrors to use and adjusted so it's easy for them
17:15:46 <pingou> it would be nice to have a mirrormanager-client that can be used by the mirrors
17:15:48 <nirik> it just uses datagrepper.
17:16:01 <nirik> so, no real requirements over being able to use the net. ;)
17:16:19 <nirik> mirrors hate to install stuff. it has to be a dirt simple script
17:16:55 <pingou> sure
17:17:01 <nirik> #info fedmsg integration, unit tests
17:17:12 <nirik> #info client side scripts documented/made easy
17:17:12 <lmacken> curling datagrepper to check if it needs to sync would be very lightweight
17:17:18 <lmacken> nirik: that code already exists?
17:17:18 <nirik> yeah, it is.
17:17:23 * nirik gets link
17:17:58 <lmacken> would be cool to have it poll datagrepper by default, but people could enable websockets push notifications if they want
17:18:00 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/4539
17:18:36 <lmacken> very cool
17:19:54 <lmacken> I guess there is value in staggering
17:20:01 <lmacken> syncs. to a certain extent.
17:20:07 <nirik> yeah, don't want a thundering herd.
17:20:57 <nirik> Oh another area we might want to look at is the internet2 stuff in mm1. It's looking like we are going to just drop our I2 link because it's slow and not useful...
17:21:18 <nirik> dunno if it simplifies mm2 to not have that, or if we want it in case we ever add something like it down the road.
17:21:25 <lmacken> tailing the logs the other day, I saw a ton of i2 traffic
17:21:52 <lmacken> is it just not worth it these days?
17:22:11 <nirik> hum, yeah, I guess it might still be in that case.
17:22:24 <nirik> the case where it's not is our download-i2 servers... for i2 mirrors to sync to.
17:22:36 <nirik> but clients could still get better speeds from i2 mirrors.
17:22:38 <nirik> so, nevermind.
17:22:42 <lmacken> we could easily write a script to see what % of traffic is i2
17:23:47 <nirik> On mirrorlists I wonder if we could get a gain from moving to something like nginx + memcached instead of apache + wsgi + pkl... I guess they might be similar...
17:24:11 <nirik> mirrorlists are also something I think might be a good thing to test as containers sometime.
17:25:51 <nirik> Any other areas we can think of? or does that help us narrow down what we want to pick matt's brain on?
17:28:22 <pingou> I don't think the i2 code is too much of an overhead
17:28:29 <pingou> it's simple boolean stored in the DB
17:28:53 <pingou> nirik: the people from centos seemed interesting in pulling in some ideas in the basket about MM2
17:28:57 <pingou> have they got back on this?
17:29:26 <nirik> not that I have seen. Might mail them again now and ask for thoughts or if someone(s) can join us on irc or in person later this week?
17:29:28 <stickster> nirik: One thing that occurs to me is more of a logistics issue. I think the RDU2 colo visit starts at ~10:45 so we need to leave the building at ~10:20 or so to get there for processing. That's not a lot of time for you, puiterwijk, and smooge to be in the brain-picking until we get back from IT meetups in the afternoon. Presumably that's OK?
17:29:36 <pingou> nirik: +1
17:30:10 <smooge> ok have to take car to rental place before time limit. back in a bit and will read the backlog
17:30:15 <smooge> stickster, ok with me
17:30:24 <nirik> stickster: yeah, I figure perhaps we could start at 9am... that would give us an hour or so to at least map things out... then we could do those meetings, etc, and come back and others can work with matt while we do that..
17:30:38 <stickster> nirik: *nod
17:30:50 <nirik> smooge: have a question when you get back... were you going to rent a car for the fad? or only after it?
17:30:59 <pingou> nirik: +1, we'll probably need to leave the hotel before 9 so that we are at work at 9
17:31:29 <pingou> stickster: is there some sort of registration to do at the door?
17:31:34 <pingou> (especially for Matt)
17:31:58 <nirik> pingou: true. it's pretty close tho.
17:32:02 <stickster> pingou: There is a visitors desk on Floor 9 when you arrive, you should check visitors in there.
17:32:04 <nirik> a few blocks walking
17:32:23 <stickster> pingou: I believe any Red Hat employee can do that. There should be more details on the internal RDU wiki page I sent you previously.
17:32:25 <pingou> nirik: I was more worried about a check-in procedure :)
17:32:26 <nirik> oh, we should mail that other local person who said they wanted to attend.
17:32:46 <nirik> see when and what they were interested in. ansible or mm or what
17:33:17 <pingou> stickster: we did ask the reception desk about taxi from the airport right?
17:33:35 <pingou> the wiki page lists something but only from office to airport apparently
17:33:37 <nirik> oh yeah, what was the answer to that?
17:33:41 <nirik> #topic Logistics
17:33:58 <stickster> pingou, nirik: Yes -- no shuttle service from airport to office/downtown.
17:34:05 <stickster> IOW, get a taxi.
17:34:27 <threebean> sorry - I missed the announce on this.
17:34:36 <pingou> stickster: but the wiki page mentions a preferential fair with one company
17:34:55 <nirik> threebean: sorry we didn't advertise better. ;(
17:35:00 <pingou> is that still true? only in for from office to airport?
17:35:17 <pingou> threebean: sorry about that, got sent late to the infra list ;/
17:35:27 <stickster> pingou: I think it's only one way, but the company in question may be able to advise. The receptionist didn't mention it.
17:35:32 <nirik> threebean: http://meetbot.fedoraproject.org/fedora-meeting-1/2014-12-01/infrastructure-pre-fad-meetup.2014-12-01-17.03.log.txt
17:35:38 <nirik> (logs so far)
17:35:44 * threebean nods
17:36:46 <pingou> might be worth double-checking this
17:37:14 <pingou> nirik: do you think you can ask them? or shall I try my best french accent? :)
17:37:36 <nirik> who? taxi place?
17:37:45 <nirik> go ahead if you can. ;) If not I can try...
17:37:49 <pingou> the taxi yes
17:37:53 <pingou> Trivia Taxi Raleigh
17:39:07 * nirik looks for web info
17:39:22 <pingou> http://www.triviataxinc.com/
17:40:46 <nirik> ok.
17:40:59 <stickster> nirik: pingou: I just called them, they cover both ways.
17:41:06 <pingou> stickster: cool :)
17:41:09 <pingou> thanks stickster
17:41:20 <pingou> puiterwijk: I'll call them to book for both of us
17:41:38 <stickster> nirik: pingou: I'll send details to the list. Basically, you'll want to call before you leave, with your name & flight arrival information, and that you are with Red Hat.
17:41:57 <nirik> cool.
17:42:22 <nirik> and they ask you trivia while you ride? ;)
17:42:51 <pingou> I wonder if they have the French edition :)
17:42:59 <stickster> ha
17:44:15 <nirik> ok, any further planning / thoughts?
17:45:32 * pingou is ok
17:45:47 <nirik> I'll look at coming up with ansible side agenda too...
17:45:48 * stickster notes, he will be on the road before 6am, but available on cell (SMS bad while driving!)
17:45:54 <nirik> when do we switch to it? sat? or sunday?
17:46:12 <stickster> I think it was Sat after mdomsch departs
17:46:24 <pingou> Sat is good
17:46:30 <nirik> cool.
17:46:38 <pingou> maybe we can make a break on Sat afternoon
17:46:49 <pingou> nirik: would give us time to do what we spoke about
17:47:23 <nirik> also, FYI logistics wise... thursday is going to have the readyness and go-no/go meetings... so I might need to watch those a bit while in the other meetings with rhit folks.
17:47:31 <nirik> pingou: indeed.
17:48:07 <pingou> nirik: we'll need to figure out how to get there (although I guess it depends on how many we are)
17:48:29 <nirik> yeah
17:48:51 <nirik> did we want to try and plan any dinner/events for sat/sunday evenings? or just figure it out as we go?
17:49:42 <pingou> planning might be nice to avoid splitting the group over two restaurants
17:49:53 <pingou> but it means knowing where we're going in advance :)
17:50:34 <nirik> indeed. perhaps we could ping ruth for ideas? would also be nice to invite spot
17:50:51 <pingou> +1 on both
17:51:46 <nirik> pingou: you want to do that? or want me to add it to my list?
17:52:21 <pingou> nirik: I can write a small email and run it by you :)
17:52:26 <nirik> sure thing
17:52:43 <nirik> also, friday dinner is all set.
17:53:17 <nirik> #info will ask for dinner suggestions for sat/sunday and try and plan them.
17:53:34 <nirik> #info trivia taxi is $30 to/from airport. call to reserve.
17:54:02 <nirik> #topic Open floor
17:54:05 <nirik> anything else?
17:56:40 <nirik> ok, thanks for coming everyone. See you later in the week!
17:56:43 <nirik> #endmeeting