15:33:28 <dgilmore> #startmeeting RELENG (2015-04-21)
15:33:28 <zodbot> Meeting started Mon Apr 20 15:33:28 2015 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:33:28 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:33:35 <dgilmore> #meetingname releng
15:33:35 <zodbot> The meeting name has been set to 'releng'
15:33:40 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou maxamillion
15:33:40 <zodbot> Current chairs: bochecha dgilmore masta maxamillion nirik pbrobinson pingou sharkcz tyll
15:33:49 <dgilmore> #topic init process
15:33:52 <bochecha> hi
15:33:53 <dgilmore> who is here
15:33:56 <nirik> morning
15:34:04 <pingou> o/
15:34:09 <maxamillion> o/
15:34:21 * sharkcz is here
15:34:48 <dgilmore> lets start by welcoming maxamillion
15:34:55 <pingou> welcome maxamillion
15:35:00 * nirik waves. welcome!
15:35:05 <dgilmore> he is tasked with helping us to become more flexible and dynamic
15:35:16 <maxamillion> Hello all! I'm extremely excited to be here and I hope I don't screw everything up ;)
15:35:48 <dgilmore> dont worry we know where you live ;)
15:36:02 <maxamillion> o.O;
15:36:24 <dgilmore> hopefully one of teh first things maxamillion will help us with is fixing docs
15:36:38 <dgilmore> and pointing out issues where they need work
15:36:39 <maxamillion> once I learn my way around, I will be happy to
15:36:59 <maxamillion> I've found a bunch of docs that need love, just don't know the content they need to be up to date (yet)
15:37:02 <dgilmore> #topic #6064 extend srpm-excluded-arch.py so it can read srpms from multiple dirs
15:37:09 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6064
15:37:22 <dgilmore> this just needs to have the patches reviewed
15:38:21 <dgilmore> one thing I think it needs is the argparse changes pulled out
15:38:44 * maxamillion looks
15:39:24 <sharkcz> dgilmore: like splitting into 2 changes - argparse + extending the function?
15:39:29 <pingou> dgilmore: where should it run?
15:40:20 <dgilmore> sharkcz: yes
15:40:21 <maxamillion> I'm pretty indifferent about argparse vs optparse so I don't have any comment there
15:40:36 <sharkcz> pingou: that another question, my idea would be at the end of updates push
15:40:36 <dgilmore> I do not know if we should switch to argparse
15:40:44 * dgilmore does not follow upstream close enough
15:40:56 <maxamillion> I'm curious why the decision on a certain depth to glob for in the 'for srpm_path in args.path:' though
15:41:03 <pingou> dgilmore: argparse is not part of stdlib
15:41:21 <dgilmore> maxamillion: I do not have a strong opinion of optparse vs argparse
15:41:26 <pingou> since 2.7
15:41:33 <maxamillion> dgilmore: oh ok, I mis-understood
15:41:42 <pingou> it is still outside for 2.6 and below
15:41:49 <pingou> s/not/now/
15:41:59 <pingou> argparse is part of stdlib
15:42:15 <dgilmore> pingou: not beingpart of teh stdlib would be a reason not to switch to it
15:42:22 <pingou> while optparse is deprecated since 2.7
15:42:24 <dgilmore> gahh
15:42:31 <dgilmore> pingou: okay
15:42:47 <pingou> so depending on which host the script runs, I think we should switch (or wait)
15:42:50 <dgilmore> as I said I do not follow upstream python all that closely, I do not have the time to do so
15:43:02 <pingou> dgilmore: team work ftw ;-)
15:43:03 <dgilmore> pingou: it runs nightly on rhel7
15:43:17 <maxamillion> then +1
15:43:25 <pingou> which has 2.7 so we're good there :)
15:43:31 <dgilmore> we do need to setup a process to run it regullary for stable fedora's
15:43:59 <dgilmore> we could probably do that as part of the updates pushes or as part of the nightly branched/rawhide composes
15:44:12 <dgilmore> updates pushes today are rhel 6 hosts
15:46:19 <dgilmore> I sent an email asking ofr it to be slipt into two patches
15:47:10 <dgilmore> #topic #5886 need method for distributing urgent fixes... urgently
15:47:16 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5886
15:47:33 <dgilmore> I think we failed in the followup here
15:47:37 <nirik> I sent a query to the test list.
15:47:42 <nirik> I got 0 replies. ;)
15:47:48 <nirik> but they might have all been busy with beta.
15:48:34 <dgilmore> so the things we know for sure it is not as simple as signing some rpms and mashing the repo
15:48:45 <nirik> right.
15:48:49 <dgilmore> we need updateinfo.xml we need an attomic tree
15:48:57 <dgilmore> atomic tree
15:49:15 <dgilmore> we also potentially need to spin all cloud images and livecds
15:49:23 <dgilmore> or some subset of them
15:49:53 <maxamillion> would we need to spin the livecds? I thought post-GA the liveCDs are static, no?
15:50:08 <dgilmore> I think the security reponse team should be the ones that evaluate what exactly needs to be updated
15:50:17 <dgilmore> maxamillion: potentially
15:50:27 <dgilmore> depends on the bug
15:50:31 <nirik> I think thats really another (related) issue.
15:50:44 <maxamillion> dgilmore: rgr
15:50:48 <pingou> two steps?
15:50:56 <nirik> I suppose it could be part of this tho...
15:51:05 <nb> when have we respun liveimages yet? just for heartbleed?
15:51:07 <dgilmore> maxamillion: see http://dl.fedoraproject.org/pub/fedora/linux/updates/20/Images/
15:51:15 <nirik> but also there's things like the cloud folks wanting respins for... whatever the feel like.
15:51:17 <dgilmore> we made new arm images and cloud images
15:51:34 <nirik> nb: yes, but we never distributed them. they were broken
15:51:36 <dgilmore> there is going ot be frequent cloud respins
15:51:37 <nb> oh
15:51:55 <maxamillion> dgilmore: ah
15:51:58 * pbrobinson is here
15:52:06 <maxamillion> pbrobinson: o/
15:52:13 <nirik> anyhow. I would say lets leave this another week and see if we get more feedback in a post beta world. ;)
15:52:23 <dgilmore> welcome pbrobinson
15:52:28 <nirik> hey pbrobinson
15:52:33 * pbrobinson waves
15:52:40 <dgilmore> nirik: okay, lets be sure to ping the messages.
15:52:45 <pingou> o/ pbrobinson
15:52:47 <dgilmore> just to bring it to the forfront
15:52:51 * masta is here
15:52:52 <dgilmore> forefront
15:52:58 <masta> sry was distracted with the push
15:54:26 <dgilmore> #info going to follow up this week with Beta out the door
15:54:31 <dgilmore> #topic #6108 Migrate to well-known CA for pkgs.fedoraproject.org
15:54:37 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6108
15:54:46 <dgilmore> so this needs a lot of planning and a flag day
15:54:58 <dgilmore> there is a lot of change we really need to roll into this
15:55:08 <dgilmore> it also will mean changes for koji.fp.o
15:55:58 <nirik> perhaps we should schedule a date after final / before f23 ramps up... and work back from that on what needs doing when?
15:56:08 <dgilmore> nirik: likely yeah
15:56:21 <pbrobinson> sounds like good timing
15:56:22 <dgilmore> ill remove the meeting keyword for now
15:56:35 <pbrobinson> and add the planning one? :-P
15:57:08 <dgilmore> we do not have a planning one
15:57:11 <dgilmore> but sure
15:57:48 <nirik> we could use tracs milestone setup more than we do.
15:58:00 <dgilmore> nirik: yeah, I do a bit of a bad job of it
15:58:11 <dgilmore> I do try to update it all when I remeber
15:58:21 <nirik> pingou: side note: milestones for pagure? ;)
15:59:32 * dgilmore just added F23 milestones
15:59:44 <pingou> nirik: hopefully 0.1 by the end of the week :)
16:00:24 <dgilmore> pingou: cool :)
16:00:30 <dgilmore> okay lets move on
16:00:42 <dgilmore> #topic Secondary Architectures updates
16:01:09 <dgilmore> generic secondary arch issue
16:01:11 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6149
16:01:20 <dgilmore> Fedora secondary arches .composeinfos are missing and need to be made
16:01:28 <dgilmore> we really need a good way to do this
16:01:49 <dgilmore> the way it is done for primary breaks on secondaries due to how we merge the arches together
16:02:11 <dgilmore> a quick fix is to move up one directory and run the build_composeinfo script there
16:02:17 <dgilmore> so in the arch directory
16:02:40 <pbrobinson> maybe something to review once we have distill/pungi4 in f-23?
16:02:59 <dgilmore> so run it http://dl.fedoraproject.org/pub/fedora-secondary/releases/21/Server/aarch64/ http://dl.fedoraproject.org/pub/fedora-secondary/releases/21/Server/ppc64/ etc
16:03:00 <pbrobinson> I think it deals with it OK for secondaries in the enterprise space
16:03:16 <dgilmore> pbrobinson: that will do something completely different
16:03:18 <maxamillion> for those of us new to the group, what's .composeinfos?
16:03:24 <dgilmore> but we do need something for f22
16:03:58 <sharkcz> and possibly for f21 too, which is now broken for beaker
16:04:01 <dgilmore> maxamillion: http://dl.fedoraproject.org/pub/fedora/linux/releases/21/Server/.composeinfo
16:04:14 <dgilmore> maxamillion: afaik the only user of it is beaker
16:04:24 <nirik> it's a file that some tools use to find out whats in a tree... beaker mainly
16:04:25 <maxamillion> ah
16:04:31 <maxamillion> thanks
16:04:35 <dgilmore> it defines all the pieces of the compose
16:04:43 <dgilmore> so that tools can get arches etc
16:04:58 <dgilmore> it tells them where to find the .treeinfo files
16:05:17 <dgilmore> http://dl.fedoraproject.org/pub/fedora/linux/releases/21/Server/i386/os/.treeinfo
16:05:42 <dgilmore> the .treeinfo file tells tools where to get the kernel, dtb, initrd etc
16:07:23 <dgilmore> so short term we likely make the .composeinfo file for each arch
16:07:31 <dgilmore> longer term we need something better
16:07:47 <dgilmore> so if people have ideas, please chime up in the ticket
16:07:56 <dgilmore> #topic Secondary Architectures update - ppc
16:08:08 <dgilmore> pbrobinson: sharkcz: how is ppc coming along?
16:08:37 <pbrobinson> RC1 went out this morning
16:08:43 <pbrobinson> beta rc1 even
16:08:53 <dgilmore> :) excellent
16:08:58 <sharkcz> build-wise looks good, a toolchain bug got fixed, etc
16:09:08 <dgilmore> do we think we will catch up by final and release on the same day?
16:09:13 <pbrobinson> we're getting there, trying to get the Power8 stuff up and running this week so I can do other images etc. Just need more time!
16:09:26 <dgilmore> always need more time :)
16:09:30 <pbrobinson> I think we're pretty close now TBH
16:09:35 <maxamillion> is .treeinfo pulled by yum/dnf or $other?
16:09:51 <dgilmore> maxamillion: its used by beaker and anaconda and virt tools etc
16:10:00 <nirik> pbrobinson: you planning on having the power8 stuff in ansible from the start?
16:10:00 <pbrobinson> I think if the few blockers are confirmed fixed we should be good to go with RC1 with luck
16:10:07 <pbrobinson> nirik: yes!!
16:10:11 <nirik> excellent.
16:10:12 <maxamillion> dgilmore: rgr, thanks
16:10:43 <pbrobinson> nirik: I'm just trying to work it all out now to get an initial install done and then will play with ansible
16:10:59 <nirik> right. sounds goodly. let me know if I can assist with anything.
16:11:14 <dgilmore> #info PPC is looking good
16:11:23 <pbrobinson> nirik: Oh I will!!
16:11:29 <dgilmore> #info pbrobinson working to get power 8 boxes up and running and in ansible
16:12:04 <maxamillion> pbrobinson: +1
16:12:10 <maxamillion> power8 is supposed to be swank
16:12:25 <pbrobinson> maxamillion: it is from my experience :)
16:12:29 <dgilmore> #topic Secondary Architectures update - s390
16:12:36 <dgilmore> sharkcz: how is s390?
16:13:24 <sharkcz> looks good, I've prepared Beta RC1 today, the vnc issue is fixed, will send an announce shortly
16:13:29 <nirik> on s390. I am going to try "soon" (yeah, dunno exactly as I am busy) to make a new hub/db for s390 on the new machine... then we can dump/load the db on it and see how it looks and figure out cutover etc.
16:13:55 <pbrobinson> woo! more ansible :)
16:14:02 <pbrobinson> on secondary arches
16:14:04 <sharkcz> nirik: ok
16:14:16 <nirik> sharkcz: will ping you when I have something for you to look at. ;)
16:14:33 <sharkcz> nirik: ok :-)
16:14:40 <dgilmore> nirik: would be nice to get it moved over shortly after f22 ga
16:14:51 <sharkcz> we are all busy especially now before GA
16:15:02 <nirik> yeah, I am hoping I can get it setup and we can poke at it and then yeah, cut over after f22
16:15:10 <maxamillion> pbrobinson: +1 - I do <3 me some ansible
16:15:12 <dgilmore> :) perfect
16:15:35 <dgilmore> #info s390 Beta RC1 out today
16:15:56 <dgilmore> sharkcz: do you think final will be the same time as primary or shortly after?
16:16:22 <sharkcz> dgilmore: I would say shortly after primary
16:16:28 <dgilmore> sharkcz: okay
16:16:41 <dgilmore> #info s390 GA likely shortly after primary GA
16:16:42 <sharkcz> it should be all ready, but ppc has higher prio
16:16:47 <dgilmore> #topic Secondary Architectures update - arm
16:16:56 <dgilmore> pbrobinson: how is aarch64?
16:16:57 <pbrobinson> beta RC1 was out today too
16:17:02 <pbrobinson> we're looking pretty good
16:17:16 <dgilmore> thinking GA same day as primary?
16:17:30 <dgilmore> #info Beta RC1 out today
16:17:30 <pbrobinson> still some minor ongoing issues with a few ghc packages but we're on track I think now for same or close to GA
16:17:40 <dgilmore> okay awesome
16:17:56 <pbrobinson> again trying to get some qemu images and a SD card image working with live media creator
16:18:11 <dgilmore> okay
16:19:00 <dgilmore> #info ghc still causing a few small issues, but GA should be on schedule
16:19:19 <dgilmore> #topic Open Floor
16:19:59 <dgilmore> I want to bring up the FAD
16:20:01 <dgilmore> https://lists.fedoraproject.org/pipermail/rel-eng/2015-April/019753.html
16:20:17 <dgilmore> we need to have some discussions about what we will deliver
16:20:36 <dgilmore> we likely should setup some tickets. use the planning keyword
16:20:36 <maxamillion> I'm probably going to be nagging almost everyone here at some point this week as I get more oriented with things, apologies in general for sucking time and if I'm interrupting something please feel free to tell me to buzz off, I will not be offended :)
16:20:42 <dgilmore> and I can setup a trac report
16:20:43 <maxamillion> dgilmore: +1
16:21:44 <pbrobinson> maxamillion: I won't be offended, will do my best
16:22:45 <pbrobinson> maxamillion: also interested in what you're actually tasked to work on, everyone seems to be saying you're going to be the rel-eng messiah..... so I wanted to have an understanding of what you're actually doing :)
16:23:01 <maxamillion> dgilmore: I think there's a lot of discussions to be had to try and hash out some of that work, yes? I had a meeting last thursday with some internal folks and I think there needs to be follow ups to frame the requirements for what's more or less being called "koji 2.0" and then make a public proposal for review/feedback/discussion ... thoughts?
16:23:17 <maxamillion> pbrobinson: lol, oh man ... messiah ... jeez
16:23:32 <maxamillion> someone's been drinking too much and telling you lies ;)
16:23:40 <dgilmore> https://fedorahosted.org/rel-eng/report/11
16:23:43 <pbrobinson> maxamillion: they weren't my words.... I just responded with "does that make him a very naughty boy?" ;=P
16:23:45 <dgilmore> we have a planning Report
16:23:46 <pbrobinson> :-P
16:23:50 <maxamillion> pbrobinson: heh
16:24:23 <dgilmore> maxamillion: yes there needs to be much more plubic discussion on it
16:25:10 <maxamillion> pbrobinson: to start with I'm tasked with helping to make a "stage" or "devel" environment for koji so that we can deploy dev builds without interrupting the whole world (what all that will take I still don't know yet, need to learn), also there's the topic of git workflow to allow us to iterate better as a group so that master branch isn't dev and prod at the same time
16:25:44 <maxamillion> pbrobinson: then the new (either brand new or refactored) build tooling for handling Fedora.Next stuff, atomic, docker, $whatever_new_hotness_comes_next, $other
16:26:05 <maxamillion> pbrobinson: and I'm sure there's going to be overlap with others, where applicable I would love to work with others and not just run around like a crazy person
16:26:12 <nirik> note that we do have a staging koji, but it's kinda busted. Needs fixing, but otherwise we had it working not long back
16:27:11 <maxamillion> nirik: I did see notes on staging-infra.txt in infra-docs ... that was on my list of things to dig into, I'm just right now learning how much I don't know so I know what all to dig into this week and try to be up to speed asap
16:27:22 <nirik> sure sure.
16:27:40 <maxamillion> I have *some* perspective as a packager and from working with koji tags and such but there's a loooot of the backend bits of rel-eng I need to dig into :)
16:27:41 <pbrobinson> maxamillion: OK, so an intro of yourself to the appropriate rel-eng list would likely be useful with an overview. Ultimately I want to make sure you're on the same page as dgilmore and I so you're producing useful stuff :)
16:27:54 <maxamillion> pbrobinson: +1
16:28:01 <dgilmore> that is one of the steps in joing releng ;)
16:28:04 <dgilmore> joining
16:28:25 <maxamillion> I'll do that after this meeting
16:28:43 <pbrobinson> maxamillion: I found when I joined from doing ARM stuff for 4+ years even that was widely different from a OS release perspective :-)
16:29:56 <dgilmore> maxamillion: I am sure we will be frequently talking
16:30:33 <maxamillion> dgilmore: absolutely, I think the three of us (and possibly others) will need to pretty regularly
16:33:00 <dgilmore> maxamillion: one thing I want in the changes is to move secondaries to using the exact same processes.
16:33:15 <dgilmore> we also need to sort out the deliverables for f23
16:33:25 <dgilmore> something FESCo has taken on board to do
16:33:49 <maxamillion> ah ok, good to know
16:33:52 <dgilmore> maxamillion: porting to python3, dnf, etc needs to be on the agenda also
16:34:11 <pbrobinson> diglmore: I'm slowly getting there on secondaries but basically was to some degree awaiting the move to distill so I didn't have to do it twice
16:34:13 <dgilmore> knowing what we have to deliver helps us know what tooling work we need to do
16:34:33 <maxamillion> dgilmore: yes, absolutely
16:34:36 <dgilmore> pbrobinson: sure. It just needs to be a focus in the move in f23
16:35:02 <pbrobinson> dgilmore: absolutely!!!
16:35:15 <maxamillion> dgilmore: I remember talking about py3 porting during the interview so I bought and read a copy of http://python3porting.com/ during my paternity leave :)
16:35:36 <pbrobinson> I should have more time, in theory at least, in a month from around May 20th as my other "enterprise" project closes out
16:35:42 <dgilmore> maxamillion: :) nice, I need to get up to speed on py3
16:36:11 <pbrobinson> maxamillion: read during all the insomnia?
16:36:15 <maxamillion> there's still things I'm sure I'm not up to par on but I feel less lost about it now, the book was pretty good even though there were a few random typos
16:36:39 <maxamillion> pbrobinson: yeah, when he wouldn't sleep unless held I tried to read instead of just watch TV ... sometimes it worked, sometimes it would put me to sleep
16:36:49 <pbrobinson> :-P
16:37:58 <dgilmore> maxamillion: I did not sleep through the night for about 3 years when my daughter was born
16:38:14 <masta> yeah
16:38:47 <masta> my 11 month old wakes up about every 90 minutes now. Probably because he is cutting his teeth.
16:39:34 <maxamillion> dgilmore: yeah, I'm pretty sure I'm on track for something similar .... I've not slept more than 3 hours straight since he was born and that was only because my wife and I would take turns staying up with him
16:40:17 <masta> maxamillion: time becomes a precious thing
16:41:02 <dgilmore> maxamillion: you adjust :)
16:41:30 <dgilmore> do we have anything specific to discuss left?
16:41:41 <dgilmore> I opened a ticket for porting to pungi 4
16:41:58 <dgilmore> feel free to open tickets for other changes we need to make
16:42:03 <maxamillion> dgilmore: I'd like to sync with you about the pungi 4 porting tasks but that's outside the scope of the meeting
16:42:07 <dgilmore> be sure to use the planning keyword
16:42:14 <dgilmore> maxamillion: sure
16:42:29 <maxamillion> trying to think of things to bring up about the FAD but everything I think of kind of lands back at a "well ... we need to talk about that with PM more first"
16:42:33 <dgilmore> maxamillion: the code doesnt even run right now
16:42:41 <maxamillion> oh, fun
16:43:13 <dgilmore> some things got renamed in the open sourcing of the pieces
16:43:18 <dgilmore> it needs to be fixed up
16:43:40 <maxamillion> ah
16:44:20 <maxamillion> that's another thing I'm trying to sort out, there's a lot of institutional knowledge and history that I imagine most of my week will be spent just trying to absorb
16:44:42 <dgilmore> maxamillion: there is, sadly too much of it is still only in my head
16:44:53 <dgilmore> though it is much much better than it used to be
16:44:55 <maxamillion> dgilmore: I shall extract it and write it down! :D
16:45:01 <masta> dgilmore: cool. We can probably look at the diff internally and see what definitions changed, then find where to change those in the scripts
16:45:08 <maxamillion> documentation for the documentation gods!
16:45:52 <dgilmore> masta: sure it could be fixed
16:45:58 <masta> dgilmore: how have you tested distill so far? got a compose config stashed somewhere?
16:46:34 <masta> dgilmore: I've troubleshot distill a bunch, I'm happy to get involved... soon I'll be having more time to address such things.
16:46:58 <dgilmore> masta: I have just tested it locally
16:47:12 <dgilmore> masta: I have some xml
16:47:22 <dgilmore> but its busted
16:47:34 <masta> dgilmore: ok cool, I'll sync with you later about that
16:48:18 <pbrobinson> maxamillion: I've been doing this for 8 months and are still learning... you won't learn it all this week ;-)
16:49:04 <maxamillion> pbrobinson: oh no, I don't think I will ... but I like to think I can absorb enough this week to start being useful next week .... I just don't like not being useful :)
16:49:18 <dgilmore> lets wrap up the meeting, we can continue discussion in #fedora-releng
16:49:38 <dgilmore> #endmeeting