15:29:40 <dgilmore> #startmeeting RELENG (2016-05-09)
15:29:40 <zodbot> Meeting started Mon May  9 15:29:40 2016 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:29:40 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:29:40 <zodbot> The meeting name has been set to 'releng_(2016-05-09)'
15:29:40 <dgilmore> #meetingname releng
15:29:40 <zodbot> The meeting name has been set to 'releng'
15:29:40 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou maxamillion
15:29:40 <zodbot> Current chairs: bochecha dgilmore masta maxamillion nirik pbrobinson pingou sharkcz tyll
15:29:44 <dgilmore> #topic init process
15:30:00 <nirik> morning
15:30:05 * sharkcz is here
15:30:13 * pbrobinson waves
15:32:03 <dgilmore> lets get started
15:32:15 <dgilmore> #topic f24 Beta
15:32:56 <maxamillion> .hello maxamillion
15:32:57 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
15:33:12 <dgilmore> Beta was a bit rough in that a pungi change that broke booting of the dvds as disks was detected really late
15:33:36 <dgilmore> but the compose requests came in really late again
15:33:54 <nirik> that anoying russian roulettee on live images bug is also a rough spot
15:34:26 <dgilmore> I think we need to sit down with adamw, tflink, and the rest of QA and see what we can do to get compose requests sooner
15:34:30 <maxamillion> nirik: I missed that one somehow, what is that?
15:34:38 <masta> hey gang
15:34:38 <dgilmore> nirik: indeed
15:34:56 <tflink> as a side note, adamw is pretty much AFK this week
15:34:56 <nirik> dgilmore: well, they wait until the blockers are addressed... so not sure there's much we can do
15:35:14 <dgilmore> maxamillion: lives randomly fail because /dev/shm in the install environment fails to unmount
15:35:20 <dgilmore> and only seems to happen in koji
15:35:26 <maxamillion> tflink: oh? hopefully he's off doing something fun :)
15:35:33 <tflink> AFAIK, yes
15:35:41 <nirik> maxamillion: https://bugzilla.redhat.com/show_bug.cgi?id=1315541 basically sometimes randomly live's fail to compose.
15:35:42 <dgilmore> tflink: okay thanks
15:35:43 <maxamillion> good for him :)
15:35:52 <maxamillion> nirik: ah, fun ... thanks for the info
15:35:54 <masta> hrm... /dev/shm
15:36:03 <pbrobinson> tflink: so are you going to wrap his virtual desk in paper for his return?
15:36:06 <dgilmore> tflink: do you have ideas on how we can gang up on folks to fix blockers better and faster
15:36:23 <tflink> dgilmore: more phasers set to "annoy"?
15:36:30 <nirik> well, /dev/shm or /dev/null or whatever
15:36:51 <tflink> i don't recall if that was a blocker that arose from a test escape or not
15:36:52 <dgilmore> tflink: perhaps we can set fmn notifications every 12 hours
15:37:13 <tflink> pbrobinson: I'll have to think of a way to actually do that :)
15:37:33 <tflink> dgilmore: I'd worry about people tuning out the notifications if they were that often
15:38:05 <dgilmore> tflink: indeed, it would need to be non optional, but then people could just /dev/null them
15:38:06 <nirik> I think those emails with a recap adamw was sending out help... perhaps do those eariler?
15:38:11 <tflink> FWIW, late composes aren't any fun for us, either
15:38:20 <nirik> ie, going over the bugs and who needs to do something for each
15:38:24 <pbrobinson> yes, I agree with nirik on the recap emails with people cc:ed
15:38:28 <dgilmore> tflink: I know, it puts us all under the crunch
15:39:04 <tflink> beyond the discussion which happens in bz?
15:40:05 <tflink> this discussion will probably be more productive with adamw instead of me, though
15:40:18 <dgilmore> anyway, we should plan a fedora 24 recap
15:40:20 * tflink has been working more on automation as of late
15:40:34 <dgilmore> #action dgilmore to plana  Fedora 24 releng recap
15:40:54 <dgilmore> tflink: well with more people in general
15:41:23 <tflink> dgilmore: sure, I meant mostly with adamw representing QA instead of me :)
15:41:30 <dgilmore> I am going to write a script that that does the release staging,
15:42:56 <dgilmore> does anyone have anything to cover for Beta?
15:43:14 <dgilmore> pbrobinson, sharkcz: anything that went really well or bad for secondaries?
15:43:59 <pbrobinson> dgilmore: so arm and ppc looks good. Still battling with storage for s390, I've got storage working but having issues with getting a runroot complete
15:44:00 <sharkcz> I would say all normal, with little progress on the s390 side, could work, but is really slow (for now)
15:46:01 <sharkcz> build-wise we are very close to primary
15:46:46 <pbrobinson> yep, agreed
15:47:54 <dgilmore> cool
15:48:42 <dgilmore> lets move on
15:48:43 <dgilmore> #topic Secondary Architectures updates
15:48:43 <dgilmore> #topic Secondary Architectures update - ppc
15:48:59 <dgilmore> i assume ppc is ready for Beta?
15:49:08 <pbrobinson> pretty much
15:49:32 <pbrobinson> we're just doing final checks but I believe we're good to go
15:49:55 <dgilmore> awesome
15:50:18 <dgilmore> #info ppc doing final checks, however Beta looks good
15:50:41 <dgilmore> anything we need to note?
15:50:50 <dgilmore> how is the hub migration looking?
15:50:51 <pbrobinson> nothing from me
15:50:56 <dgilmore> #topic Secondary Architectures update - s390
15:51:26 <dgilmore> sharkcz: beta is a bit delayed as we deal with runroot?
15:51:34 <pbrobinson> still trying to get the runroot stuff completing, other than that everything looking OK, will get it completed this week one way or another
15:51:48 <sharkcz> yep, as pbrobinson mentioned we need to solve the access speeds to /mnt/koji
15:52:21 * dgilmore will brb, need to deal with cat allergies
15:52:34 <pbrobinson> dgilmore: shotgun?
15:53:18 <nirik> pbrobinson: what all did you need me to do on the new ppc hub? just look at it and see it looks ok, and do a db dump/restore from the old one? I would expect it to be fine since it was made just like the others.
15:53:37 <pbrobinson> nirik: yep, pretty much
15:53:56 <nirik> pbrobinson: is the old hub running the latest koji? or older?
15:54:29 <pbrobinson> nirik: um, might just need the signed repo DB change, but other than that it's up to date
15:55:07 <pbrobinson> nirik: being moved to a new IP I'm not sure if there's other infra changes needed too (firewall/load balancer/NAT/PAT or related)
15:55:46 <nirik> oh yeah, thats a RHIT ticket. ;( When did we want to move it? I can try and line up RHIT folks to do the change then...
15:56:03 <pbrobinson> nirik: for me when ever is good
15:56:05 <nirik> Just wondering on versions if we need db schema updates. Sounds like not.
15:56:35 <pbrobinson> nirik: yes, the signed repo change is just a flag change I think
15:57:51 <dgilmore> back
15:58:11 <nirik> pbrobinson: will see how the rest of my week looks and we can schedule something.
15:58:29 <nirik> it's already pretty packed since freeze will be up and I have a pile of things pending.
15:58:40 <pbrobinson> nirik: early next week?
15:58:45 <nirik> might be good.
16:01:51 <dgilmore> cool
16:02:16 <dgilmore> #info s390 working on sorting out builder access to /mnt/koji
16:02:24 <dgilmore> #topic Secondary Architectures update - arm
16:02:26 <dgilmore> pbrobinson: hows aarch64?
16:02:44 <pbrobinson> signed off, signed and pushed to the mirrors
16:03:08 <dgilmore> #info aarch64 Beta on the mirrors and ready to go
16:06:25 <dgilmore> I am guessing nothing else
16:06:45 <dgilmore> #topic spin-kickstarts and comps future
16:06:47 <dgilmore> nirik:
16:06:53 <dgilmore> nirik: you're up
16:06:57 <nirik> yeah, so I want to move these soonish...
16:07:21 <nirik> is: fedora-comps and fedora-kickstarts ok names in pagure?
16:07:48 <dgilmore> I think so
16:07:52 <nirik> and I was thinking of moving them friday morning. we will need to adjust fedora-pungi at least...
16:08:07 <dgilmore> I think the atomic installer lorax templates should live somewhere else
16:08:14 <dgilmore> ideally packaged up
16:08:41 <nirik> ok.
16:08:43 <pbrobinson> +1 from me for fedora-comps and fedora-kickstarts
16:08:43 <maxamillion> +1
16:08:50 <maxamillion> I like the names, think they make sense
16:08:58 <nirik> might need also to change koji (to allow pagure) and builders firewalls
16:09:02 <maxamillion> spin-kickstarts probably hasn't been accurate for at least a few years
16:09:21 <dgilmore> we will need to make some firewall changes and kojid config changes
16:09:35 <nirik> yeah, I can line up those changes
16:10:08 <nirik> also did we want to change anything about the workflows?
16:10:21 <nirik> right now comps is just master branch always with all files.
16:10:30 <nirik> spin-kickstarts has a bunch of commiters
16:10:34 <dgilmore> yeah
16:11:00 <dgilmore> I would like to cut down the kickstarts commiters and just use pull requests
16:11:02 <nirik> I am not sure branches would help comps any.
16:11:12 <pbrobinson> I think we should move the kickstarts to a pull request model
16:11:31 <dgilmore> I am not sure that branches would help comps, and it would take work in bodhi and other places to deal with branches
16:11:41 <pbrobinson> nirik: yes, I agree with not much point having branches for comps
16:11:47 <nirik> seems reasonable to me, since we have always wanted to freeze better for kickstarts.
16:12:01 <dgilmore> but I would like to setup jenkins jobs for spin kickstarts that run validation on all pull requests
16:12:03 <nirik> should comps stay open to all packagers?
16:12:19 <maxamillion> I'll elect to remove myself from kickstarts committer access, I would be happy submitting pull requests
16:12:20 <dgilmore> that should have been comps
16:12:31 <pbrobinson> we don't tend to have too many problems with it so I don't see a major issue there
16:12:34 <nirik> dgilmore: we might be able to do a validate in a hook?
16:12:51 <dgilmore> nirik: maybe
16:13:17 <nirik> but yeah, a validate is really a good idea there, no matter how it's technically done.
16:13:26 <dgilmore> I am pretty sure that most packagers do not realise that they can edit comps
16:13:55 <nirik> yeah.
16:14:24 <nirik> for that matter for kickstarts we could run ksvalidator on PR's
16:14:39 <nirik> lots of automation available if we sink some time into it
16:14:53 <dgilmore> nirik: indeed
16:15:33 <dgilmore> running ksvalidator on pull requests seems like a good idea to me
16:15:59 <nirik> I am not sure we can actually have comps open to all packagers on pagure... asking about it now.
16:16:10 <dgilmore> nirik: I am leaning towards comps being limited to some subset and everyone can submit pull requests
16:16:32 <dgilmore> that subset should be releng group
16:16:39 <dgilmore> and probably qa
16:17:12 <nirik> well, groups on pagure are different from fas groups tho
16:17:27 <dgilmore> nirik: right, there is a pagure releng group
16:17:42 <nirik> sure, could allow that...
16:18:05 <nirik> the PR model should be fine, as long as we are good about merging in a timely manner.
16:18:13 <dgilmore> right
16:18:24 <maxamillion> +1
16:18:55 <maxamillion> PR model has more or less become the defacto standard, I'd be surprised if anyone had a big complaint about it
16:18:56 <dgilmore> it will need people to quickly respond, either merge or provide feedback
16:19:31 <dgilmore> maxamillion: I think the main issue is making sure PR's do not sit there for periods of time
16:19:40 <nirik> ok. I can mail devel-announce, and spins list and releng list about this migration...
16:19:52 <maxamillion> dgilmore: ah, yes +1
16:20:21 <dgilmore> #action nirik to email devel-announce, spins and releng lists about the migration
16:20:47 <nirik> also, for fedora-spins I am planning to try and find contacts for each of the ks'es
16:20:57 <nirik> so we can actually assign tickets to them
16:20:59 <maxamillion> +1
16:21:09 <dgilmore> #info fedora-comps and fedora-kickstarts will move to a pull request based model. with releng  and QA groups in pagure having accees to merge
16:21:18 <nirik> and if there's spins with no 'owners' we should look at dropping them next cycle
16:21:20 <dgilmore> nirik: that will be awesom
16:21:32 <nirik> or asking for someone to step up
16:21:46 <dgilmore> #action nirik to ensure all spins have owners
16:22:41 <dgilmore> #info automation of validating kickstarts and comps changes to be taken
16:22:59 <nirik> I might hand off the validation part to someone else after the migration...
16:23:05 <nirik> if anyone has cycles to work on it
16:23:18 <dgilmore> #action dgilmore to talk to walters about an appropriate home for the lorax templates for atomic installer dvd
16:23:33 <nirik> dgilmore: are those in spin-kickstarts now?
16:23:45 <dgilmore> nirik: they are
16:24:08 <dgilmore> nirik: I was thinking of asking lsedlar to help with the automation, he setup pungi's
16:24:21 <nirik> so if we move without moving those we break that. ;(
16:25:12 <dgilmore> nirik: yeah
16:25:35 <dgilmore> nirik: we may be able to just move it in where the atomic json file lists
16:25:38 <dgilmore> lives
16:25:50 <dgilmore> https://pagure.io/fedora-atomic
16:25:59 <nirik> ok. we could also keep it in fedora-kickstarts for a short while until it finds a new home
16:26:25 <dgilmore> I think fedora-atomic would be a good short term stopgap
16:26:38 <nirik> ok, we can sort it out of meeting anyhow.
16:26:42 <dgilmore> yep
16:26:46 <dgilmore> anything else?
16:27:24 * paragan thinks moving more projects to Pagure means, more information about how to use pagure need to be either blogged or sent on devel list. I don't think its exact replica of github and people need to aware of pagure usage.
16:28:40 <dgilmore> paragan: likely it would help. I expect that eventually fedorahosted will go away and we will have only pagure
16:28:51 <dgilmore> #topic Open Floor
16:28:57 <nirik> well,t he workflow is pretty similar to github, but yeah, more info is always good
16:29:07 <dgilmore> does anyone have anything else?
16:29:41 * nirik can try a blog or two on it... not sure how soon tho. Perhaps the commops folks will be able to do a few.
16:29:59 <paragan> yes
16:30:12 <maxamillion> nirik: +1 - might be good to get with them and see if we can get something on the Fedora Magazine site or $similar
16:30:45 <nirik> They did have some pagure articles a while back, but more specific ones would be nice.
16:30:53 <nirik> like how to make a PR
16:32:52 <dgilmore> nirik: and updating a PR
16:33:08 <dgilmore> how to make and update your feature branch etc also
16:33:15 <nirik> yep. all great ones.
16:33:16 <dgilmore> anyway
16:33:31 * nirik has nothing more for the meeting. ;)
16:33:36 * maxamillion either
16:33:40 <dgilmore> if no one has anything else we are 3 minutes over
16:33:46 <dgilmore> #endmeeting