15:40:38 #startmeeting RELENG (2015-05-11) 15:40:38 Meeting started Mon May 11 15:40:38 2015 UTC. The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:40:38 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:40:44 #meetingname releng 15:40:44 The meeting name has been set to 'releng' 15:40:44 #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou maxamillion 15:40:44 Current chairs: bochecha dgilmore masta maxamillion nirik pbrobinson pingou sharkcz tyll 15:40:47 #topic init process 15:40:53 * masta is here 15:40:58 howdy folks 15:41:11 * maxamillion is here 15:41:13 hi there 15:41:14 * pbrobinson is here 15:41:20 bloody squirrel 15:41:22 * nirik is here 15:41:23 * sharkcz is here 15:41:31 * bochecha is here 15:41:37 he is in my back yard digging up where I just put down grass seed 15:42:02 it's legal to own guns there, don't see your issue! 15:42:19 never eaten squirrel 15:42:21 pbrobinson: it's not legal to fire them in residential areas though ... it's not quite the wild west anymore 15:42:37 maxamillion: it is illinois, kinda is :P 15:42:43 dgilmore: heh 15:42:58 bb gun. ;) Just startle him 15:43:12 okay 15:43:21 LOL 15:43:22 #topic #6158 Request to discuss Rel-Eng Project Planning Proposal 15:43:26 Your daughter could use a dog dgilmore 15:43:28 https://fedorahosted.org/rel-eng/ticket/6158 15:43:45 masta: I had one, the ex has him 15:43:56 so maxamillion setup a meeting for this 15:43:59 I did 15:44:14 to discuss with others so we get something useful for all of fedora 15:44:17 after talking about it here last week it kind of seemed like something larger in scope than just the rel-eng group 15:44:22 +1 15:44:29 though QA is heavily invested inphabricator 15:44:46 and does not seem to want to change 15:44:48 I should probably make a wiki page or something for the meeting with background information 15:44:56 maxamillion: that would be good 15:45:10 * pbrobinson didn't get a chance to look @ whenisgood but i'm fairly flexible 15:45:26 #info metting on wednesday to discuss 15:45:34 pbrobinson: https://apps.fedoraproject.org/calendar/infrastructure/2015/5/13/?tzone=UTC#m2538 15:45:42 #action maxamillion to setup a wiki page with background 15:46:15 I think it is best for us to use what is agreed upon, I will chime in at the meeting 15:46:26 maxamillion: looks OK 15:46:44 #topic #6159 Request to discuss Git workflow for rel-eng tooling 15:46:50 https://fedorahosted.org/rel-eng/ticket/6159 15:46:59 #info going to move to pagure 15:47:19 #info nirik was setting up a production instance last night 15:47:30 nirik: how far did you get? 15:47:40 no where. :) 15:47:48 okay 15:47:53 I got confused on how exactly we wanted to call it... 15:47:59 pingou: ? 15:47:59 pagure01? 15:48:01 and then got caught in mm2 doom 15:48:18 pagure01.fedoraproject.org ? 15:48:28 and it can answer for pagure.org and pagure.io ? 15:48:30 I started reinstalling the stg instance on el7 today, but we managed to lock ourself out :) 15:48:31 dns wise it will be available at pagure.org and pagure.io right? 15:48:38 nirik: yes I think so 15:48:47 ok, I can set it up. 15:48:56 dgilmore: we have the hand at the dns level for both domains so we can adjust as needed :) 15:49:05 #action nirik to setup prod pagure 15:49:09 +1 15:49:11 we have a cert for pagure.io... not sure about the others. 15:49:15 nirik: still at osuol02 right? 15:49:15 but we can set it up 15:49:15 prod pagure ++ 15:49:20 does anyone have a strong preference for either domain? 15:49:37 people seems to like the .org better 15:49:39 pagure.io is one character shorter 15:49:46 but there is that :) 15:49:50 and .io are trendy :) 15:49:52 * nirik doesn't care, can adapt to whatever. 15:49:57 * maxamillion has no preference 15:50:18 * pingou doesn't really care either 15:50:33 * pbrobinson things what evah!! 15:50:40 * jkurik likes pagure.org 15:50:40 s/thing/thinks 15:50:42 I perosnaly think .org is a little more respectable 15:50:51 but I do not have a strong opinion 15:50:59 but it's not as hipster! 15:51:07 pagure.ninja is clearly the way to go 15:51:27 ha 15:51:28 nirik: how annoying/expensive is it to get new certs? 15:51:42 pingou: what do we have certs for? 15:51:45 .io 15:51:48 somewhat... we have to do a dance to get authorized to order certs for that domain, then order one. 15:51:50 then lets use that 15:52:06 * masta kinda likes .io 15:52:08 sounds fine 15:52:16 problem solved :) 15:52:22 singe there is a pagure.io cert lets just use that 15:52:40 #info domain will be pagure.io due to existing ssl cert 15:53:08 #action dgilmore to get everything setup and moved over once it is in prod and announce 15:53:18 #topic #6164 bodhi2 status update requested 15:53:24 https://fedorahosted.org/rel-eng/ticket/6164 15:53:34 lmacken: ping, what happened in bodhi2 this week? 15:53:58 bodhi2 was going to head to staging last week, but bodhi1 in production exploded 15:54:08 :( okay 15:54:10 so mash was upgraded on releng04/relepel01 after not having been for 2 years 15:54:17 which caused all sorts of issues to arise 15:54:30 along with the issues from eariler in the week before that 15:54:30 fun 15:54:30 so, it would be great to start running mash in staging 15:54:37 +1000 15:54:48 sounds nice 15:54:57 I am planning on a doing some mash dev work in the next few weeks 15:55:00 so I deployed a new version of bodhi1 yesterday with a workaround 15:55:05 there is some things not working right 15:55:09 and plan on doing work on bodhi2 in staging this week 15:55:17 and there is areas where we should be performing better 15:55:33 #info lmacken to get bodhi2 in staging this week 15:55:34 yeah, soon mash will no longer be the bottleneck in the push process, which is awesome. 15:55:48 I nuked the old/broken/notused releng01.stg in favor of a new ansible made bodhi-backend01.stg 15:55:57 nirik: cool, thanks 15:56:15 if we think 2 is going to be out before too long we could just repurpose it for bodhi2. 15:56:28 since out stg setup for bodhi1 is... not functional 15:56:29 sounds good 15:57:00 ideally we get bodhi2 out soon after f22 is out, we need it to deliver all the things atomic wants 15:57:14 that would be just fine with me. ;) 15:57:24 hopefully it is flexible enough that we can easily extend it for new and different outputs 15:57:27 I'm not sure if bodhi should be The Thing that delivers the atomic stuff, but it can definitely trigger it 15:58:07 ideally our task-running artifact manager system will handle the heavy lifting. "koji 2.0" or whatever. 15:58:09 lmacken: as long as we have reliable integrated pushing processes, it is less important what does what 15:58:12 would need to verify it's sane too to some degree? Not sure you can just fire and hope it all works out 15:58:18 lmacken: it will never end, feature creep. 15:58:23 masta: srsly 15:58:31 the thing that is important is that when updates go out, the updates for everything goes out 15:59:12 Just wait for rkt/app-c 15:59:14 when we get the next heartbleed, we are able to update all the bits at once 15:59:24 pbrobinson: right 15:59:40 likely we will need to make docker base images as part of the updates process 15:59:51 and tehn updated docker layered images when tehy come along 16:00:03 as well as different container formats etc 16:00:30 I only see the list of deliverables for updates rapidly increasing 16:00:32 and I want a pony! 16:01:19 pbrobinson: why stop there, we have unicorns to get 16:02:15 what we need to get from a updates push, tomorrow will be different and more than the package repos of yesterday, and the package repos and atomic repo of today 16:02:19 unicorn dragon kittens for everyone! 16:02:36 anyway enough on that. I look forward to seeing how bodhi2 comes along in staging this week 16:02:46 #topic Secondary Architectures updates 16:02:47 #topic Secondary Architectures update - ppc 16:02:52 nirik: +! 16:02:54 +1 16:03:04 pbrobinson: how was ppc this week? 16:03:14 how is the migration to ansible going? 16:03:38 * nirik saw a power8 builder in ansible. ;) 16:04:56 we're getting there 16:05:10 I spent a bunch of time debugging oz/imagefactory and go 16:05:12 co 16:05:17 about to get back to that now 16:05:54 cool 16:06:04 pbrobinson: how did the TC1, you tried earlier today, go? 16:06:05 there's an issue I discovered with some optimised/extra packages which I need to quiz dgilmore over later 16:06:06 * nirik still hasn't had time to make s390 hub/db. Too many fires to put out and not enough hours in the day. 16:06:14 so for f23 we can/will have different images for ppc 16:06:28 sharkcz: it's mashed, just need to look at the compose side 16:06:29 pbrobinson: happy to help 16:06:38 pbrobinson: can go over it now if its quick. 16:06:44 dgilmore: better later 16:06:48 if not we should discuss in #fedora-releng later 16:06:50 lets get the meeting done 16:07:00 #topic Secondary Architectures update - s390 16:07:11 sharkcz: ping, how is s390 this week 16:07:34 nirik: have we started on teh new hub/db boxes for s390 yet? 16:07:44 * nirik still hasn't had time to make s390 hub/db. Too many fires to put out and not enough hours in the day. 16:07:45 no major news here, I'm going to start composing TC1 in the morning, didn't get to it today 16:07:51 okay 16:07:55 #topic Secondary Architectures update - arm 16:07:59 hopefully with freeze I will get some more time to. 16:08:00 updated lorax is in repos, so this time with productized images 16:08:05 pbrobinson: aarch64 this week 16:08:12 sharkcz: cool 16:08:21 similar to ppc I'm in process of doing TC1 16:08:52 cool. Final Change freeze takes effect at 00:00 UTC tomorrow 16:08:57 so in about 8 hours 16:09:31 I expect at least one more TC before we get a RC request 16:09:39 oh, so not midnight tues/wed like I thought.... oops 16:10:03 pbrobinson: no, as of Tuesday things a frozen 16:10:45 #topic Open Floor 16:10:51 I have one thing for open floor 16:10:52 I had one quick item: 16:10:58 nirik: go ahead 16:11:02 I have something also when nirik is done 16:11:50 I updated all the primary builders. Applied updates and rebooted them. There was a firewall issue on buildhw, but I corrected it now. There was a mergerepo_c issue that has a bug filed, so we downgraded koji back a bit. Otherwise let me know if you see any problems. 16:12:08 cool 16:13:09 nirik: what was the mergerepo_c issue? 16:13:32 I thought mergerepo_c was supposed to fix things 16:13:50 https://bugzilla.redhat.com/show_bug.cgi?id=1220082 16:13:56 maxamillion: the issue was that the koji internal repo was pointing at non existant locations 16:13:59 it wasn't setting the prefix right and broke epel builds 16:14:26 maxamillion: go with your issue, I will do mine last 16:14:55 maxamillion: like all things... fix 1 thing, break 6 others ;-) 16:17:14 maxamillion: what is your item? 16:17:20 just a sec, typing 16:17:52 squeezing mine in, as it's just an fyi kind of thing: I have some patches to rpkg that I posted to the list last week, which would be nice to review/merge (among other things, they would help migrate away from md5 after f22 is released) 16:17:56 I just wanted to point out that I'm going to be working on some "Fedora" Atomic stuff (in quotes because it might be a Remix in the man time) to work with upstream to try and sort out how we're going to handle the Atomic bits better in the future ... for now Project Atomic is kind of focusing on CentOS more so than Fedora because it's the path of least resistance, I wanted to try and open a c 16:18:02 onversation around how I can work on integrating some of the end result of this work when we eventually come to something that's workable for everyone involved 16:18:30 it's highly possible this is a bigger conversation point than for the open floor but I wanted to bring it up 16:19:00 maxamillion: we can fit them in if they come and talk to us and work with us. to date they ahve done a really poor job of that 16:19:00 brainstorming and proof of concept work is happening here https://github.com/CentOS/sig-atomic-buildscripts .... and I'm going to do my best to participate there and make sure that Fedora concerns are taken into account 16:19:41 maxamillion: when they come last minute and say we need all this stuff, it makes it impossible toa ccomodate them 16:19:45 to accomodate 16:20:08 I know they want to have a spin of fedora atomic every two weeks based on the latest stable fedora bits 16:20:15 dgilmore: I guess I don't know what you mean by "talk to us and work with us" ... what's the formal process for that? I've had irc conversations with them in #fedora-releng a number of times 16:20:19 we need to figure out how to do that automatically 16:20:36 maxamillion: us is releng 16:21:16 maxamillion: they have never come and tried to work with releng, they have come and made demands. and the times they have done so it is too late and difficult in teh release cycle to make changes 16:21:32 dgilmore: yes, I understand the concept of who the greater "us" is ... I'm asking what's the process to formally request some sort of co-op? is there a document somewhere on how to handle this properly? 16:22:00 maxamillion: as features are being developed that need new deliverables, atomic needs to talk to releng so that we can develop plans to integrate the tooling 16:22:25 maxamillion: the process is to talk on teh mailing list. 16:22:35 maxamillion: keeping dialog open 16:22:51 as there is concrete thinsg for releng to do file tickets 16:23:22 dgilmore: maxamillion: I think it's a great idea and we can discuss the implications at the FAD which is in a couple of weeks 16:23:31 dgilmore: hasn't this been done? also, I wrote a patch for the lorax wrapper weeks ago to help enable some of that and it's still unreviewed 16:23:46 ultimately the Atomic team have been terrible at communication 16:23:49 maxamillion: to date communications have been very poor. that needs to change to enable everyone to be able to be productive and get the new things done 16:23:56 pbrobinson: I don't want the FAD to be a giant brain storm session, I was hoping to do actual work there ... but alright 16:24:41 maxamillion: it is too late for F22 tooling changes. I have not reviewed because f22 tasks are taking priority over rawhide things 16:24:44 maxamillion: well if you have more solid stuff by then it's not a brainstorm session but a high bandwidth discussion of the proposal and implementation 16:24:50 which is where that needs to go 16:25:06 maxamillion: the FAD will not be that 16:25:30 The FAD will be many things 16:25:31 I agree with dgilmore in that the F-22 boat has pretty much sailed, we/they should be looking @ F-23 now 16:25:36 that leads into my item. I sent a quick note with FAD deliverables to the list last night https://lists.fedoraproject.org/pipermail/rel-eng/2015-May/020075.html 16:25:54 there is a few things I know i missed 16:26:13 dgilmore: koji 2.0? 16:26:23 and I can expand on the items with more detail if they are not clear to all 16:26:42 maxamillion: its not going to happen for the fad or f23 16:27:07 maxamillion: I am trying to get people to talk about the needed features and set a road map 16:27:11 dgilmore: it's going to have to, I've been told I need to be working on it within the next 6-ish weeks 16:27:21 so far I am getting less traction than I would like 16:27:24 dgilmore: nice list. 16:27:39 maxamillion: we have to make thinsg work without koji 2 16:27:56 dgilmore: I'm getting traction, I had a meeting with mikem last week and we're going to be discussing it again this Thurs 16:28:01 I think it should be on the list. mikem was going to get a public discussion/wiki happening since devconf and he hasn't 16:28:02 maxamillion: we need it and it has to be part of the longer term goals 16:28:05 dgilmore: I really want to get pungi4 going soon. the #1 item on that list. 16:28:06 dgilmore: you have the meeting invite for Thurs 16:28:08 its not going to magically happen 16:28:15 maxamillion: ? 16:28:27 maxamillion: what meeting? 16:28:30 maxamillion: it would be nice to get some of that discussion happening on the public list 16:28:45 or minutes at least :) 16:28:51 dgilmore: we've talked about it a couple times ... Amanda C. set it up 16:28:55 maxamillion: the discussion for koji 2 has to happen on the buildsys list or a new dedicated koji devel list 16:29:01 maxamillion: thats not koji 2 16:29:05 dgilmore: you should have it to both your @redhat.com and personal account 16:29:10 maxamillion: that is only layered docker images 16:29:14 dgilmore: part of that meeting will be to discuss koji 2 16:29:27 no, it's not ... that meeting has morphed into a lot of things 16:29:28 maxamillion: not that I know of 16:30:01 we're way off in the weeds and it doesn't matter, my point is that I would like to see koji 2 on the roadmap for the FAD in at least *some* capacity 16:30:32 maxamillion: Its not happening, you can add it but we are no where near close enough to start on code for it 16:31:07 maxamillion: if you want the fad to be all talk and no action then koji2 will take up the whole thing 16:31:09 +1 to have it on the FAD agenda 16:31:42 dgilmore: mikem is going to publish docs on it this week so we can get a lot of discussion in between now and the FAD on the mailing list 16:32:04 maxamillion: whatever 16:32:10 dgilmore: what? 16:32:17 I think being able to utilize our build farm for composing all of the updates deliverables is essential. Whether that's koji 2.0 or distill + koji1 + runroot plugin 16:32:37 maxamillion: sure. Right now the koji2 stuff is still in planning stage, but the koji folks will be there. It will happen =) 16:33:01 masta: dgilmore just said it won't, I'm trying to understand 16:33:02 lmacken: that why we're aiming distill for F-23 with it's functionality 16:33:34 pbrobinson: good, so I'll probably be working on plugins for that during the fad. 16:33:45 maxamillion: IMO the koji2 discussion needs to be more public, not 1-2-1 calls 16:33:53 maxamillion: mikem will be at the FAD, so... it can be formal or informal, but it should not eat away of valuable time spent on FAD things. 16:33:56 pbrobinson: I'm trying *very* hard to make that happen 16:34:24 maxamillion: we have a simple way we have to opperate and if any of what you are talking about with koji 2 is true, 1) it is new to me and in violation of how things are supposed to be done. 16:34:28 pbrobinson: every time I have a call with different teams internally I press the issue of getting it all out in the open so we can have an open discussion with the larger Fedora community 16:34:38 maxamillion: I am going on what I know and has been discussed in a public forum 16:35:06 maxamillion: the internal people need to stop having internal calls and do things in the public in an open forum 16:35:14 without that we will all fail 16:35:28 yeah 16:35:29 I'm trying to pull things out in the open, we can't fix the problems of things of internal things being internal by ignoring them 16:35:49 like, literally ... I'm nagging that issue to the point I'm sure people are sick of me 16:35:51 I am not saying we can\ 16:36:01 I am saying they need to be told do things right 16:36:22 maxamillion: nag the good nag, nag conscientiously. 16:36:27 masta: :) 16:38:00 maxamillion: in the end we need to take steps to being more open and transparent. the list of deliverables is based on what I think we can do, given the knowledge I have of teh state of things 16:38:22 dgilmore: fair 16:38:23 so to round back, as someone who is not yet intimiately familiar with all things Fedora Rel-Eng, what exactly do I need to be aware of and take into consideration so I can advise the Atomic team on how to properly request something so that it's not just considered a demand? 16:38:28 if my knowledge is wrong we need to get the people who have other plans to open up about tehm in the appropriate places 16:38:36 dgilmore: I completely agree 16:39:00 maxamillion: https://fedoraproject.org/wiki/ReleaseEngineering/Philosophy 16:39:46 there needs to be communicatiion in teh planning of new deliverables so that we can advise how things should be integrated 16:39:55 dgilmore: I've read that, I like the doc 16:40:01 dgilmore: alright, thanks 16:40:35 * nirik thinks koji2 may be a ton of work, I don't think it will be done in a fad or quickly. 16:40:35 maxamillion: it is a requirement that for any new deliverable before a Change is filed releng is approached about how that deliverable will be made and delivered 16:41:01 dgilmore: alright 16:41:41 nirik: I don't think it will be done, but I would like it to at least get moving ... I've heard rumors about koji 2.0 for *years* and I just want to make sure it finally makes its way into reality 16:42:24 maxamillion: the most concrete discussions on koji 2.0 was at fosdem and devconf this year 16:42:47 before that it was only when we do koji 2.0 we should do foo 16:43:03 so I would be surprissed if you have hearing chatter about it for years 16:43:05 dgilmore: on fad task #6, in your opinion does that depend on pungi4/distill working, or can it depend on task #8 runroot working, or could it be made into a pure koji task instead? 16:43:41 maxamillion: are you sure that's not bodhi 2.0 who you heard ? 16:43:48 masta: I was pplanning to do taht in koji itself 16:43:52 * masta tries to clarify what things may entail before cherry picking fad tasks 16:44:00 dgilmore: ok 16:44:08 dgilmore: I heard it mentioned the first time a FUDCon Lawrence 16:44:10 maxamillion: bodhi 2.0 has been in development for years now 16:44:15 dgilmore: yes, I know 16:44:25 maxamillion: that would be news to me 16:45:00 anyways ... none of that matters 16:45:38 what does matter is we need to get a concerete list of things that we want to achieve at the fad 16:45:58 the things I listed are all things I think we can do at the FAD 16:46:11 dgilmore: yes 16:46:28 there is probbaly things that need to be added 16:47:27 so if there is things people want added. contribute to that thread 16:47:34 right, I was just trying to suggest koji2 be something on the list ... what the scope of that means can be worked out into an unit of work that is consumable within the timeframe of the FAD, I wasn't suggesting we rebuild Rome in a weekend 16:47:37 having pagure before the FAD will be excellent. 16:47:53 if there is things I listed people do not think should be, contribute to that thread 16:48:12 maxamillion: we can add it 16:48:37 I will speak to mikem today and try sort out what is going on and get things pulled onto buildsys list 16:48:40 masta: +1 16:49:20 dgilmore: he said he'd post something to env-and-stacks this week 16:50:12 maxamillion: that would be the wrong place I think 16:50:23 buildsys would be the right place for koji 16:50:28 dgilmore: rgr 16:50:40 but maybe there is something I am missing 16:50:47 because all of this is news to me 16:51:05 dgilmore: I thought maybe he knew something I didn't so I was going to defer to him on where to post about it 16:52:36 does anyone have anything else for open floor? 16:55:55 I will take that as no 16:55:59 #endmeeting