17:02:15 <mboddu> #startmeeting RELENG (2018-09-13)
17:02:15 <zodbot> Meeting started Thu Sep 13 17:02:15 2018 UTC.
17:02:15 <zodbot> This meeting is logged and archived in a public location.
17:02:15 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:15 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:02:15 <zodbot> The meeting name has been set to 'releng_(2018-09-13)'
17:02:16 <mboddu> #meetingname releng
17:02:16 <mboddu> #chair nirik tyll sharkcz masta pbrobinson pingou puiterwijk maxamillion mboddu Kellin dustymabe
17:02:16 <mboddu> #topic init process
17:02:16 <zodbot> The meeting name has been set to 'releng'
17:02:16 <zodbot> Current chairs: Kellin dustymabe masta maxamillion mboddu nirik pbrobinson pingou puiterwijk sharkcz tyll
17:02:40 <nirik> morning
17:04:55 <mboddu> nirik: morning, I hope people will show up for this meeting
17:05:12 <nirik> well, go/no-go might be more exciting. :)
17:05:49 <mboddu> Yeah, I agree
17:08:23 <mboddu> nirik: Okay, what do you want to do?
17:08:32 <mboddu> Should we skip it for this week?
17:08:49 <nirik> well, what all do you have on the agenda?
17:08:50 <mboddu> Probably we will have to face the same scenario again next week
17:08:53 <smooge> will next week be the same?
17:09:07 <mboddu> smooge: I fear it would be same
17:09:12 <mboddu> nirik: I have couple of things
17:09:37 <mboddu> nirik: https://pagure.io/releng/issue/7802 and https://pagure.io/releng/issue/7740
17:09:46 <smooge> what I end up doing when I have something like this is go over the agenda items, close the meeting and get people to either come to a different time or answer in meail
17:10:32 <smooge> but I am a jealous chair about meetings...
17:11:01 <nirik> well, 7802 I am not sure what to do. It really needs more investigation... some of those listed are valid
17:11:20 <mboddu> smooge: Okay, good suggestion, but https://pagure.io/releng/issue/7802 I dont know where the issue is and https://pagure.io/releng/issue/7740 is something we should discuss and come up with a plan (its not easy)
17:11:30 <mboddu> nirik: Yeah, I agree
17:11:56 <nirik> we don't want to remove things that are being used/valid
17:12:23 <nirik> on 7740, we aren't supposed to untag things from rawhide once they have gone out, but we could ask fesco for an exception I suppose.
17:12:37 <nirik> were those built against f28 ?
17:13:12 <mboddu> nirik: Thats a weird module build - https://src.fedoraproject.org/modules/stratis/blob/master/f/stratis.yaml#_8-16
17:13:32 <mboddu> nirik: It goes back to tagging builds based on build reqs rather than run time reqs
17:13:40 <nirik> yeah, that seems... wrong
17:13:55 <nirik> I'd like to get puiterwijk's thoughts on this since he dealt with it
17:13:56 <nirik> before
17:14:11 <mboddu> Also, even if fesco gives us an exception, how are we going to remove the module build from mirrors?
17:14:41 <nirik> well, when we untag it it should disappear from the next pungi run/compose
17:14:45 <nirik> I would think
17:14:54 * puiterwijk reads back
17:16:45 <puiterwijk> Right... So, all of this is.. non-ideal.
17:16:48 <mboddu> nirik: I thought they wouldn't never get removed from mirrors, hence we never untag them, but I might be wrong
17:16:59 <nirik> so they build against 30 but ship on older? that seems odd
17:17:26 <nirik> mboddu: it only works for rawhide/branched before we turn off the nightly. stable release don't change after GA
17:17:28 <puiterwijk> nirik: yep. And the reason is becuase they don't want to go through the process of building all the deps for f27/f28
17:18:04 <puiterwijk> And they also didn't want to add the deps to the module
17:18:12 <mboddu> nirik: Okay, I didn't know that and yes after GA we dont push anything to that repo, so it would never get changed
17:18:39 <puiterwijk> So, yes, we can just untag from rawhide and tag it into the tags they asked. It'll work with bodhi, same for future cases.
17:18:40 <nirik> so does that even actually work?
17:18:48 <puiterwijk> nirik: it builds....
17:18:57 <puiterwijk> And it's supposed to run because it's all statically compiled.
17:19:06 <nirik> well, yeah, but at runtime what if there's a f30 dynamic lib linked to?
17:19:15 <puiterwijk> It's all static libs
17:19:16 <nirik> is that pulled into the module? or just.... missing
17:19:21 <nirik> ok
17:19:30 <puiterwijk> With dynamic libs, this would not work
17:19:37 <puiterwijk> This specific module is Rust, so build statically
17:19:40 <nirik> yeah, I wouldn't think so.
17:19:58 <puiterwijk> mboddu: so, for *just* stratis, we can do this for now.
17:20:21 <mboddu> puiterwijk: Okay, and do we need FESCo approval?
17:20:47 * mboddu thinks we should, just for keeping them in loop on whats going on and what to expect in future
17:21:04 <nirik> in theory for the untag yeah... I can ask.
17:21:13 <patricku> mboddu: for the rawhide  untagging now, yes
17:21:33 <patricku> mboddu: for the rawhide  untagging now, yes
17:21:33 <nirik> so how did it tag into f30? because the platform f30?
17:21:42 <mboddu> nirik, patricku : thanks for your input
17:21:43 <mboddu> nirik: Yes
17:21:57 * dustymabe has an item for open floor later,, please ping me during that time
17:22:02 <patricku> mboddu: also!!!! Don’t tag into f28 etc. but into their respective candidate tags!
17:22:18 <nirik> yeah, signing is important
17:22:21 <mboddu> patricku: Sure, they have to go through the update process, I understand
17:22:33 <nirik> and updates flow yeah
17:22:34 <mboddu> And signing too
17:22:39 <mboddu> nirik: Haha :)
17:24:16 <mboddu> #info nirik will check with FESCo to make an exception in this case so that we can untag it from rawhide and tag them into their respective releases' candidate tag
17:24:38 <mboddu> Those are the two things I had
17:25:33 <mboddu> #undo
17:25:33 <zodbot> Removing item from minutes: INFO by mboddu at 17:24:16 : nirik will check with FESCo to make an exception in this case so that we can untag it from rawhide and tag them into their respective releases' candidate tag
17:25:55 <mboddu> #topic #7740 stratis module updates for F28/F29
17:26:02 <mboddu> #link https://pagure.io/releng/issue/7740
17:26:09 <mboddu> #info nirik will check with FESCo to make an exception in this case so that we can untag it from rawhide and tag them into their respective releases' candidate tag
17:26:14 <mboddu> #info nirik will check with FESCo to make an exception in this case so that we can untag it from rawhide and tag them into their respective releases' candidate tag
17:26:25 <mboddu> nirik: Anything else you want to add?
17:27:08 <nirik> nope...
17:27:21 <mboddu> Okay, open floor it is
17:27:26 <mboddu> #topic Open Floor
17:27:41 * dustymabe waves
17:27:45 <mboddu> dustymabe: your turn
17:27:56 * mboddu waves back at dustymabe
17:28:22 <dustymabe> would like to draw some attention to https://github.com/coreos/fedora-coreos-tracker/issues/23#issuecomment-420760366
17:28:59 <dustymabe> TL;DR we are trying to evaluate how we are going to deliver ostree updates to end users of Fedora CoreOS and we think we have settled on OSTree repo (like we already have today)
17:29:07 <nirik> I have been reading it, but have had no time to compose a reply yet
17:29:22 <dustymabe> so we can start moving forward with actually doing mirroring right in fedora releng/infra
17:29:34 <dustymabe> s/mirroring/ostree mirroring/
17:30:06 <dustymabe> my thoughts on next steps are for us to get together with patrick and work out any issues and start to implement it properly
17:30:21 <nirik> whats the change from what we have now?
17:30:37 <dustymabe> there is an issue somehwere where we had already agreed on "next steps" for OSTree mirroring a while back
17:30:43 <dustymabe> i'll need to resurrect that issue
17:30:48 <mboddu> nirik: We have to support ostree mirroring to start with...
17:30:57 <dustymabe> nirik: yep, ostree mirroring
17:31:12 <dustymabe> right now we just have CDN, which is like caching and we still get complaints
17:31:43 <dustymabe> in my mind the 'mirroring work' has been "on hold" for a long time because we were possibly going to use rojig
17:31:45 <nirik> well, thats just a matter of putting them back on the master mirrors?
17:31:46 <nirik> or ?
17:31:59 <dustymabe> nirik: one sec, let me see if I can find the issue
17:32:19 <nirik> sure.
17:32:27 <dustymabe> https://pagure.io/fedora-infrastructure/issue/5970
17:32:32 <nirik> if it's that infra ticket... yeah.
17:33:05 <dustymabe> https://pagure.io/fedora-infrastructure/issue/5970#comment-435694
17:33:09 <nirik> ok, so we need metalink support, does that exist? then we need mm to make ostree metalinks and then we can readd to mirrors ? right?
17:33:31 <dustymabe> nirik: i think it's supposed to exist, but i have not tested it\
17:34:01 <nirik> yeah, we should probibly gather status up on it in ticket or something.
17:34:12 <dustymabe> anyways.. this was mostly an FYI - not asking anyone for anything right now
17:34:17 <nirik> I need to check on mm ostree stuff. I thought that also got added, but not sure.
17:34:27 <dustymabe> patricku: any thoughts ?
17:34:37 <nirik> is there a timeframe for this? before f29? before f30 beta?
17:34:54 <dustymabe> FCOS target is f30
17:35:14 <dustymabe> but this should help iot and silverblue if we get it implemented sooner
17:35:34 <mboddu> I dont think we can get it done any sooner than F30
17:35:55 <mboddu> As, F29 will go GA in about month and the freeze starts in couple of weeks
17:35:58 <dustymabe> right yeah, not expecting anything before f30
17:36:19 <dustymabe> i'm mostly trying to get out way ahead of this issue,
17:36:22 <patricku> I don’t think it’s that complicated once we have the items from said ticket. Rpm ostree has metalink support but mirrormanager doesn’t have support for ostree yet.
17:36:45 <dustymabe> patricku: +1 so we can start to look at that
17:36:53 <dustymabe> patricku: i have one other question for you and the rest
17:37:07 <nirik> also possibly there's the cert pinning thing walters really wants... unless we can convince him to use cert transparency. but perhaps thats all a different topic
17:37:28 <dustymabe> so there are some of the spins that don't do large releases (i.e. silverblue, fedora iot (right now))
17:37:35 <patricku> nirik: different topic. But that needs work on the rpmostree end.
17:37:54 <dustymabe> right now the proposal for mirroring just puts static deltas (from latest atomic host release) on the mirrors
17:38:07 <dustymabe> but for silverblue/iot there aren't any static deltas
17:38:32 <dustymabe> could we just put the last release of silverblue/iot small files on the mirrors (i.e. prune older ones)
17:38:41 <dustymabe> or would the mirrors not be happy about the small files at all ?
17:38:43 <patricku> Correct. Maybe we just add support to pungi to create a delta per compose?
17:39:13 <dustymabe> patricku: yeah that's one option, but would it be a delta from scratch or a delta from N-1 ?
17:39:14 <patricku> I don’t think mirrors will be happy with thousands of small files.. but a static delta per compose is just one a day
17:39:36 <patricku> Dunno.
17:39:39 <dustymabe> if people don't update every day then they would never actually use the static delta
17:39:39 <nirik> that seems pretty possibly... N-1 ?
17:39:44 <nirik> oh true
17:40:02 <dustymabe> anywho we can probably discuss this somewhere else (i don't want to take over the meeting)
17:40:13 <patricku> dustymabe: I don’t known the best idea right now and I’m not at home for now. Maybe open a ticket and we can discuss there?
17:40:21 <mboddu> dustymabe: I guess from scratch is a better option, since everyone wants to update to the latest where ever they are...
17:40:29 <dustymabe> patricku: +1000 thanks so much!
17:40:30 <patricku> Or a discussion on irc later?
17:40:40 <dustymabe> I'll open a ticket
17:40:43 <nirik> mboddu: yeah, could be...
17:41:51 <nirik> Oh, so I had one observation/thing to mention...
17:42:17 <nirik> when going to merge pr's in the various repos we have... pungi-fedora, releng, kickstarts, comps, etc...
17:42:52 <nirik> we have a number of really really old PR's. Can we close those and ask people to refile? or something? it really makes it hard to tell whats still needing merging.
17:42:57 <mboddu> #info We are planning to support ostree mirroring which needs some work on mirror manager and this helps fedora silverblue and fedora iot a lot
17:43:25 <mboddu> nirik: I would say close them and ask them to reopen it if necessary
17:43:55 <dustymabe> nirik: i'd say if they are older than 6 months then close them and ask people to re-open if still an issue or still a PR they want to pursue
17:44:07 <nirik> https://pagure.io/fedora-kickstarts/pull-requests
17:44:15 <nirik> there's 2 older than 6 months.
17:44:41 <nirik> one here: https://pagure.io/fedora-comps/pull-requests (but perhaps we should just frigging merge that one)
17:45:01 <nirik> tons here: https://pagure.io/pungi-fedora/pull-requests
17:45:48 <nirik> and the worst one: https://pagure.io/releng/pull-requests
17:46:00 <nirik> 9 of them over 6 months there
17:46:55 <nirik> anyhow, I would love to clean them out... I can try and close or merge them or someone else can?
17:50:27 <mboddu> nirik: I will look at pungi-fedora and releng and you can take the remaining, does that sound okay?
17:51:06 <nirik> sure, sounds fine. ;)
17:51:18 <nirik> ask people to rebase / wait a week, close.
17:51:44 <mboddu> nirik: Yup
17:52:38 <dustymabe> nirik: i wonder if "freeze" is the right time to do this ?
17:52:48 <dustymabe> i.e. people will act and then we'll all have to wait again
17:53:00 <nirik> dustymabe: true, we could wait until after beta is signed off on...
17:53:13 <nirik> although giving them until then to rebase might be good?
17:53:59 <mboddu> #info We will going through all the PR's on multiple repo's and we will ask the reporters to rebase them or we will merge them. If the reporters wont get back to us in a week or after beta freeze ends, we will close all the PR's that are more than 6 months old.
17:56:26 <mboddu> Anything else guys?
17:56:40 <nirik> not from me
17:56:54 <mboddu> Okay, thanks guys for joining
17:57:18 <mboddu> I thought we have to skip for today, but seems like we covered for the entire hour-ish
17:57:27 <mboddu> #endmeeting