16:30:24 <jlebon> #startmeeting fedora_coreos_meeting
16:30:24 <zodbot> Meeting started Wed Apr 14 16:30:24 2021 UTC.
16:30:24 <zodbot> This meeting is logged and archived in a public location.
16:30:24 <zodbot> The chair is jlebon. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:24 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:24 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:28 <lucab> .hello2
16:30:32 <zodbot> lucab: lucab 'Luca Bruno' <lucab@redhat.com>
16:30:33 <jlebon> #topic roll call
16:30:34 <dustymabe> .hello2
16:30:35 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:30:37 <slowrie> .hello2
16:30:38 <zodbot> slowrie: slowrie 'Stephen Lowrie' <slowrie@redhat.com>
16:30:40 <jbrooks> .hello jasonbrooks
16:30:41 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com>
16:31:20 <jlebon> #chair lucab dustymabe slowrie jbrooks
16:31:20 <zodbot> Current chairs: dustymabe jbrooks jlebon lucab slowrie
16:31:34 <jaimelm> theoretically, my cloak was added this morning.
16:31:38 <jaimelm> .hello2
16:31:39 <zodbot> jaimelm: jaimelm 'Jaime Magiera' <jaimelm@umich.edu>
16:31:43 <jaimelm> excellent
16:31:52 <jlebon> #chair jaimelm
16:31:52 <zodbot> Current chairs: dustymabe jaimelm jbrooks jlebon lucab slowrie
16:32:30 <cyberpear> .hi
16:32:32 <zodbot> cyberpear: cyberpear 'James Cassell' <fedoraproject@cyberpear.com>
16:32:39 <bgilbert> .hello2
16:32:40 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:32:55 <jlebon> #chair cyberpear bgilbert
16:32:55 <zodbot> Current chairs: bgilbert cyberpear dustymabe jaimelm jbrooks jlebon lucab slowrie
16:32:59 <fifofonix> .hello2
16:33:00 <zodbot> fifofonix: fifofonix 'Fifo Phonics' <fifofonix@gmail.com>
16:33:11 <jlebon> #chair fifofonix
16:33:11 <zodbot> Current chairs: bgilbert cyberpear dustymabe fifofonix jaimelm jbrooks jlebon lucab slowrie
16:33:25 <jlebon> lots of people today, nice :)
16:33:32 <copperi_> .hello copperi
16:33:34 <zodbot> copperi_: copperi 'Jan Kuparinen' <copper_fin@hotmail.com>
16:33:45 <jlebon> #chair copperi_
16:33:45 <zodbot> Current chairs: bgilbert copperi_ cyberpear dustymabe fifofonix jaimelm jbrooks jlebon lucab slowrie
16:33:55 <jlebon> ok, let's begin!
16:34:05 <jlebon> #topic Action items from last meeting
16:34:14 <jlebon> * travier  to summarize outcome in https://github.com/coreos/fedora-coreos-tracker/issues/768
16:34:17 <jlebon> * jlebon to send communication about cgroupsv2 to coreos-status@ and coreos@ mailing lists.
16:34:20 <jlebon> * jaimelm to work on engaging with community on adding .ign/.bu editor support
16:34:23 <jlebon> * bgilbert to investigate updating the Ignition type registration
16:34:42 <lucab> I think travier is offline today
16:34:44 <jlebon> #info jlebon sent https://lists.fedoraproject.org/archives/list/coreos-status@lists.fedoraproject.org/thread/6NGBXYMJ4YU3V667XN627WOGCJA47POT/
16:34:48 <bgilbert> #action bgilbert to investigate updating the Ignition type registration
16:35:08 <jlebon> lucab: ack, let's reaction that one since it still looks pending
16:35:15 <jlebon> #action travier  to summarize outcome in https://github.com/coreos/fedora-coreos-tracker/issues/768
16:35:49 <jlebon> jaimelm: anything to report for your AI? should we re-action it?
16:35:51 <jaimelm> haven't started community enagement stuff yet. Was out of town most of the past 7 days.
16:35:54 <jaimelm> please reactoin
16:36:08 <dustymabe> I forget, was part of that setting up the test day?
16:36:14 <dustymabe> otherwise I'll create a ticket for that now
16:36:25 <jaimelm> I was going to create a ticket for that this afternoon.
16:36:30 <jlebon> #action jaimelm to work on engaging with community on adding .ign/.bu editor support
16:36:41 <dustymabe> ok
16:36:44 <jlebon> cool
16:36:48 <dustymabe> here's the previous ticket: https://github.com/coreos/fedora-coreos-tracker/issues/659
16:36:52 <dustymabe> from last time
16:36:59 <jaimelm> cool
16:37:06 <jlebon> #action jaimelm to file ticket for test day
16:37:21 <jlebon> anything else before we move to meeting items?
16:38:20 <jlebon> #topic tracker: Fedora 34 rebase work
16:38:24 <jlebon> #link https://github.com/coreos/fedora-coreos-tracker/issues/704
16:38:45 <jlebon> things are progressing in this area
16:39:00 <jlebon> dustymabe is chasing down a bug with latest dracut
16:39:13 <jlebon> and there's also another bug with docker we're still investigating
16:39:25 <jlebon> but overall, things are looking good
16:39:58 <jlebon> timing-wise, ideally we'd like to release f34 on testing in the next release (in two weeks)
16:40:01 <dustymabe> docker swarm bug: https://github.com/coreos/fedora-coreos-tracker/issues/793
16:40:09 <walters> .hello2
16:40:11 <jlebon> f34 GA is next week
16:40:12 <zodbot> walters: walters 'Colin Walters' <walters@redhat.com>
16:40:15 <jlebon> #chair walters
16:40:15 <zodbot> Current chairs: bgilbert copperi_ cyberpear dustymabe fifofonix jaimelm jbrooks jlebon lucab slowrie walters
16:40:26 <dustymabe> thanks fifofonix for helping us find that and working with upstream/us to help get it fixed
16:40:45 <dustymabe> hopefully we'll know the root cause soon
16:40:51 <jlebon> dustymabe had a suggestion to do an async release on GA with GA content to gain more confidence before rebasing testing the week after
16:40:54 <fifofonix> yw.  that docker swarm issue gets worse in mixed swarm situations.   its a blocker but hopefully we can solve soon.
16:41:22 <fifofonix> not comfortable with it going to testing as-is.
16:41:45 <dustymabe> fifofonix: yeah. i'd much prefer to get it fixed before we promote
16:41:50 <dustymabe> let's hope that happens soon
16:41:56 <lucab> jlebon: a `next` async release right after F34 GA, right?
16:42:01 <jlebon> so then, the schedule would look like this:
16:42:01 <lorbus> .hello2
16:42:01 <jlebon> - Apr 20th: async next release with GA content
16:42:01 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:42:02 <jlebon> - Apr 27th: triple release, testing rebase to f34
16:42:02 <jlebon> - May 11th: triple release, stable rebase to f34
16:42:13 <jlebon> #chair lorbus
16:42:13 <zodbot> Current chairs: bgilbert copperi_ cyberpear dustymabe fifofonix jaimelm jbrooks jlebon lorbus lucab slowrie walters
16:42:21 <jlebon> lucab: yeah, exactly
16:43:13 <dustymabe> jlebon: that is assuming the 34 GA happens April 20th week
16:43:17 <jlebon> this is assuming that everything goes well (i.e. that f34 is go in tmw's go/no-go meeting, and that we don't have blocker bugs)
16:43:56 <dustymabe> yep. if it gets pushed back I still say we do an ad-hoc `next` next week
16:44:39 <dustymabe> fifofonix: if we get any new info on that moby-engine/containerd/docker swarm issue will you be around later this week?
16:44:53 <dustymabe> I can give you development builds if we get some new content to try
16:45:18 <jlebon> dustymabe: and the schedule above stays the same otherwise? (i.e. testing rebase on GA)
16:45:22 <fifofonix> yes.  eager to help out.  also want to try out jlebon's abfab notes on cg2.
16:45:40 <dustymabe> fifofonix: +1
16:45:53 <dustymabe> jlebon: yeah I think so.
16:46:36 * dustymabe has one other thing for this topic
16:46:44 <lucab> we can maybe shift the schedule a bit and have the "testing rebase" on May 3rd
16:46:45 <jlebon> i think that sounds good to me, with discretion based on why the f34 GA was moved
16:47:20 <dustymabe> should we just write down both scenarios ?
16:47:59 <jlebon> sure, we can do that
16:48:28 <jlebon> https://hackmd.io/OEtORqisSV20lTxJmtCwJA
16:49:36 <jlebon> lucab: hmm, but then it'd be in testing for just 1 week instead of the usual 2 before going to stable
16:50:19 <lucab> jlebon: I mean, shifting everything afterwards too
16:50:38 <jlebon> ahh gotcha
16:52:25 <lucab> the "async next" then gets (almost) the usual 2w before promotion
16:53:22 <jlebon> lucab: want to fill in 2B?
16:55:12 <dustymabe> lucab: so option 1B is to shift our cadence?
16:56:28 <lucab> dustymabe: yes
16:57:13 <lucab> I guess we want anyway to have a proper 'next' release with F34 GA before rebasing testing
16:57:13 <dustymabe> got ya
16:57:21 <jlebon> so i think in the end it comes down to how much time we want f34 GA to cook in next before propagating
16:58:19 <jlebon> if we're opening to shifting our schedule, we can just say: "next release whenever GA is, then testing rebase N weeks after, and follow on new cadence"
16:58:28 <jlebon> and we can decide if N is 1 or 2
16:58:39 <dustymabe> i think ties into the ticket jdoss opened
16:58:48 <dustymabe> https://github.com/coreos/fedora-coreos-tracker/issues/785
16:59:26 <dustymabe> some people might depend a bit on the current two week cadence we have
16:59:29 <jlebon> yeah, i think we're essentially working on that right now :)
16:59:48 <dustymabe> jlebon: I do think there is a difference between what we're doing for this release and what we want to get to in the future
16:59:53 <fifofonix> Agreed on jdoss's questions with emphasis on security releases.
17:00:35 <jlebon> i'm also a little hesitant to change the cadence, but OTOH we did say it was subject to change
17:00:59 <dustymabe> cadence is the same :) - just shifted to a different week
17:01:01 <jlebon> well, we're not changing the frequency, just the phase :)
17:01:07 <dustymabe> right
17:01:31 <lucab> yes, basically phase-resync with Fedora major GA
17:02:03 <dustymabe> the other thing we could say is:
17:02:11 <cyberpear> are our releases at all lined up w/ Silverblue?
17:02:24 <dustymabe> approaching fedora 34 GA we'll do weekly `next` stream releases
17:02:45 <dustymabe> the first triple release week after GA, we'll move over the `testing` stream
17:02:47 <jlebon> cyberpear: silverblue composes daily, so it's whenever GA is
17:03:02 <dustymabe> and the next triple release, `stable`
17:03:37 <fifofonix> jlebon: why are you hesitant to change cadence?  worried about uncertainty it causes upstream or downstream?
17:03:55 <dustymabe> clarification 'the first triple release week after GA, we'll move over the `testing` stream. If the GA lands on a triple release week, we'll move it the same week.
17:05:05 <jlebon> fifofonix: to clarify, s/cadence/phase/, but: just whatever tooling some downstream users might have that assume the current dates
17:05:20 <jlebon> but i don't think we should worry too much about those
17:05:44 <bgilbert> we already have quite a bit of jitter wrt release day within the release week
17:06:16 <lucab> jlebon: I won't worry too much on the phase, the next dates can be tracked by the audience from the github tickets
17:06:35 <dustymabe> lucab: if I could modify Option 1B slightly
17:06:52 <dustymabe> Apr 27th: async `next` release, bumped content
17:07:07 <jdoss> .hello2
17:07:08 <zodbot> jdoss: jdoss 'Joe Doss' <joe@solidadmin.com>
17:07:17 <jdoss> Sorry was in another meeting.
17:07:25 <dustymabe> np
17:07:33 <jlebon> #chair jdoss
17:07:33 <zodbot> Current chairs: bgilbert copperi_ cyberpear dustymabe fifofonix jaimelm jbrooks jdoss jlebon lorbus lucab slowrie walters
17:08:08 <jdoss> I think what I was trying to write in that ticket was we should have something written down that is flexible enough for the folks doing the release to be able to not be under a crunch.
17:08:36 <dustymabe> yeah, we're actually not talking about that ticket specifically just yet :)
17:08:40 <jdoss> crap
17:08:41 <lucab> dustymabe: i.e. always avoid .0 releases ;)
17:08:41 <fifofonix> jlebon: thanks for clarifying.  agree though that you shouldn't worry too much about that.  i think downstream cares that you're releasing quickly/reliably and you are.
17:08:43 <jdoss> sorry!
17:08:51 <dustymabe> jdoss: no. it overlaps a lot!
17:08:55 <dustymabe> which is why I brought it up
17:09:26 <dustymabe> lucab: kinda, there is a lot of 0 day fixes that land after GA
17:09:58 <dustymabe> ok let's try to tie this off
17:10:10 <dustymabe> for GA on apr 27th we only have one option that is on the table
17:10:15 <dustymabe> so let's go ahead and highlight that:
17:10:22 <dustymabe> Assuming GA is Apr 27th:
17:10:24 <dustymabe> Option 2A:
17:10:26 <dustymabe> Apr 20th: async next release with bumped content
17:10:28 <dustymabe> Apr 27th (GA): triple release, testing rebase to f34
17:10:30 <dustymabe> May 4th: no releases
17:10:32 <dustymabe> May 11th: triple release, stable rebase to f34
17:11:13 <dustymabe> for GA on Apr 20th, we've got two options
17:11:27 <dustymabe> one includes shifting our release weeks around, one doesn't
17:11:47 <dustymabe> so really it comes down to, do we care about keeping the same release weeks?
17:11:59 <dustymabe> I don't think it matters too much
17:12:11 <jlebon> i think i prefer we'd focus on time after GA
17:12:28 <dustymabe> +1 - proposal?
17:12:30 <jlebon> so i lean more towards lucab's idea of just resyncing with the GA date
17:13:12 <jlebon> e.g. whenever GA is, 1 week later is when testing goes to f34, and it defines the next 2-week phase
17:13:33 <dustymabe> ok
17:13:40 <jlebon> +1 to next release every week
17:13:55 <dustymabe> SGTM
17:14:04 <walters> (I still hope we eventually fix or at least strongly influence the Fedora release process itself - a long 6 month slowly freezing plus a torrent of unconstrainted "0 day updates" just doesn't make sense anymore; then we wouldn't be having this discussion specifically for FCOS)
17:15:03 <dustymabe> so Something like this:
17:15:04 <jaimelm> ^^
17:15:12 <dustymabe> Week 0 (GA): next release with bumped content
17:15:13 <dustymabe> Week 1: testing release based on Fedora N
17:15:15 <dustymabe> Week 4: stable release based on last testing
17:15:43 <jlebon> s/4/3/ no?
17:16:05 <dustymabe> ahh, so we're not baking in 2 weeks before stable?
17:16:25 <dustymabe> oh sorry
17:16:27 <dustymabe> :)
17:16:31 <dustymabe> yes, you are right
17:16:38 <jlebon> :)
17:16:56 <jlebon> this SGTM! lucab? anyone else?
17:16:58 <dustymabe> Week 0 (GA): next release with bumped content
17:16:59 <dustymabe> Week 1: testing release based on last next
17:17:01 <dustymabe> Week 3: stable release based on last testing
17:17:31 <jlebon> we can debate further in the ticket maybe if there's still some concerns.  we should move to other tickets
17:17:58 <dustymabe> yes. I have one other concern to bring up on this topic specifically
17:18:18 <jlebon> dustymabe: ok, go for it
17:18:19 <dustymabe> f33 keeps chugging along, f34 enters some sort of "freeze" during this period
17:18:42 <dustymabe> should we do work specifically to make sure no packages "downgrade" as part of any upgrade path?
17:19:06 <lucab> I'm fine with whatever final decision, it looks anyway we embraced the "shift phase to resync" point
17:19:15 <dustymabe> lucab: yes
17:19:43 <jlebon> i think we should try, yeah
17:20:08 <dustymabe> an example to illustrate my point.. if pkg-4.0.2.f33 exists in f33 and in f34 it is pkg-4.0.1.f34
17:20:26 <dustymabe> should we just cherry-pick pkg-4.0.2.f34 that is probably hanging out in bodhi in f34?
17:20:45 <jlebon> i think this is already the case no? don't we enable bodhi repos in composes?
17:21:03 <jlebon> or are you discussing whether we should not do that to match the rest of Fedora?
17:21:46 <dustymabe> right now f34 has no 'updates' repo
17:22:21 <jlebon> isn't it https://github.com/coreos/fedora-coreos-config/blob/947945b4ecd24d5c713845165ed92f49637dcb5c/manifest.yaml#L16 ?
17:22:54 <dustymabe> i guess there are some details here that we probably don't have time to discuss, but in general just trying to guage: "if there is a case where a package would be downgraded, we want to try to avoid that?"
17:23:36 <jlebon> IMO yes
17:23:58 <jlebon> our f34 validation should include the deluge of bodhi updates pending
17:24:01 <dustymabe> ok jlebon let's discuss details of if that is or is not already handled right after the meeting
17:24:28 <jlebon> ack sounds good!
17:24:45 <jlebon> dustymabe: should we discuss the cri-o ticket now or next time?
17:24:58 <dustymabe> probably next time
17:25:01 <dustymabe> I can open floor it
17:25:06 <jlebon> ok sounds good
17:25:14 <jlebon> #topic Open Floor
17:25:52 <jlebon> anyone has anything they want to bring up?
17:26:15 <fifofonix> quickie.  crowdstrike.
17:26:16 <dustymabe> I'm going to try to dig into https://github.com/coreos/fedora-coreos-tracker/issues/767 soon
17:26:29 <dustymabe> if you have anything to add, let me know
17:26:30 <lucab> nothing special to report, this week rollouts are ongoing https://github.com/coreos/fedora-coreos-streams/pull/293
17:26:37 <dustymabe> lucab: +1
17:26:58 <fifofonix> met with them earlier lobbying for okd/fcos support.  they say yes.  just drumming it home repeatedly.
17:27:05 <dustymabe> jlebon: should we take an action item to try to come up with a proposal for jdoss' ticket?
17:27:13 <fifofonix> i think jaimelm cares.  maybe darkmuggle too.
17:27:19 <jaimelm> I need to contact them to be another voice.
17:27:20 <dustymabe> fifofonix: nice
17:27:22 * jaimelm cares
17:27:25 <jlebon> lucab: thanks for doing the releases!
17:27:29 <jlebon> dustymabe: SGTM
17:27:48 <fifofonix> yes, thanks for openssl release in stable.  felt a little slower than coreos days but that takes us back to cadence. etc.
17:27:58 <jlebon> #action jlebon and dustymabe to write up proposal for https://github.com/coreos/fedora-coreos-tracker/issues/785
17:28:11 <fifofonix> aob2: video next time?
17:29:02 <jlebon> i think we said beginning of each month IIRC
17:29:22 <dustymabe> yep
17:29:27 <fifofonix> +1
17:29:55 <jlebon> cool, anything else?
17:30:05 <jlebon> (wow we might actually finish on time this week?)
17:30:14 <jlebon> ok, closing in 45s :)
17:30:27 <dustymabe> 🕰️
17:30:34 <fifofonix> jaimelm: yes pls on contacting crowdtrike as other voice
17:30:43 <jaimelm> will do
17:30:59 <jlebon> #endmeeting