16:30:44 <jlebon> #startmeeting fedora_coreos_meeting
16:30:44 <zodbot> Meeting started Wed Jun  8 16:30:44 2022 UTC.
16:30:44 <zodbot> This meeting is logged and archived in a public location.
16:30:44 <zodbot> The chair is jlebon. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:30:44 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:44 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:50 <jlebon> #topic roll call
16:31:06 <dustymabe> .hi
16:31:08 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:31:19 <jlebon> #chair dustymabe
16:31:19 <zodbot> Current chairs: dustymabe jlebon
16:31:24 * dustymabe will have to drop in 10-15 to go get kid from camp
16:31:36 <aaradhak[m]> .hi
16:31:37 <zodbot> aaradhak[m]: Sorry, but user 'aaradhak [m]' does not exist
16:32:01 <jlebon> #chair aaradhak[m]
16:32:01 <zodbot> Current chairs: aaradhak[m] dustymabe jlebon
16:32:05 <jlebon> dustymabe: ack
16:32:33 <fifofonix> .hi
16:32:34 <zodbot> fifofonix: fifofonix 'Fifo Phonics' <fifofonix@gmail.com>
16:32:56 <saqali> .hi
16:32:57 <zodbot> saqali: saqali 'Saqib Ali' <saqali@redhat.com>
16:33:27 <jlebon> #chair fifofonix saqali
16:33:27 <zodbot> Current chairs: aaradhak[m] dustymabe fifofonix jlebon saqali
16:33:32 <marmijo> .hi
16:33:33 <zodbot> marmijo: marmijo 'Michael Armijo' <marmijo@redhat.com>
16:33:49 <jlebon> #chair marmijo
16:33:49 <zodbot> Current chairs: aaradhak[m] dustymabe fifofonix jlebon marmijo saqali
16:34:13 <lorbus> .hi
16:34:14 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:34:32 <jlebon> hmm, no travier, lucab, or bgilbert today
16:34:37 <jlebon> #chair lorbus
16:34:37 <zodbot> Current chairs: aaradhak[m] dustymabe fifofonix jlebon lorbus marmijo saqali
16:34:49 <jlebon> is walters here?
16:35:01 <ravanelli> .hi
16:35:02 <zodbot> ravanelli: ravanelli 'Renata Ravanelli' <renata.ravanelli@gmail.com>
16:35:29 <jlebon> #chair ravanelli
16:35:29 <zodbot> Current chairs: aaradhak[m] dustymabe fifofonix jlebon lorbus marmijo ravanelli saqali
16:36:06 <jlebon> ok, let's get started!
16:36:08 <jlebon> #topic Action items from last meeting
16:36:20 <jlebon> the previous meeting was a video meeting
16:36:27 <bgilbert> .hi
16:36:28 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:36:38 <jlebon> the action items were:
16:36:38 <jlebon> @cverna to open ticket for FCOS as an edition and update
16:36:39 <jlebon> @jlebon to open tickets for user stories for CoreOS Layering
16:36:44 <jlebon> #chair bgilbert
16:36:44 <zodbot> Current chairs: aaradhak[m] bgilbert dustymabe fifofonix jlebon lorbus marmijo ravanelli saqali
16:37:00 <jlebon> #info jlebon filed https://github.com/coreos/fedora-coreos-tracker/issues/1219
16:37:24 <jlebon> i don't think cverna opened a ticket
16:37:41 <jlebon> let's reaction that one
16:37:48 <jlebon> #action cverna to open ticket for FCOS as an edition and update
16:38:04 <jlebon> the one I filed is one of the meeting topics, so we'll get to it :)
16:38:32 <jlebon> let's start with...
16:38:35 <jlebon> #topic tracker: Fedora 37 changes considerations
16:38:39 <jlebon> #link https://github.com/coreos/fedora-coreos-tracker/issues/1222
16:38:50 <jlebon> dustymabe: want to introduce this one?
16:39:06 <dustymabe> yep - it's that time again
16:39:12 <jlebon> did you want to go over the changes now or just do it async?
16:39:28 <dustymabe> this time we're trying to get on top of changes a bit earlier in the process
16:39:48 <dustymabe> jlebon: I guess that depends. I can't remember if last  time we did an ad-hoc session outside of the meeting or not
16:40:09 <jlebon> dustymabe: i think so. we did an ad-hoc meeting just before the community mtg and went over the results and remaining ones together
16:40:24 <jlebon> let's schedule that for next week maybe?
16:40:36 <dustymabe> SGTM
16:40:57 <jlebon> #action dustymabe to schedule ad-hoc meeting to go over f37 changes ahead of next week's community meeting
16:41:00 <jlebon> :)
16:41:25 <jlebon> please reach out if you're interested in participating
16:41:52 <jlebon> of course, feel free also to discuss on the ticket
16:42:05 <dustymabe> yeah. I can make the meeting invite public, too
16:42:20 <jlebon> +1
16:42:50 <jlebon> it's nice that we're doing it even earlier this time
16:42:55 <jlebon> anything else on this before we move on?
16:44:01 <jlebon> ok, doesn't seem like it
16:44:09 <jlebon> #topic Produce official s390x artifacts
16:44:12 <jlebon> #link https://github.com/coreos/fedora-coreos-tracker/issues/1085
16:44:34 <jlebon> dustymabe: not sure if you're still around, otherwise we can bump it to the end or next week
16:44:42 <dustymabe> still here
16:45:15 <dustymabe> just wanted to bring up that we are making progress here and should be spitting out artifacts out of the pipeline soon
16:45:52 <dustymabe> so (unless there are objections) we'd eventually have it all wired up in stream metadata etc..
16:46:45 <jlebon> cool!
16:47:18 <dustymabe> do we know of things that need updating?
16:47:55 <dustymabe> i.e. places in our process that manually hardcode x86_64 and aarch64 versus inspecting the available architectures?
16:48:02 <jlebon> probably docs.  also sample metadata in the tracker would be good.
16:48:44 <jlebon> i would've expected we shaked most of those out with the addition of aarch64
16:48:45 <dustymabe> unofficial builds browser? website? any of our stream metadata/update metadta processing?
16:49:24 <dustymabe> #chair LakshmiRavichand
16:49:24 <zodbot> Current chairs: LakshmiRavichand aaradhak[m] bgilbert dustymabe fifofonix jlebon lorbus marmijo ravanelli saqali
16:49:39 <dustymabe> FYI LakshmiRavichand is the one who has been doing most of the enablement work
16:49:43 <bgilbert> websites should maybe be audited
16:49:47 <jlebon> build browser is a good one.  i should change it to use builds.json
16:49:58 <dustymabe> LakshmiRavichand++
16:50:44 <dustymabe> eventually we will build an ibmcloud artifact and run tests in ibmcloud (they have s390x instances)
16:51:02 <dustymabe> That's all I had
16:51:20 <LakshmiRavichand> thanks for adding me here, I second Dusty on the topic where we want to publish the s390x artifacts and need the necessary metadata change
16:51:21 <jlebon> bgilbert: in a quick perusal, it looks like it does the right thing
16:52:12 <jlebon> and eventually we'll upload to ibmcloud too, right dustymabe?
16:52:34 <dustymabe> jlebon: yes, the goal is to eventually offer images that other people can use in IBMCloud
16:52:45 <jlebon> would be cool to have a quicklaunch link on the getfedora.org website. probably the only realistic way most people are going to use this
16:52:47 <dustymabe> I've got a contact over there to work through thedetails with
16:52:49 <dustymabe> but it's slow
16:52:59 <bgilbert> +1 to tests; will be good to get big-endian exercised more
16:53:00 * dustymabe leaves to pick kid up from camp
16:53:29 <jlebon> cool.  alrighty, let's move on then!
16:53:47 <jlebon> #topic remove console=ttyS0 on metal
16:53:51 <jlebon> #link https://github.com/coreos/fedora-coreos-tracker/issues/567
16:54:01 <jlebon> bgilbert: i'll let you take this one :)
16:54:20 <bgilbert> so, this is moving again
16:54:29 <jlebon> 🎉
16:54:45 <bgilbert> background: we currently set "console=ttyS0,115200 console=tty0" kernel arguments on all platforms
16:54:55 <bgilbert> whoops, other way around
16:55:23 <bgilbert> the result is that the first serial port is the "primary" console and the graphical console is secondary
16:55:40 <bgilbert> there are a few problems with that, including:
16:55:58 <bgilbert> - certain log messages, notably Ignition failures, only go to the primary console.  historically the initrd emergency shell also only went to primary, though that's fixed now.
16:56:25 <bgilbert> (dustymabe just merged a PR to send Ignition failures to secondary console also)
16:56:41 <bgilbert> - some machines don't have a serial port, and will fail mysteriously if you try to use it
16:57:11 <bgilbert> - on some machines, serial port logging can make the system boot verrrry sllooooowwwly
16:57:19 <bgilbert> - it confuses people
16:58:03 <bgilbert> so, last year we agreed to switch to per-platform console configuration
16:58:35 <jlebon> - some machines don't have a seriali port or the port is on a different number*
16:58:39 <bgilbert> yup
16:58:41 <bgilbert> qemu images will stay as-is.  metal will default to no console= kargs at all, and let the user add them via the normal kargs mechanisms (coreos-installer or Ignition) if they want
16:59:00 <bgilbert> for other platforms, we'll do whatever the platform prefers.  e.g. on AWS, we'll only use serial console
16:59:20 <bgilbert> (sometimes the platform offers both and we have to make a decision)
17:00:07 <bgilbert> that also implies: `coreos-installer install --platform` (for doing a "bare metal" install in a VM) also needs to update the console settings to match the target platform's defaults
17:00:57 <bgilbert> the infrastructure for all of this has now landed, and a new coreos-installer release will happen soon.
17:01:07 <jlebon> hmm so e.g. on AWS there's the serial logs, but you also have the VGA screenshot. will there now be less logging there?
17:01:16 <bgilbert> jlebon: yes
17:01:36 <walters> awesome work!
17:01:39 <bgilbert> jlebon: mind commenting in the f-c-c PR?  we'll probably want to set VGA console as secondary
17:01:43 <jlebon> ok, that's interesting. any reason for that?
17:01:47 <jlebon> bgilbert: ahh k, cool
17:02:07 <bgilbert> we're still building with the legacy defaults.  the question for us now is, how do we want to switch over?
17:02:17 <jlebon> indeed, this is going to be so much cleaner!
17:02:54 <bgilbert> this is a breaking change for some workflows (notably bare metal clusters currently using serial console) so we should announce on coreos-status and give advance notice
17:03:17 <bgilbert> 1. how much notice should we give?
17:03:41 <bgilbert> 2. should we bake this in `next` before pushing to `testing`?
17:03:45 <bgilbert> EOM
17:03:54 <jlebon> bgilbert: should the console auto-forwarding be figured out before the switchover? it'd make action steps easier in the status message
17:04:36 <bgilbert> jlebon: you mean https://github.com/coreos/coreos-installer/pull/877 ?
17:04:45 <jlebon> for 2, i'd say yes. that should be our default workflow
17:04:55 <jlebon> yeah 877
17:05:26 <bgilbert> jlebon: for 877, I'm inclined to defer that until after the switch, and treat it as a quality-of-life improvement only
17:05:49 <bgilbert> jlebon: because otherwise we need to enable that behavior only for images not in legacy mode
17:06:18 <bgilbert> jlebon: which would require temporary glue to check for `platforms.json` before forwarding a karg
17:06:18 <jlebon> so for a live ISO install, users will want to do e.g. `coreos-installer iso customize --live-karg-append console=... --dest-karg--append console=...`, right?
17:07:04 <bgilbert> right.  Ignition kargs also works, but of course the first boot will be silent
17:07:33 <bgilbert> notably they already have to do --live-karg-append because the ISO image has always defaulted to no serial console
17:08:02 <jlebon> yup indeed. heh that was my original motivation for adding ISO kargs
17:08:31 <bgilbert> jlebon: agree/disagree on 877?
17:08:47 <jlebon> bgilbert: agree, yeah
17:09:00 <bgilbert> +1
17:09:08 <jlebon> for 1., it's hard to have a good measure because it's hard to tell how large of an impact this will have
17:09:41 <jlebon> but to be conservative, we should probably err on the safer side. so maybe... 3 months?
17:10:29 <jlebon> 1 month for next, then 2 months after that for promoting to testing?
17:11:15 <bgilbert> in general, how much notice do we want to give for next?
17:11:24 <bgilbert> it's documented as being more experimental
17:12:19 <jlebon> i'm fine with shorter for next. there's only one step under 1 month, which is 2 weeks :)
17:12:21 <bgilbert> and it's nice to give users a platform to test the upcoming changes
17:12:23 <bgilbert> right
17:12:41 <bgilbert> s/platform/place/ for less ambiguity
17:13:10 <jlebon> so, 2 weeks and then 2 months?
17:13:32 <bgilbert> yeah, I was about to say that 2 months would be my proposal
17:14:30 <jlebon> to be clear, do you mean 1.5 months after next or 2? i was saying the latter
17:14:42 <bgilbert> the latter
17:15:02 <jlebon> +1 cool
17:15:14 <jlebon> do you want to make a proposal?
17:15:14 <bgilbert> #proposed We will make the serial console changes in `next` two weeks after the coreos-status announcement, and in `testing` two months after `next`.
17:15:34 <jlebon> ack
17:16:20 <jlebon> fifofonix, ravanelli, walters, saqali thoughts?
17:16:49 <ravanelli> +1
17:17:14 <bgilbert> (we may also discover in `next` that some of our platform-specific defaults are wrong)
17:18:11 <saqali> proposition sounds good to me +1
17:18:53 <bgilbert> #agreed We will make the serial console changes in `next` two weeks after the coreos-status announcement, and in `testing` two months after `next`.
17:19:04 <walters> +1
17:19:24 <jlebon> thanks bgilbert!
17:19:26 <jlebon> let's move on to our final topic
17:19:33 <jlebon> #topic Develop Fedora CoreOS layering user stories
17:19:37 <jlebon> #link https://github.com/coreos/fedora-coreos-tracker/issues/1219
17:20:17 <jlebon> so, in the previous video meeting, we talked about CoreOS layering and how to make it fit into FCOS
17:20:40 <jlebon> one of the outcomes was that we should write down some user stories where CoreOS layering would help
17:21:08 <jlebon> this is the ticket tracking these stories. we can then split out the stories that interest us the most into separate tickets for further investigation
17:21:12 <walters> I missed the last meeting so I don't have the context, but this seems really broad
17:21:50 <jlebon> walters: that's somewhat by design :)  the implications of CoreOS layering are really broad
17:22:36 <jlebon> this is an attempt at tackling it from the UX angle to understand what we actually have to do
17:22:46 <bgilbert> walters: right, I think this would be a useful exercise for managing the scope
17:23:28 <bgilbert> ideally we can recommend layering for solving a specific set of problems
17:23:56 <jlebon> now, internally for OpenShift we did of course do a lot of user story writing already. and there's definitely going to be overlap.
17:24:39 <jlebon> but I also expect things that appear only in OCP or in FCOS
17:26:16 <jlebon> anyway, the intent isn't to come up with the use cases here, just to signal we have a place for them now. we'll have next steps for filling it out, but of course feel free also to comment there!
17:26:35 <bgilbert> +1
17:26:46 <fifofonix> +1
17:26:50 <jlebon> anything else on this topic before we move to open floor?
17:27:14 <dustymabe> jlebon: is the goal to come up with use cases here or should have have a dedicated session for that?
17:27:43 <jlebon> dustymabe: i think it'd be more productive to start with an ad-hoc session and then come back with something to present
17:27:53 <dustymabe> SGTM
17:27:59 <dustymabe> action item?
17:28:07 <jlebon> fair :)
17:28:37 <jlebon> #action jlebon to schedule meeting to discuss Fedora CoreOS layering use cases
17:28:44 <jlebon> ok let's do a quick open floor
17:28:48 <jlebon> #topic Open Floor
17:28:55 <jlebon> anyone have anything?
17:30:00 <dustymabe> I think the Nest conference is now accepting proposals for talks
17:30:31 <jlebon> +1
17:30:39 <dustymabe> there are also fedora elections going on now
17:30:54 <dustymabe> https://communityblog.fedoraproject.org/f36-elections-voting-now-open/
17:31:20 <dustymabe> I encourage you all to vote to take part in the Fedora community
17:31:45 <jlebon> thanks dustymabe
17:31:48 <fifofonix> i have one.
17:31:57 <jlebon> fifofonix: shoot
17:32:09 <fifofonix> has anyone ever successfully managed to get vscode to work with a remote fcos node?
17:32:17 <fifofonix> (totally random, haven't googled it, etc etc)
17:33:04 <dustymabe> fifofonix: good question.. I'm not sure. I know some people who use VSCode but they run linux on their laptop
17:33:27 <dustymabe> probably would be a good discussion forum question to reach a bigger audience?
17:33:59 <fifofonix> ok, will post there.
17:34:13 <jlebon> definitely sounds interesting
17:34:38 <jlebon> ending in 40s
17:34:57 <fifofonix> thanks for running jlebon
17:35:18 <jlebon> #endmeeting