16:00:06 <bowlofeggs> #startmeeting FESCO (2017-09-08)
16:00:06 <zodbot> Meeting started Fri Sep  8 16:00:06 2017 UTC.  The chair is bowlofeggs. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:06 <zodbot> The meeting name has been set to 'fesco_(2017-09-08)'
16:00:06 <bowlofeggs> #meetingname fesco
16:00:06 <zodbot> The meeting name has been set to 'fesco'
16:00:06 <bowlofeggs> #chair maxamillion dgilmore nirik jforbes jsmith kalev sgallagh bowlofeggs till
16:00:06 <zodbot> Current chairs: bowlofeggs dgilmore jforbes jsmith kalev maxamillion nirik sgallagh till
16:00:06 <bowlofeggs> #topic init process
16:00:13 <sgallagh> .hello2
16:00:14 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
16:00:15 <contyk> .hello psabata
16:00:17 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
16:00:19 <langdon> .hello2
16:00:19 <tyll> .hello till
16:00:20 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
16:00:23 <zodbot> tyll: till 'Till Maas' <opensource@till.name>
16:00:30 <jforbes> .hello jforbes
16:00:31 <zodbot> jforbes: jforbes 'Justin M. Forbes' <jforbes@redhat.com>
16:00:41 <strigazi> .hello strigazi
16:00:43 <zodbot> strigazi: strigazi 'Spyros Trigazis' <strigazi@gmail.com>
16:02:28 <langdon> bowlofeggs: i would just like to request we push modular server discussions to early in the q as I have a meeting conflict
16:02:29 <kalev> .hello kalev
16:02:30 <zodbot> kalev: kalev 'Kalev Lember' <klember@redhat.com>
16:02:35 <maxamillion> .hello maxamillion
16:02:36 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
16:02:41 <maxamillion> sorry I'm late
16:03:12 <jsmith> .hello jsmith
16:03:15 <zodbot> jsmith: jsmith 'Jared Smith' <jsmith.fedora@gmail.com>
16:03:16 <jsmith> Sorry I"m late as well
16:03:25 <jkurik> .hello2
16:03:26 <zodbot> jkurik: jkurik 'Jan Kurik' <jkurik@redhat.com>
16:03:55 <bowlofeggs> looks like we do have a quorum
16:04:07 <bowlofeggs> so let's get this pizartay started
16:04:23 <bowlofeggs> #topic #1737 Proposal: i686 SIG needs to be functional by F27 release
16:04:23 <bowlofeggs> date or we drop i686 kernel from F28
16:04:23 <bowlofeggs> .fesco 1737
16:04:23 <bowlofeggs> https://pagure.io/fesco/issue/1737
16:04:26 <zodbot> bowlofeggs: Issue #1737: Proposal: i686 SIG needs to be functional by F27 release date or we drop i686 kernel from F28 - fesco - Pagure - https://pagure.io/fesco/issue/1737
16:06:05 <maxamillion> so there's criteria in the ticket, is everyone happy with that?
16:06:11 <jsmith> I guess I'm OK with the definition of "functional" in the ticket...
16:06:21 <bowlofeggs> As I understand it, our current task with this issue is to define what criteria measures how "functional" the x86 SIG is?
16:06:25 <bowlofeggs> yeah i'm fine with it
16:06:34 <jsmith> ... it'll still be hard to say whether they're "responsive" and "helpful", but yeah -- I'm +1
16:06:42 <bowlofeggs> yeah it's a bit fuzzy
16:07:09 <bowlofeggs> so they would still need to achieve some of these metrics before F27 release, right?
16:07:15 <jforbes> I am a little concerned with some of the timeframes listed, 1-2 weeks for triage (not fix). That might leave things in a bad state for quite some time
16:07:22 <sgallagh> How are these measurable?
16:07:41 <sgallagh> let me rephrase
16:07:50 <kalev> I'm fine with the criteria. If it turns out not to be enough in practice, we can always refine things later.
16:08:03 <sgallagh> At what point do we step in and say "these promises aren't being met"?
16:08:11 <bowlofeggs> jforbes: good point. should we ask them to lower that particular criteria? 2-3 days on triage?
16:08:25 <sgallagh> (Snarky answer: "in four weeks")
16:08:35 <jforbes> bowlofeggs: I would say so.
16:09:09 <jforbes> I am also curious to see what the hardware requirements are there, do we continue supporting PAE, etc?
16:09:11 <bowlofeggs> sgallagh: it might just be a judgment call. perhaps similar to how we might say "is packager X absent?"
16:09:23 <tyll> how fast are non i686 kernel bugs triaged currently?
16:09:41 <sgallagh> bowlofeggs: We actually have a pretty detailed policy on that...
16:09:47 <bowlofeggs> yeah true
16:09:52 <bowlofeggs> i realized that after typing it:)
16:10:30 <bowlofeggs> would we want criteria about how many active members are in the SIG?
16:10:54 <bowlofeggs> like, a SIG of one person hardly fulfills the G in SIG :)
16:11:12 <bowlofeggs> and the defining "active" is difficult too
16:11:14 <jforbes> tyll: It's a bit different in that regard because we don't have "triage" in the common sense, but they are looked at by the next working day after they are filed pretty much at the latest
16:12:06 <kalev> I'm pretty content with having a SIG with a mailing list and a promise to look at things for now
16:12:29 <kalev> after all, i686 support in Fedora is in pretty good condition right now and hopefully we don't need that much work going into this in the beginning anyway
16:12:51 <jforbes> kalev: actually, there is an i686 specific build issue in rawhide right now
16:13:08 <kalev> fair enough, that sounds like something to put to their list to look at then
16:13:13 <kalev> and then we see how well it works in practice?
16:13:22 <jforbes> Probably a good idea, yes
16:14:27 <bowlofeggs> sounds like we want them to 0) lower the triage promise to 2-3 days, 1) define hardware requirement (PAE?), 2) deal with the current rawhide issue, 3) ...?
16:14:59 <tyll> how about requiring at least one successful test result for the default products for i686 according to https://fedoraproject.org/wiki/QA:Release_validation_test_plan done by the SIG to decide whether we will continue to ship the kernel?
16:15:14 <jforbes> Yes
16:15:21 <kalev> ah yes, that's an imprtant thing
16:15:22 <tyll> this should prove that at least someone is running the kernel successfully
16:15:41 <bowlofeggs> +1
16:15:50 <jforbes> tyll: My expectation is that the release passes full validation by F27 release timeframe
16:16:28 <alexpl> Please excuse me for cutting in, but which is the bug in question?
16:16:42 <bowlofeggs> .proposal 0) lower the triage promise to 2-3 days, 1) define hardware requirement (PAE?), 2) deal with the current rawhide issue, 3) require successful test result for default products for i686 according to https://fedoraproject.org/wiki/QA:Release_validation_test_plan
16:17:16 <jforbes> alexpl: I haven't filed it yet. It just appeared yesterday and I had to fix a different packaging issue across the other arch, but it will be filed today.
16:17:38 <bowlofeggs> cool, if you can send that my way when you file it i can mention it on the ticket
16:17:44 <bowlofeggs> (the fesco ticket)
16:18:12 <kalev> I think I'd like to remove the triage 2-3 days promise. That seems overly much to ask for community members who might occasionally be busy elsewhere
16:18:24 <kalev> other things in the list seem fine to me
16:18:24 <tyll> Isn't 2-3 days triage time a death sentence in practice? It seems to be quite a short time for community members. But it is also a better expectation that I would have for any bug filed for Fedora
16:18:35 <tyll> kalev: same thoughts :-)
16:18:39 * kalev nods.
16:18:45 <jforbes> alexpl: but the error is: ERROR: "__udivdi3" [net/netfilter/xt_hashlimit.ko] undefined!
16:19:12 <bowlofeggs> ok, should we compromise and ask them to just say "1 week" instead of "1-2 weeks"?
16:19:31 <bowlofeggs> we have reached 15 minutes on this one, btw
16:19:39 <bowlofeggs> do we wish to continue on it?
16:19:44 <tyll> +1 for 1 week and the other proposals
16:20:41 <jsmith> +1 for 1 week and the other proposals
16:20:50 <kalev> I think I'd keep it at 1-2 weeks, but sure, I can be +1 to that as well
16:21:30 <jforbes> I think honestly, it depends on the severity of the bug.
16:22:01 <jforbes> Kernel won't build, that's a problem.  Do we excludearch until it is fixed?
16:22:05 <kalev> build issues probably need someone to look at them quickly
16:22:08 * kalev nods.
16:22:31 <bowlofeggs> we could ask for a higher SLA on issues that block the primary arch kernel (like build)
16:22:50 <tyll> jforbes: yes, but we canot define this in detail, so I guess some reasonable expectations should suffice - if there are actuall severe issues then we could still revisit this
16:23:02 <jforbes> If that's what we do, I am all for it.  If not, we can't have 2+ weeks of no builds. That is literally thousands of changes during a merge window
16:23:23 <kalev> yes, I concur, 2+ weeks with no builds doesn't fly here
16:23:29 <sgallagh> I'd be willing to say: 2-3 days for turnaround on a build issue or it gets ExcludeArched until figured out
16:23:33 <bowlofeggs> yeah i agree that this shouldn't be able to block other arches from releasing
16:23:55 <sgallagh> With reasonable leeway given if the problem is being actively worked
16:23:58 <bowlofeggs> haha this proposal is getting complex, but i agree
16:24:11 <tyll> +1 for the excludearch supplement
16:24:25 <jsmith> Sure, +1 from me too
16:24:36 <bowlofeggs> ok, we will allow a 1 week SLA, but for issues that block primary arches we need a 2-3 day turnaround or the excludearch can be used
16:25:20 <bowlofeggs> i think i have 4 +1's on the current "excludearch supplement" proposal
16:25:24 <jforbes> I would propose we accept it as is, but append the excludearch thing. A build error is immediately excludearched
16:25:36 <jforbes> And added back when they get it fixed
16:25:49 <sgallagh> jforbes: "immediately"?
16:26:03 <jforbes> sgallagh: immediately.
16:26:04 <bowlofeggs> (we've now been on this ticket for 20 minutes)
16:26:08 <alexpl> Apologies again, should I wait for the end of the meeting for a couple of questions?
16:26:23 <sgallagh> bowlofeggs: This is an important topic. I vote to continue
16:26:34 <jforbes> I would agree
16:26:40 <bowlofeggs> fine by me - i just saw that it's my job to bring up the time :)
16:26:49 <sgallagh> alexpl: If it's relevant to this topic, please raise it
16:27:00 <jforbes> alexpl: as a member of the x86 SIG, your questions and feedback would be more than welcome here
16:27:16 <alexpl> OK then,
16:27:42 <bowlofeggs> i think i am ok with "immediately"
16:27:49 <alexpl> we hadn't discussed something like that during our first meeting, but why would there be a problem if you use the excludearch right away?
16:28:08 <jforbes> sgallagh: immediately simply because of the amount of churn during the development cycle, we are talking about rawhide here, and the change rate is massive during the merge window.
16:28:21 <sgallagh> jforbes: Okay, you'd know better than I about that.
16:28:37 <jsmith> Apologies, folks, but I have to run :-(
16:28:41 <sgallagh> I was concerned it would appear hostile to the x86 folks
16:28:44 <jforbes> Days of not being able to push a build on any arch makes it a lot more difficult to track down when a bug was actually introduced, this is why we do daily builds
16:28:52 <bowlofeggs> i know the kernel team is pretty overwhelmed as it is, i wouldn't want to require them to wait to do their jobs when there are build issues
16:28:58 <tyll> jsmith: would you be ok with: Requires bugs to be triaged within 1 week, allow to excludearch i686 if it breaks the build, require hardware requirements, request successful test results for default products for i686 according to https://fedoraproject.org/wiki/QA:Release_validation_test_plan
16:29:39 <jsmith> Yes
16:29:49 <jsmith> (I've voted on the other two tickets in Pagure)
16:30:04 <bowlofeggs> thanks jsmith, enjoy your weekend
16:30:06 <jsmith> Wish I could stick around... but life calls...
16:30:09 <jsmith> Thansk bowlofeggs
16:30:13 <bowlofeggs> we still have a quorum, so we can continue
16:30:21 <tyll> jsmith: thx, have a nice weekend
16:31:05 <bowlofeggs> is anyone currently opposed to the excludearch "hammer"?
16:31:07 <jforbes> sgallagh: I don't think it is hostile, we used to do the same for arm.  They were very responsive with it.
16:31:10 <alexpl> sgallagh: OK, I can understand the issue of perceived hostility and I can't speak for anyone else, as we haven't discussed this, but I guess I would be OK with it
16:31:14 <tyll> how about the last proposal? it would allow to just use excludearch for the current rawhide isse?
16:31:31 <kalev> jforbes: if it's the same treatment for arm as well, then I think it makes sense to use that for i686 as well
16:31:34 <sgallagh> jforbes, alexpl: OK, I withdraw my concern.
16:31:57 <sgallagh> Or rather, my concern has been addressed
16:31:59 <kalev> I just don't want to have higher bar for i686 than what we have for arm and other secondary arches
16:32:11 <bowlofeggs> .proposal 0) lower the triage promise to 1 week but allow main kernel to excludearch for build issues, 1) define hardware requirement (PAE?), 2) deal with the current rawhide issue, 3) require successful test result for default products for i686 according to https://fedoraproject.org/wiki/QA:Release_validation_test_plan
16:32:19 <sgallagh> kalev: arm is actually a primary arch...
16:32:20 <jforbes> Well, arm isn't secondary anymore
16:32:40 <alexpl> and the second question is about "triage" and how exactly you define it
16:33:26 <bowlofeggs> to me, triage means understanding the bug enough to be able to prioritize it
16:33:27 <jforbes> alexpl: We have a kernel triage page which walks you through how to triage, though for x86, I expect it needs some slight modification because the sig needs to be aware/CCed.
16:33:38 <sgallagh> alexpl: I'd define it as "someone from the x86 SIG comments on the bug addressing its severity
16:33:56 <jforbes> alexpl: https://fedoraproject.org/wiki/KernelBugTriage
16:34:05 <bowlofeggs> ah nice, formally documented
16:34:07 <kalev> can't we just add amend that kernel gets ExcludeArch'd when i686 fails to build and it's up to the SIG to fix that, and accept the proposal as is?
16:34:19 <kalev> and then see how it goes?
16:34:23 <tyll> alexpl: alexpl there is some description here: https://pagure.io/fesco/issue/1737#comment-464266
16:34:32 <bowlofeggs> kalev: i think that's the current .proposal
16:35:22 <alexpl> I got a little confused, so the primary concern is with the kernel and not just any 32-bit-specific bug that appears on any other package?
16:35:51 <bowlofeggs> alexpl: the comment from jsbackus does say that
16:35:59 <bowlofeggs> "The question is specifically about support for x86 hardware, especially the kernel and boot components."
16:36:04 <bowlofeggs> so i guess grub as well
16:36:05 <tyll> alexpl: ther kernel and that the install media is tested/works on i686 systems
16:36:11 <kalev> the whole topic to drop i686 was brought up by the kernel team who don't have the resources to support that, so a lot of focus is here on the kernel, yes
16:36:32 <jforbes> alexpl: kernel/bootloader, but yes. There is no proposal for dropping the rest of i686, it is required for multilib at the least
16:36:37 <alexpl> Great, thanks for the clarifications
16:36:40 <bowlofeggs> but we also want the sig to be able to pass the release validation test plan
16:37:12 <sgallagh> bowlofeggs: That sets the bar kind of high for a non-release blocking spin
16:37:38 <sgallagh> I don't think I'd require that; it's more than we ask for any other arch or spin
16:38:33 <bowlofeggs> i agree we shouldn't require anything of this SIG that we wouldn't for other non-primary arches (secondary? alternative? i can't ever rmember which word we use)
16:38:43 <alexpl> Are these arch-specific requirements documented someplace in the wiki?
16:39:26 <bowlofeggs> we have now been on this topic for 35 minutes: are we ready to vote, or do we want more discussion?
16:39:27 <jforbes> sgallagh: I agree.  My only concern is that we end up with a zombie because there is a SIG "in place" that doesn't actually do anything and i686 just further bit rots.
16:39:29 <sgallagh> alexpl: it's not arch-specific. It's about release-blocking or not
16:39:54 <kalev> right, I don't want to make the bar for i686 higher than what it is for other secondary arches. like I said earlier, can we just go with what the SIG proposed and only amend the kernel excludearch thing?
16:39:55 <jforbes> there needs to be some way of measuring that they are actually making effort
16:39:56 <bowlofeggs> jforbes: we could revisit it if there is a sense that that is happening
16:40:02 <kalev> and if there are concerns that this isn't working we can take this up again?
16:40:14 <alexpl> I won't delay you any longer, are the other details covered in https://fedoraproject.org/wiki/Architectures ?
16:40:24 <tyll> if the i686 image is promoted on the main download page it is IMHO reasonable to require some kind of release-readiness testing for it
16:40:54 <athos> jforbes: if it isn't too much to ask, if you are  reporting a bug about setting the ExcludeArch, could you block FE-ExcludeArch-x86?
16:41:09 <sgallagh> tyll: We haven't committed to allowing that
16:41:24 <sgallagh> I'd prefer that it end up on the Spins page if anything
16:42:04 <jforbes> Okay, so proposal: The functional SIG definition is approved. Build issues will be ExcludeArched with a block to FE-ExcludeArch-x86. We would like to see a hardware support detail soon.
16:42:20 <bowlofeggs> +1
16:42:22 <kalev> +1 to jforbes
16:42:25 <jforbes> athos: absolutely.
16:42:37 <tyll> +1
16:42:39 <sgallagh> jforbes: +1
16:42:57 <jforbes> +1 to myself
16:43:09 <tyll> maxamillion: ?
16:43:31 <tyll> afaics hes is the only one missing
16:43:38 <alexpl> thanks again, I don't know if athos has anything else to add, I am shutting up
16:43:46 <kalev> thanks alexpl
16:43:53 <bowlofeggs> that's 5 votes
16:43:54 <bowlofeggs> maxamillion: ?
16:44:05 <tyll> 5 votes would be enough, though
16:44:11 <bowlofeggs> true
16:44:27 <bowlofeggs> #agreed  The functional SIG definition is approved. Build issues will be ExcludeArched with a block to FE-ExcludeArch-x86. We would like to see a hardware support detail soon. (+5, 1, -0)
16:44:44 <bowlofeggs> #topic #1760 F27 approved Changes not in MODIFIED status (considered as
16:44:44 <bowlofeggs> not testable)
16:44:44 <bowlofeggs> .fesco 1760
16:44:44 <bowlofeggs> https://pagure.io/fesco/issue/1760
16:44:46 <zodbot> bowlofeggs: Issue #1760: F27 approved Changes not in MODIFIED status (considered as not testable) - fesco - Pagure - https://pagure.io/fesco/issue/1760
16:45:33 <maxamillion> tyll: bowlofeggs: sorry ... multi tasking badly, I'm back now
16:45:34 <tyll> if langdon is still there, should we maybe first discuss modular server?
16:45:36 <kalev> are the relevant people here to ask questions? langdon, otaylor?
16:45:38 <langdon> im here
16:45:39 * kalev nods.
16:45:42 <otaylor> i'm here
16:45:46 <langdon> but in a lyft.. so.. you know
16:46:05 <langdon> contyk and threebean should also be able to help out
16:46:14 <bowlofeggs> and i'm here to talk about bodhi
16:46:15 <bowlofeggs> :
16:46:16 <bowlofeggs> :)
16:46:23 <contyk> hey
16:47:32 <tyll> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/EVSL6KWU4TRZFQU5TU32Q5H6JTL5OILJ/ is the today discussion about modular server
16:47:54 * threebean too
16:48:01 <langdon> yeah.. i posted a comment there a bit ago
16:48:05 <bowlofeggs> we are currently scheduled to release the f27 beta in 11 days. i'm concerned about the current state of modular server, and the bodhi change is related to this too (and isn't ready yet)
16:48:22 <langdon> what is the bodhi change waiting on?
16:49:03 <threebean> just tests, iirc.. and not deploying during beta freeze.
16:50:03 <bowlofeggs> langdon: also, it could ideally also use formal testing
16:50:11 <bowlofeggs> in addition to unit tests, i mean
16:50:19 <langdon> ok.. gotcha
16:50:51 <langdon> so.. re the server, we have been impacted by the uefi changes much like the rest of fedora.. we have many modules building but no "image" yet
16:51:04 <langdon> and the contingency plan is already in place
16:51:08 <bowlofeggs> i also feel nervous about putting such a change in place in the window between the beta and final freezes, as i mentioned in the thread
16:51:35 <langdon> bowlofeggs: does the module change for bodhi affect anything "not module"?
16:51:43 <langdon> *effect
16:51:46 <threebean> bowlofeggs: otoh, it's one of those things that is segregated into a different code path in the masher.  if it bombs, we should be able to turn it off and just not mash modules.
16:51:47 <contyk> I put my comments in the Host & Platform change bugzilla ticket earlier today
16:52:08 <contyk> there were/still are several issues delaying the work
16:52:09 <bowlofeggs> langdon: there isn't a setting to turn it off, so if it blows up we will have to patch bodhi to remove it in some way
16:52:29 <tyll> at the current state I would prefer to defer since we have a short schedule anyhow to get back on track with the release dates, therefore not defering to F28 sounds very risky
16:52:36 <bowlofeggs> which isn't "difficult" but it's not just a settings change
16:52:51 <contyk> but we think we might have a working compose with buildinstall + some images somtime next week
16:53:01 <langdon> tyll: i guess i am not sure, aside from the bodhi change, how deferring would help anything
16:53:11 <bowlofeggs> i think i also am inclined to think that deferring to F28 might be best
16:53:53 <langdon> as contyk says.. we should have working images end of next week
16:54:38 <threebean> yeah - can we revisit this next week (friday)?
16:54:44 <bowlofeggs> we would also be making a beta that we can't ship updates for, for at least some time
16:54:55 <langdon> bowlofeggs: why is that?
16:55:12 <langdon> like if we ship modular server without the bodhi change?
16:55:16 <bowlofeggs> langdon: because the bodhi that can mash modules can't go in during the freeze
16:55:50 <threebean> bowlofeggs: huh - I thought you were ok with targetting deployment of that after beta freeze?
16:55:52 <bowlofeggs> this also means that we can't test updating modular server for a while
16:55:58 <langdon> ohh.. well.. i don't think we should ship mdoular server if we can't do updates
16:56:17 <maxamillion> langdon: +1
16:56:21 <langdon> like we need bodhi change in place else no modular server, basically period
16:56:23 <bowlofeggs> threebean: as a bodhi dev, i am ok with that. as a fesco member, i am worried about it. so i'm conflicted if that makes sense
16:56:33 <langdon> we could do boltron-2.0 .. but not mainline
16:56:34 <threebean> I see.
16:57:21 <threebean> we're not going to see verification of the bodhi change until after Beta unless we file a freeze break exception and go for it next week.
16:57:34 <langdon> I don't know.. i guess i am pro some level of risk for innovation
16:58:05 <bowlofeggs> threebean: yeah that's my concern. an FBR is also risky because it's not an insignificant patch
16:58:37 <langdon> bowlofeggs: i assume it would be difficult to put in a "turn off module masher" switch?
16:59:02 <threebean> releng is still producing a traditional Server edition for beta atm (in addition to the modular one in a separate compose).  if the bodhi change bombs after beta, we can always resort to shipping the traditional Server for final.
16:59:30 <langdon> threebean: right.. this is why i don't think there is much risk here
16:59:39 <bowlofeggs> langdon: well it would be a weird switch, because module updates would be sitting around waiting to get pushed - we'd get pinged by devs, and the switch would be there for other users of bodhi (fedora isnt' the only user of bodhi)\
16:59:59 <langdon> we are already doing the contingency plan .. and we know it works.. and we can decide a week before final if we had too
17:00:16 <bowlofeggs> i usually am averse to adding special workarounds to upstream bodhi, especially if they're just temporary
17:01:03 <bowlofeggs> we've now been on this topic for 15 minutes. i vote to continue.
17:01:20 <sgallagh> +1 to continue
17:01:29 <jforbes> +1
17:01:40 <maxamillion> yeah, if things aren't ready, then "Boltron 2.0" would be an option IMO so that people who want to play with the new shiny can, we don't block people from working on things, but we also don't derail f27 contingent on things that are still relatively in flight
17:02:03 <bowlofeggs> the problem is that the beta release and the final release wouldn't be the same
17:02:04 <tyll> don't we also need to update the websites/release notes/... to show which server variant we will ship? Deciding everything on the last minute might create extra stress here, too
17:02:08 <tyll> +1 to continue
17:02:29 <langdon> bowlofeggs: what would it take to deploy new bodhi and run an update through ? like could we just plan a day when we have an image and an update, deploy bodhi, try it, and roll back if it is a problem?
17:02:35 <bowlofeggs> so if we use the contingency plan after the beta, the beta that was tested isn't related to the thing we ship
17:03:03 <sgallagh> bowlofeggs: Yeah, I concur.
17:03:16 <sgallagh> If we ship Modular for Beta, we commit to shipping Modular for Final
17:03:22 <sgallagh> And the reverse, of  course
17:03:31 <bowlofeggs> langdon: that's possible, but rolling bodhi back is not trivial when there are migrations involved. i don't know off the top of my head if there are migrations coming, but i don't think there are...
17:03:41 <threebean> I don't think there are, either.
17:04:02 <bowlofeggs> doing that during a freeze would still probably not be great
17:04:13 <bowlofeggs> especially since rolling back *can* be weird
17:04:24 <langdon> :(
17:04:31 <langdon> levaing aside bodhi for the moment.. how about we revisit next meeting and make a final call? or even the meeting after that assuming beta hits the rain date rather than "real"?
17:04:51 <langdon> and ensure beta / final map correctly?
17:07:07 <sgallagh> langdon: If we were to forcibly invoke the "rain date" as an Act of FESCo right now... what would be your certainty level on delivering Modular?
17:07:10 <bowlofeggs> i'm nervous that we aren't sticking to the schedule
17:07:13 <contyk> can't we have the bodhi change without modular content? just have it and not use it?
17:07:43 <bowlofeggs> because this stuff was supposed to have been done by aug 01, and we're on an exception already
17:08:01 <langdon> sgallagh: i think it is more contyk's confidence.. mine is pretty high for our bits.. but h&p is still not building
17:08:05 <bowlofeggs> the schedule is there to protect the quality of the release for the users
17:08:21 <langdon> *building images
17:08:27 <bowlofeggs> so giving even more exception to it doesn't sit right with me
17:08:52 <contyk> h&p is building fine
17:08:56 <contyk> it's just the compose
17:09:04 <langdon> yeah... word choice is tough
17:09:56 <Southern_Gentlem> Null clear your cache and look again
17:10:25 <tyll> bowlofeggs: I share your opinion
17:10:32 <sgallagh> Southern_Gentlem: Wrong channel?
17:12:57 <jforbes> It seems that disucssion is deadlocked
17:13:27 * contyk waits
17:13:56 <sgallagh> I'm highly conflicted.
17:14:08 <sgallagh> I really want to see this get out the door, and it's very close.
17:14:26 <sgallagh> On the other hand, it's pretty risky, especially at this point in the schedule.
17:14:41 <bowlofeggs> i have the same internal conflict
17:14:57 <tyll> How about we decide not allow to delay the current schedule by the modular server but in case it gets ready somehow for Beta we still allow it? Not sure how realistic this is.
17:14:59 <bowlofeggs> if modularity goes badly in f27 it could harm it's perception too
17:15:10 <sgallagh> tyll: Not realistic, it sounds
17:15:20 <sgallagh> Either we have to be willing to block for it, or we have to defer it.
17:15:37 <langdon> i guess i still struggle with seeing the risk.. we can fall back to the traditional server pretty easily as long as we do it before the beta, right? which isn't for a few weeks
17:15:42 <sgallagh> bowlofeggs: That's also a fair point. It's an uphill battle already.
17:16:06 <sgallagh> The Beta go/no-go is next Thursday
17:16:08 <sgallagh> langdon: ^^
17:16:17 <jforbes> So here is my concern. Bodhi sounds like a point of contention. the fact that bowlofeggs isn't pushing that things are ready doesn't give me warm fuzzies.
17:16:18 <langdon> its not like the team is gonna stop working on it either way
17:16:33 <langdon> jforbes: +1
17:16:50 <sgallagh> Yeah, if we do this, I want us to do it right.
17:17:16 <sgallagh> But there's also a lot of pressure on us to do this *soon* :-|
17:17:26 <langdon> the tradeoff is if it is off to the side ... it doesn't get used
17:17:29 <jforbes> Yes, it is a matter of flipping a switch to go between modular release and contingency plan, and that's cool, but getting a release ready and not being able to ship updates is somewhat scarey
17:17:32 <contyk> I still don't see why the bodhi change depends on the content being ready
17:17:45 <threebean> contyk: it doesn't.
17:17:56 <threebean> we could do it now - but we're in freeze.
17:18:01 <langdon> jforbes: right.. i think the two decisions should be decoupled..
17:18:41 <threebean> so how about this... I can't promise it because it needs to be scoped and cleared with the infra team, but I can look into rebasing the modular changes as a hotfix to bodhi and trying a staging upgrade sooner, before beta is out.
17:18:46 <jforbes> langdon: I definitely do not, I don't care if you have a perfect release ready for beta, if we don't have confidence that we can ship updates, the beta is useless
17:18:56 <langdon> isn't the only reason bodhi wasn't done before freeze was time? the code has been ready for quite awhile?
17:19:10 <threebean> if, next week, we can revisit this on two points.  one with respect to the images in the compose, and a second with respect to bodhi.
17:19:12 <langdon> jforbes: sorry.. thats what i was trying to say :)
17:19:38 <threebean> .. then fesco can make a final call on whether or not to formally activate the contingency.
17:19:42 <maxamillion> threebean: if infra signs off on that, I think that's a decent option but I don't know the real impact of that and would defer to infra for that decision and would support their recommendation either way
17:19:49 <langdon> as in ... we keep shipping it.. but if we can' t get updates in place.. we go to contingency.. but we don't defer modular server to f28 unless it is because bodhi can get deployed
17:20:18 <langdon> *can't get deployed .. fingers failing me
17:20:46 <bowlofeggs> as bodhi dude, i am ok with threebean's proposal to make a bodhi release with this, assuming that infra allows the FBR. if he can do that next week before the go/no-go, we could use the result of go/no-go to make a call at next week's meeting?
17:21:05 * mattdm has been reading along....
17:21:08 * threebean puts his hard hat on.
17:21:18 <mattdm> I think the go/no-go meeting as decision point makes sense
17:21:19 <bowlofeggs> it's important to me that whatever we ship as beta be the same as whatever we ship at final release
17:21:30 <langdon> bowlofeggs: i can agree with that
17:21:34 <mattdm> yeah me too
17:21:39 <maxamillion> bowlofeggs: yeah, if you're good with it and infra signs off, then I think that's a good idea ... either way I'm on board with defering to next week
17:22:59 <maxamillion> thoughts from the group?
17:23:16 <tyll> if the decission on thursday is "go" can we still invoke the contingency plan in time? Or will it be no-go anyhow?
17:23:18 <jforbes> +1 bowlofeggs
17:23:46 <mattdm> tyll: The contingency plan is easy because that's the current state of things right now anyway -- traditional server is still being built
17:23:54 <langdon> tyll: contingency is already "in place" we actually have to do work to make traditional server *not* the one we ship
17:24:06 <langdon> yeah.. what he said ;)
17:24:19 <bowlofeggs> ok so current proposal:
17:25:16 <bowlofeggs> .proposal threebean will ask for an FBR for a special bodhi release that adds modular mashing for next week. we will revisit this in light of the go/no-go meeting
17:25:24 <tyll> so the only difference is in what we copy to the mirrors? No need to change anything else?
17:25:34 <mattdm> tyll: websites.
17:26:03 <mattdm> ping robyduck for future reference since you're probably not online right now ^ see above
17:26:05 <jforbes> bowlofeggs: +1
17:26:13 <tyll> mattdm: so if we continue with modula we will not promote the traditional server on the websites for beta?
17:26:43 <tyll> mattdm: and if we still use traditional we will not mention modular there?
17:26:47 <bowlofeggs> tyll: yes, that is my understanding
17:26:47 <langdon> tyll: yeah.. i think that would be the plan.. the traditional server would still be available if the content you want is not modularized yet
17:27:06 <mattdm> tyll: I think we'll mention the other one in a sidebar. I'll work with langdon, sgallagh, and robyduck on that.
17:27:16 * mattdm waits on proposal voting though
17:28:56 <tyll> bowlofeggs: would you also test in staging whether we can easily revert the modular chnages in Bodhi if everything breaks?
17:29:44 <threebean> tyll: yes.  that's on my list.
17:30:29 <bowlofeggs> maxamillion, kalev, tyll, sgallagh: are you ready to vote on current .proposal, or more info/discussion needed?
17:30:38 <bowlofeggs> (we're at +1)
17:30:57 <bowlofeggs> (and we've been on this topic for 46 minutes
17:30:58 <bowlofeggs> )
17:31:05 <tyll> ok, +1 from me to test Bodhi update/downgrade on staging, ask for FBR and revisit after go/no-go
17:31:19 <bowlofeggs> cool, that's actually +3
17:31:25 <maxamillion> +1 to bowlofeggs proposal
17:31:29 <bowlofeggs> (i forgot myself when i said +1)
17:31:50 <kalev> +1
17:31:53 <sgallagh> bowlofeggs: +1, I suppose
17:31:59 <bowlofeggs> ok, that's +6
17:32:31 <bowlofeggs> #agreed threebean will ask for an FBR for a special bodhi release that adds modular mashing for next week. we will revisit this at next week's FESCo meeting, in light of the go/no-go meeting's result (+6, 0, -0)
17:33:01 <langdon> ok.. im wicked late for my other meeting.. anything else you need from me? and to clarify, this is in regards to the 3 different change proposals (modular server, release, and h&p)
17:33:03 <bowlofeggs> that's actually only one of the things we needed to revisit (or two i guess, counting bodhi)
17:33:26 <langdon> meant to have "?" at the end of last
17:33:53 <bowlofeggs> not that i know of
17:34:05 <bowlofeggs> shall we discuss flatpak too, since otaylor is here?
17:34:32 <bowlofeggs> (do we make sub-topics for tickets like this one, where there are many changes being discussed in one ticket?)
17:34:35 <maxamillion> bowlofeggs: I'm in
17:34:39 <langdon> ok.. im off.. ill still be sorta here but I will be on mobile
17:35:13 <bowlofeggs> ok then, flatpaks are on deck. otaylor: what's the current status?
17:35:16 <otaylor> https://bugzilla.redhat.com/show_bug.cgi?id=1474769#c5
17:36:21 <otaylor> It doesn't seem like a f27 feature at this point in the sense of anything we can advertise to users or even most Fedora developers
17:36:36 <jforbes> So defer to F28 at this point seems logical
17:36:56 <bowlofeggs> since it's not critical for the release and can be added at any time, i'm +1 to deferring (and getting it done in between freezes sounds great too)
17:36:56 * sgallagh nods
17:36:58 <otaylor> Hopefully (and as discussed with maxamillion and bowlofeggs) we can keep the infrastructure pieces moving
17:37:05 <maxamillion> +1
17:37:08 <bowlofeggs> +1
17:37:15 <kalev> +1
17:37:16 <jforbes> +1
17:37:17 <maxamillion> I don't see any reason to not continue working on this from the backend side of things
17:37:30 <tyll> +1
17:37:35 <bowlofeggs> excellent
17:38:05 <bowlofeggs> #agreed Flatpaks will be deferred to F28 and can be done in between freezes (+6, 0, -0)
17:38:05 <sgallagh> +1
17:38:13 <bowlofeggs> whoops
17:38:16 <bowlofeggs> well, it's 6 now
17:38:18 <bowlofeggs> i miscounted
17:38:18 <bowlofeggs> sorry
17:38:30 <bowlofeggs> ok
17:38:45 <bowlofeggs> now bodhi - we basically talked about it together with modular server, right?
17:38:54 <sgallagh> yes
17:38:56 <bowlofeggs> any reason to discuss it too, or also defer
17:39:07 <maxamillion> I'd say also defer
17:39:09 <bowlofeggs> +1
17:39:14 <bowlofeggs> ok
17:39:15 <jforbes> They are codependent
17:39:18 <jforbes> +1
17:39:21 <maxamillion> jforbes: +1
17:39:30 <bowlofeggs> there's a Java 9 one
17:39:55 <bowlofeggs> oh i guess that one was already agreed on, so no further discussion?
17:40:00 <bowlofeggs> i wasnt' there for that meeting
17:40:21 <bowlofeggs> ok i think we are done with this ticket
17:40:27 <kalev> yes, I think we agreed to allow Java 9 and that's one of the changes that didn't need further discussion
17:40:48 <jkurik> bowlofeggs: Java 9 is already implemented and is currently under testing: https://bugzilla.redhat.com/show_bug.cgi?id=1447237
17:40:52 <bowlofeggs> ack
17:41:01 <bowlofeggs> #topic #1770 Allow PRs for non-packagers
17:41:02 <bowlofeggs> .fesco 1770
17:41:02 <bowlofeggs> https://pagure.io/fesco/issue/1770
17:41:03 <zodbot> bowlofeggs: Issue #1770: Allow PRs for non-packagers - fesco - Pagure - https://pagure.io/fesco/issue/1770
17:41:42 <tyll> I believe it is mostly a technical issue to properly implement this
17:41:48 <bowlofeggs> indeed
17:41:50 <tyll> from the policy side I am +1
17:41:57 <bowlofeggs> +1
17:42:01 <kalev> Sure, I think this makes a lot of sense -- not sure it needs FESCo to decide anything though
17:42:12 <kalev> just a missing feature
17:42:47 <maxamillion> +1 here as well as far as the policy goes
17:43:12 <jforbes> It actually goes a long way towards lowering the barrier to entry, making it easier for people to get involved.  This has been an issue for years
17:43:17 <jforbes> I am +1
17:43:23 <bowlofeggs> excellent
17:43:28 <bowlofeggs> i think that's a +6
17:43:37 <sgallagh> Well, it's about what the policy should be
17:43:54 <sgallagh> We should have it written somewhere so pagure admins can make sure reality matches
17:44:01 <bowlofeggs> good point
17:44:16 <bowlofeggs> sgallagh: i liked the wording on your comment there
17:44:20 <sgallagh> I'm +1 to my proposal, obviously
17:44:28 <bowlofeggs> haha
17:44:32 <maxamillion> +1 to sgallagh's proposal
17:44:40 <tyll> I believe it is because on the previous dist-git system it required packager group membership as basic requirement for SSH access
17:45:21 <tyll> we did not have any non-packager repos on the machine so it made sense
17:46:15 <bowlofeggs> ok, so let's formalize the proposal and vote to be clear
17:46:33 <bowlofeggs> .proposal Go with sgallagh's proposal in his first comment on the ticket
17:46:38 <bowlofeggs> +1
17:46:40 <maxamillion> bowlofeggs: +1
17:46:44 <jforbes> +1
17:46:48 <maxamillion> bowlofeggs: also, I don't think .proposal is a macro for anything
17:46:53 <bowlofeggs> haha
17:46:59 <bowlofeggs> i think i saw dgilmore do that last week?
17:47:02 <sgallagh> +1
17:47:04 <maxamillion> maybe #proposal ?
17:47:12 <maxamillion> for meetbot
17:47:18 <maxamillion> I dunno .
17:47:25 <kalev> +1
17:47:31 <tyll> or just #info proposal:
17:47:34 <tyll> +1
17:47:38 <bowlofeggs> ok that's 6
17:48:08 <bowlofeggs> #agreed anyone who has signed the FPCA to be able to fork, push to their private fork and submit PRs against the master. packager group is required to merge to main repo (+6, 0, -0)
17:48:25 <bowlofeggs> #topic #1771 Unresponsive maintainer: kanarip
17:48:26 <bowlofeggs> .fesco 1771
17:48:28 <zodbot> bowlofeggs: Issue #1771: Unresponsive maintainer: kanarip - fesco - Pagure - https://pagure.io/fesco/issue/1771
17:48:46 <kalev> +1 to orphaning all kanarip's packages
17:48:51 <bowlofeggs> +1
17:48:52 <tyll> +1
17:49:02 <sgallagh> +1
17:49:42 <bowlofeggs> that's +5 so far, including jsmith
17:49:53 <bowlofeggs> maxamillion, jforbes: care to vote?
17:50:00 <jforbes> +1
17:50:05 <maxamillion> +1
17:50:24 <bowlofeggs> #agreed orphan all of kanarip's packages (+7, 0, -0)
17:50:36 <tyll> I can take care of this
17:50:43 <bowlofeggs> ok, one last (big) ticket
17:50:47 <bowlofeggs> thanks tyll!
17:50:55 <tyll> #action till orphane all of kanarip's packages
17:51:02 <bowlofeggs> #topic #1773 F27 Changes not in ON_QA status (<100% completed)
17:51:03 <bowlofeggs> .fesco 1773
17:51:04 <zodbot> bowlofeggs: Issue #1773: F27 Changes not in ON_QA status (<100% completed) - fesco - Pagure - https://pagure.io/fesco/issue/1773
17:51:27 <bowlofeggs> there are a lot of these
17:51:57 <bowlofeggs> i think i will set a topic for each to make the meeting notes clearer
17:52:03 <jforbes> pjones: 32bit UEFI status?
17:52:11 <bowlofeggs> #topic 32 bit UEFI Support
17:52:20 <bowlofeggs> https://bugzilla.redhat.com/show_bug.cgi?id=1474861
17:52:25 <bowlofeggs> https://fedoraproject.org//wiki/Changes/32BitUefiSupport
17:53:30 <bowlofeggs> is this just waiting on signing from MS?
17:55:02 <jforbes> Appears so
17:55:08 <bowlofeggs> should we defer since pjones isn't responding?
17:55:30 <bowlofeggs> beta freeze was the contingency deadline
17:55:55 <sgallagh> Well, the contingency here should just be a trivial deferral, yes?
17:55:57 <jforbes> I would recommend that. Chances are he won't be here next week due to Plumbers, but I am sharing a room with him, I will have him update the ticket before the meeting
17:57:39 <bowlofeggs> ok
17:57:45 <bowlofeggs> #agreed defer until next week
17:57:53 <bowlofeggs> well i guess we didn't vote
17:57:55 <kalev> should we perhaps defer everything to next week? we're already at 2 hours meeting time here
17:58:38 <tyll> +1 to deferring
17:58:44 <bowlofeggs> i could be a +1 to defer until next week
17:59:11 <sgallagh> Let's defer, yes
17:59:25 <bowlofeggs> though next week may also have a long meeting with the recurrence of modularity since we deferred that too
17:59:29 <bowlofeggs> but i'm still a +1
17:59:40 <maxamillion> +1
18:00:01 <jforbes> +1
18:00:43 <bowlofeggs> #agreed defer all decisions of any kind, including in our personal lives, until next week's meeting (+6, 0, -0)
18:00:52 <bowlofeggs> #topic Next week's chair
18:00:59 <bowlofeggs> volunteers?
18:01:15 <maxamillion> I'll take next week
18:01:27 <bowlofeggs> #action maxamillion will chair next week's meeting
18:01:32 <bowlofeggs> #topic open floor
18:01:45 <jforbes> I will likely be out due to Plumbers, but I will try to vote in tickets before the meeting
18:02:05 <bowlofeggs> excellent, enjoy!
18:02:26 <maxamillion> jforbes: safe travels and have fun!
18:02:34 <bowlofeggs> any open floor topics?
18:03:08 <bowlofeggs> going once
18:03:22 <bowlofeggs> going twice
18:03:34 <bowlofeggs> sold!
18:03:38 <bowlofeggs> #endmeeting