15:00:34 <tyll> #startmeeting FESCO (2018-03-23)
15:00:34 <zodbot> Meeting started Fri Mar 23 15:00:34 2018 UTC.  The chair is tyll. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:34 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:34 <zodbot> The meeting name has been set to 'fesco_(2018-03-23)'
15:00:37 <zbyszek> .hello2
15:00:38 <zodbot> zbyszek: zbyszek 'Zbigniew Jędrzejewski-Szmek' <zbyszek@in.waw.pl>
15:00:38 <bowlofeggs> .hello2
15:00:41 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:01:01 <tyll> :D axamillion dgilmore nirik jsmith sgallagh bowlofeggs tyll jwb zbyszek
15:01:10 <jsmith> .hello2
15:01:11 <zodbot> jsmith: jsmith 'Jared Smith' <jsmith.fedora@gmail.com>
15:01:11 <tyll> #chair maxamillion dgilmore nirik jsmith sgallagh bowlofeggs tyll jwb zbyszek
15:01:11 <zodbot> Current chairs: bowlofeggs dgilmore jsmith jwb maxamillion nirik sgallagh tyll zbyszek
15:01:12 <nirik> morning
15:01:18 <tyll> #meetingname fesco
15:01:18 <zodbot> The meeting name has been set to 'fesco'
15:01:25 <tyll> #topic init process
15:01:26 <sgallagh> .hello2
15:01:27 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
15:01:31 <tyll> .hello till
15:01:32 <zodbot> tyll: till 'Till Maas' <opensource@till.name>
15:02:33 <tyll> I count 6 members, so we have quorum
15:02:52 <jwb> hi
15:03:07 <tyll> now 7 :-)
15:03:12 <jsmith> w00t!
15:03:15 <tyll> 16:01:19 <zodbot> The meeting name has been set to 'fesco' A status (<100% completed)
15:03:19 <tyll> #topic #1861 F28 Changes not in ON_QA status (<100% completed)
15:03:36 <tyll> .fesco 1861
15:03:38 <zodbot> tyll: Issue #1861: F28 Changes not in ON_QA status (<100% completed) - fesco - Pagure - https://pagure.io/fesco/issue/1861
15:03:52 <zbyszek> Can we discuss them one by one?
15:03:54 <tyll> there are only issues from last meeting
15:03:58 <tyll> #topic Kerberos in Python modernization
15:04:02 <tyll> https://fedoraproject.org/wiki/Changes/kerberos-in-python-modernization
15:04:05 <tyll> https://bugzilla.redhat.com/show_bug.cgi?id=1537249
15:04:09 <tyll> zbyszek: sure
15:04:27 <zbyszek> We were supposed to ask the maintainer to update the page, but nothing happened there
15:04:32 <tyll> AGREED: (+1:6,+0:0,-1:0) - ask change owner to update change with what will be shipped in f28 and file any later changes in an F29 change request
15:04:43 <tyll> yes, this was the last item ^
15:04:48 <tyll> last decission
15:04:56 <tyll> does someone volunteer to ask this time?
15:05:01 <zbyszek> I will.
15:05:04 <jsmith> Did someone actually reach out to the change owner?
15:05:14 <jsmith> Thanks zbyszek
15:05:15 <zbyszek> #action zbyszek to reach out to the change owner
15:05:53 <tyll> #info nothing happened since last meeting, will discuss it again next meeting
15:05:59 <tyll> any objections?
15:06:02 <bowlofeggs> +1
15:06:12 <sgallagh> ack
15:06:16 <jsmith> ACK
15:06:23 <nirik> ack
15:06:45 <zbyszek> ack
15:07:20 <tyll> ok, next subitem
15:07:25 <tyll> #topic Add-On Modularity
15:07:29 <tyll> https://fedoraproject.org/wiki/Changes/F28AddonModularity
15:07:32 <tyll> https://bugzilla.redhat.com/show_bug.cgi?id=1537253
15:07:50 <zbyszek> sgallagh?
15:07:52 <bowlofeggs> bodhi is now ready for f28 modularity afaik
15:07:56 <tyll> last meeting: AGREED: The critieria posted in comments 6 and 7 at https://bugzilla.redhat.com/show_bug.cgi?id=1537253 are required for modularity (+8, 0, -0)
15:07:57 <tyll> @sgallagh will inform the QA team
15:07:58 <nirik> I think we are in a state where everything can at least be tested. ;)
15:08:04 <jsmith> From what (little) I understand, the modularity stuff is coming along and pretty much ready to go
15:08:14 <sgallagh> Yes, we're in much better shape
15:09:01 <sgallagh> As far as I know, we are meeting all of the blocking criteria we established
15:09:13 <sgallagh> I'm going to continue to put it through it's paces before Go/No-Go next week
15:09:29 <zbyszek> Cool.
15:09:45 <bowlofeggs> should the ticket be moved to ON_QA then?
15:09:55 <zbyszek> It is already.
15:09:56 <sgallagh> bowlofeggs: it already is
15:10:16 <bowlofeggs> ah then we don't really need to discuss it here
15:10:27 <bowlofeggs> since the fesco ticket is about tickets not on ON_QA :)
15:10:31 <zbyszek> Yeah, seems to be on track for beta.
15:11:33 <jsmith> +1 to moving on
15:12:09 <tyll> ok, next item:
15:12:11 <tyll> #topic Replace glibc's libcrypt with libxcrypt
15:12:16 <tyll> https://fedoraproject.org/wiki/Changes/Replace_glibc_libcrypt_with_libxcrypt
15:12:21 <tyll> https://bugzilla.redhat.com/show_bug.cgi?id=1537261
15:12:30 <zbyszek> We're still waiting for ppp to be updated.
15:12:36 <zbyszek> .whoowns2 ppp
15:12:37 <zodbot> zbyszek: User: msekleta, Name: Michal Sekletar, email: msekleta@redhat.com, Creation: 2011-08-02, IRC Nick: msekleta, Timezone: Europe/Prague, Locale: en, GPG key ID: , Status: active
15:12:40 <zodbot> zbyszek: Approved Groups: @gitchkconfig qa-beaker-user qa-automation-shell packager fedorabugs cla_fpca cla_done gitinitscripts
15:13:02 * sgallagh is kind of sad that PPP is still a thing in 2018
15:13:16 <bowlofeggs> this one is also ON_QA
15:13:17 <tyll> info from last meeting:  Appears to be mostly done.
15:13:25 <nirik> well, since it's broken... perhaps not. ;)
15:14:26 <bowlofeggs> true
15:14:41 <bowlofeggs> hmm, it's been ~1 week since this BZ was filed
15:14:42 <tyll> isn't it still needed for 3G/4G data cards?
15:14:47 <bowlofeggs> and porting code might not be quick
15:15:06 <sgallagh> PPP is not a blocking package, so far as I know. I'm okay with letting the maintainers sort this out.
15:15:14 <sgallagh> Not worth trying to revert this change over it
15:15:35 <bowlofeggs> don't some VPNs use it too?
15:15:36 <msekleta> hi!
15:15:54 <bowlofeggs> msekleta: greetings! what can you tell us about ppp?
15:16:00 <nirik> or ppoe
15:16:29 <zbyszek> #link https://bugzilla.redhat.com/show_bug.cgi?id=1556132
15:17:38 <msekleta> Honestly, I don't do much on ppp these days
15:18:04 <msekleta> RHEL/CentOS maint. is Jaroslav Skarvada and we agreed that he will be taking care of Fedora as well
15:18:36 <bowlofeggs> msekleta: can you tell us whether ppp is used for things that might be considered important?
15:19:06 <maxamillion> .hello2
15:19:09 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
15:19:12 <maxamillion> sorry I'm late, had physical therapy
15:19:35 <bowlofeggs> now i don't get to +2
15:20:01 <tyll> maxamillion: you did not miss much
15:20:29 <msekleta> bowlofeggs, there are use cases for it, but it is becoming less and less important over time
15:20:42 <msekleta> bowlofeggs, not sure how can I quantify that
15:21:06 <bowlofeggs> cool. so you might not consider it to be a blocker if it doesn't make it into f28?
15:21:24 <maxamillion> bowlofeggs: :)
15:21:42 <maxamillion> bowlofeggs:  you just keep doing +2's and I'll yell if I disagree ... could be an interesting social experiment ;)
15:22:35 <zbyszek> $ dnf repoquery --whatrequires ppp --alldeps --qf '%{name}'
15:22:37 <zbyszek> NetworkManager-fortisslvpn
15:22:37 <zbyszek> NetworkManager-l2tp
15:22:37 <zbyszek> NetworkManager-ppp
15:22:38 <zbyszek> NetworkManager-pptp
15:22:40 <zbyszek> NetworkManager-sstp
15:22:42 <zbyszek> glpi
15:22:45 <zbyszek> iipsrv
15:22:47 * dgilmore is here now
15:22:47 <zbyszek> kppp
15:22:50 <zbyszek> openfortivpn
15:22:52 <zbyszek> php-horde-horde
15:22:55 <zbyszek> ppp-devel
15:22:57 <zbyszek> pptp
15:23:00 <zbyszek> pptpd
15:23:02 <zbyszek> rp-pppoe
15:23:05 <zbyszek> sstp-client
15:23:07 <zbyszek> synce-connector
15:23:10 <zbyszek> wvdial
15:23:12 <zbyszek> xl2tpd
15:23:15 <zbyszek> wow, full house
15:23:30 <zbyszek> It looks like at least upgrade from F27 will be broken if ppp is not upgradeable, because of the dependency in NetworkManager.
15:23:38 <zbyszek> It looks like we _need_ to fix this.
15:23:43 <jsmith> Yeah :-/
15:24:00 <bowlofeggs> and those packages might be dependencies for others, and so on
15:24:25 <dgilmore> it is a core piece of most systems
15:24:39 <zbyszek> msekleta: can you reach out to Jaroslav and look at #1556132? If it's not fixable, we'll need to find some work-around.
15:25:25 <bowlofeggs> could we ask the change maintainer to bring libcrypt back while *also* having libxcrypt? i.e., are they mutually exclusive?
15:25:33 <bowlofeggs> and then they can try to get rid of libcrypt for f29?
15:25:49 <zbyszek> I think they are mutually exclusive.
15:25:53 <nirik> I think that would be a massive amount of work...
15:25:58 <zbyszek> Also.
15:26:17 <bowlofeggs> ok
15:26:18 <msekleta> zbyszek, yes, I will talk to him
15:26:31 <zbyszek> Thanks.
15:26:35 <bowlofeggs> in that case, shoudl fesco declare this a blocker?
15:26:46 <bowlofeggs> and if so, a beta or final blocker?
15:26:56 <bowlofeggs> seems like a lot of stuff depends on it, so beta might be appropriate
15:27:17 <bowlofeggs> does anybody have a sense of how much work we are asking for from the ppp maintainer?
15:28:08 <tyll> the bug report only mentions DES so it seems to be one encryption routine
15:28:35 <bowlofeggs> ok so maybe not so bad
15:29:25 * jsmith is incline to call it a blocker
15:29:29 <jsmith> inclined, that is
15:29:31 <bowlofeggs> i lean towards beta blocker, but that also creates a high likelyhood that the beta won't be ready next week
15:30:03 <nirik> we may be able to ask the change owner to look too? perhaps they can help port.
15:30:20 <sgallagh> I'd prefer not to block *beta* on this
15:30:43 <sgallagh> It hasn't caused any of our usual tests to fail.
15:30:44 <maxamillion> bowlofeggs: agreed
15:30:50 <zbyszek> sgallagh: +1
15:31:01 <sgallagh> *some* version of PPP is available on F28; I don't know for sure if it works though
15:31:06 <sgallagh> ppp-2.4.7-14.fc28.x86_64
15:31:09 <bowlofeggs> i wouldn't oppose it being a final non-beta blocker, but i would prefer beta blocker
15:31:24 <nirik> yeah, that one says it's ported to openssl 1.1
15:31:34 <nirik> so I am not sure what broke after that
15:31:51 <zbyszek> It seems that it's just a few lines in pppd/pppcrypt.c, so somebody who knows the code should be able to fix it relatively quickly.
15:31:55 <bowlofeggs> i would expect that version to have some broken linking
15:32:05 <bowlofeggs> since it was built on the old lib
15:32:38 <zbyszek> Oh, no, it's installable, but just FTBFS.
15:32:50 <bowlofeggs> installable doesn't mean it functions though
15:33:04 <bowlofeggs> if it references libcrypt code that doesn't exist
15:33:07 <zbyszek> No, the Change was backwards compatible in this way
15:33:14 <bowlofeggs> oh?
15:33:53 <bowlofeggs> i'm surprised that it would be runtime backwards compatible without being build time backwards compatible
15:33:56 <zbyszek> "On Linux-based systems, by default libxcrypt will be binary backward compatible with the libcrypt.so.1 shipped as part of the GNU C Library. This means that all existing binary executables linked against glibc's libcrypt should work unmodified with this library's libcrypt.so.1"
15:34:23 <tyll> FYI: we are discussing this item now for 20 minutes
15:34:34 <bowlofeggs> ah interesting
15:34:36 <bowlofeggs> yeah i see that now
15:34:41 <jsmith> Proposal: Make this a beta blocker
15:34:47 <sgallagh> Proposal: Ask QA to test whether PPP functionality has regressed in F28
15:35:01 <zbyszek> jsmith: -1, no need.
15:35:06 <jsmith> sgallagh: Is QA equipped to test this?
15:35:09 <sgallagh> adamw: ^^
15:35:13 <sgallagh> kparal: ^^
15:35:35 <bowlofeggs> how about we ask QA to tell us if they think this should be a beta blocker?
15:35:47 <maxamillion> bowlofeggs: +1
15:35:50 <tyll> bowlofeggs: +1
15:36:01 <zbyszek> I think that chances of ppp regressing are very low. It's only a matter of not being able to rebuild the package if needed.
15:36:02 <nirik> well, what critera does it meet?
15:36:03 <sgallagh> bowlofeggs: Then propose it for blocker consideration
15:36:14 <sgallagh> There's a whole group of people that discusses things like that.
15:36:20 <bowlofeggs> sure
15:36:23 * kparal doesn't know what PPP is
15:36:32 <bowlofeggs> i think that's a good way to handle this one
15:36:35 <nirik> If it's not fixed by final I think that might be cause for concern.
15:36:37 <sgallagh> nirik: Well, if it truly breaks dial-up PPP or PPPoE (DSL), then it will fail update criteria
15:36:41 <bowlofeggs> not being able to update it could be bad if there are CVEs
15:36:41 <dgilmore> kparal: its the software to make dialup connections
15:36:42 <nirik> sgallagh: sure.
15:36:52 <kparal> like, a modem? from 1990's?
15:36:56 <dgilmore> kparal: used by pppoe for ADSL
15:36:56 <nirik> sgallagh: well, it doesn't break updates...
15:36:59 <sgallagh> kparal: Or DSL
15:37:07 <bowlofeggs> DSL is still widely used
15:37:11 <kparal> we don't have any way to test that in our brno office
15:37:13 <sgallagh> nirik: The update criteria is a stand-in for network tests
15:37:21 <dgilmore> kparal: also used over bluetooth to mobile phones for tethering in some cases
15:37:22 <nirik> hum?
15:37:33 <dgilmore> kparal: my fiber connection uses it
15:37:36 <kparal> dgilmore: that could be tested, I guess
15:37:39 <kparal> over the phone
15:37:44 <sgallagh> nirik: It tests whether the system can reach the repos, download from them and apply updates
15:37:55 <sgallagh> If PPP is truly broken, network updates are impossible
15:38:12 <bowlofeggs> proposal: we submit this to be considered for blocker for monday's blocker meeting and let them decide
15:38:19 <sgallagh> bowlofeggs: +1
15:38:22 <dgilmore> sgallagh: in cases where you rely on using ppp to make a connection to the internet
15:38:22 <jsmith> bowlofeggs: +1
15:38:30 <kparal> bowlofeggs: +1
15:38:33 <dgilmore> +1
15:38:39 <sgallagh> dgilmore: Yes, I was referring to DSL/Dial-up
15:38:50 <dgilmore> sgallagh: being explict is better here
15:38:50 <nirik> sgallagh: I don't think that was intended to mean "any way to connect via network"
15:38:58 <zbyszek> -1, I think it's creating unnecessary work, since we have all the data we need to make a decision now.
15:38:59 <sgallagh> dgilmore: Agreed
15:39:02 <tyll> bowlofeggs: +1
15:39:51 <zbyszek> I'd just make this a final blocker now instead.
15:39:52 <bowlofeggs> i would also still +1 us saying it is a blocker here
15:39:52 <nirik> sure, +1 to propose it and get regular blocker votes.
15:40:06 <nirik> zbyszek: yeah, I can see the case for that
15:40:17 <sgallagh> I'm -1 as a Beta blocker without knowing whether it ACTUALLY impacts our users
15:40:27 <tyll> zbyszek: but do we know that it breaks pppoe as it is now?
15:40:35 <bowlofeggs> we don't know what is broken now
15:40:41 <bowlofeggs> other than building
15:41:02 * sgallagh will try to find a neighbor on DSL and test
15:41:03 <zbyszek> Well, if pppoe is broken (or anything else) that'd mean the whole change is busted. There has been no indication of that.
15:41:03 <tyll> therefore IMHO we should first figure out if it is still working
15:41:49 <bowlofeggs> filing it as a blocker proposal will allow QA to consider it and i think that's the best way to go
15:42:07 <dgilmore> I think we need to get someone to test it
15:42:22 <bowlofeggs> i think it def needs to build by final release, but QA should determine if there's a problem for beta or not imo
15:42:22 <dgilmore> kinda silly to file a bug saying maybe there is a bug here, we are not sure
15:43:00 <bowlofeggs> true, but are any of us equipped to test it? if so, sure
15:43:07 <bowlofeggs> i don't have a ppp connection to use to test
15:43:24 <bowlofeggs> and there are several types of such connections
15:43:27 <tyll> I can try with DSL
15:43:44 <dgilmore> I can possibly try
15:43:51 <bowlofeggs> i think looping in QA would be wise even if we don't know of a bug other than building
15:44:05 <bowlofeggs> they might just say "meh" and that's fine, but i think communicating it to them would be wise
15:44:08 <tyll> note sure if I have a SIM card to use for a 3G card
15:44:14 <dgilmore> when i switched to gigabit my connection went from pppoe to static
15:44:14 <sgallagh> Hmm, seems that all of my neighbors have switched to Fiber. Smart of them :-D
15:44:28 <dgilmore> I am not sure if using pppoe will work or not
15:44:42 <sgallagh> dgilmore: Do you happen to know which phones use PPPoE for tethering?
15:44:59 <dgilmore> sgallagh: No.
15:45:18 <dgilmore> sgallagh: a 3g data modem is typically using ppp also
15:45:26 <sgallagh> hmm
15:45:59 <bowlofeggs> we are agreed that this is at least a final blocker right?
15:46:12 <bowlofeggs> because you can't update it if you can't build it
15:46:19 <zbyszek> yes
15:46:43 <nirik> yeah.
15:46:48 <sgallagh> bowlofeggs: +1
15:47:02 <bowlofeggs> we do have enough votes (though non-unaimous) to just let the beta blocker meeting decide whether it's beta also or not
15:47:07 <dgilmore> sgallagh: I have an old laptop that I think I have a 3g modem in, and I have a t-mobile data only sim
15:47:13 <bowlofeggs> i say we declare it final blocker and let them decide if it's also beta blocker
15:47:14 <dgilmore> I may be able to test
15:47:19 <sgallagh> dgilmore: Thanks
15:47:27 <sgallagh> I just tried with my phone, but it's not PPPoE, apparently
15:47:41 <msekleta> Just, FYI, you don't really need to test this with phones or anything. pppoe can by easily tested with two VMs on a same ethernet network.
15:47:41 <bowlofeggs> we've also been on this topic a real long time
15:47:45 <tyll> propsal: Make it a final blocker and everyone tries to test it until Monday to see if it breaks PPPOE/3G devices?
15:47:48 <dgilmore> sgallagh: using bluetooth?
15:47:49 <bowlofeggs> we dont' haev to be unanimous all the time - it's ok
15:47:51 <sgallagh> dgilmore: Yes
15:47:58 <zbyszek> tyll: +1
15:48:02 <dgilmore> sgallagh: ppp over bluetooth may be old at this point
15:48:10 * sgallagh nods
15:48:21 <dgilmore> +1 tyll
15:48:30 <sgallagh> tyll: +1
15:48:32 <tyll> msekleta: can you test this with two VMs and report back?
15:48:33 <bowlofeggs> i still think we need to at least also bring it to QAs attention
15:48:45 <msekleta> tyll, sure
15:48:54 <tyll> msekleta: awesome, thank you
15:49:11 <msekleta> tyll, when do you what me to report on ppp status?
15:49:14 <sgallagh> I'll ping the Westford office and see if anyone has a 3G modem handy
15:49:25 <tyll> bowlofeggs: would you notify QA?
15:49:29 <bowlofeggs> tyll: sure
15:49:38 <tyll> msekleta: before the blocker meeting on Monday would be great
15:49:51 <msekleta> tyll, ok, I will test it over weekend
15:49:53 <tyll> #action bowlofeggs will notify QA that ppp might be a beta blocker
15:49:54 <bowlofeggs> #action bowlofeggs will notify QA about https://bugzilla.redhat.com/show_bug.cgi?id=1556132
15:49:57 <bowlofeggs> haha
15:50:08 <tyll> #action msekleta will test pppoe with a VM setup
15:50:14 <tyll> bowlofeggs: double action :-D
15:50:44 <bowlofeggs> +1 to the current proposal
15:51:22 <tyll> nirik: ? maxamillion ? jsmith ? jwb ?
15:51:29 <nb> +1
15:51:40 <jsmith> +1
15:51:45 <nirik> +1
15:52:01 <maxamillion> +1
15:52:35 <bowlofeggs> note that we also agreed that this is a final blocker (though without a formal vote?) so should we go ahead and mark it as such?
15:52:57 <tyll> AGREED make it a final blocker and everyone will try to test it to see if it breaks PPPoE/3g devices
15:53:07 <tyll> AGREED make it a final blocker and everyone will try to test it to see if it breaks PPPoE/3g devices (+7, 0, -0)
15:53:45 <sgallagh> tyll: missing leading #
15:54:03 <tyll> #agree make it a final blocker and everyone will try to test it to see if it breaks PPPoE/3g devices (+7, 0, -0)
15:54:07 <tyll> sgallagh: thx
15:54:41 <tyll> bowlofeggs: will you mark it as a final blocker?
15:57:35 <bowlofeggs> #action bowlofeggs will mark it as a final blocker
15:57:44 <tyll> perfect, bowlofeggs++
15:57:44 <zodbot> tyll: Karma for bowlofeggs changed to 15 (for the f27 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:57:54 <tyll> #topic Atomic, Cloud and Container images for s390x
15:58:01 <tyll> https://fedoraproject.org/wiki/Changes/Atomic_Cloud_and_Docker_images_for_s390x
15:58:06 <tyll> https://bugzilla.redhat.com/show_bug.cgi?id=1547235
15:58:06 <zbyszek> This was recently updated.
15:58:18 <zbyszek> "Now, We have some successful cloud-base image built in F28 nightly composes which can be tested"
15:58:28 <zbyszek> "Change wiki page has been updated to only include cloud-base and container images for s390x in F28"
15:58:32 <zbyszek> I think we can move on.
15:58:33 <nirik> I think the only thing missing is atomic/ostree... which is just too slow right now.
15:58:35 <nirik> yep.
15:59:14 <bowlofeggs> yeah let's move on
15:59:52 <jsmith> +1 to move on
16:00:00 <tyll> so do we need to change the bug to ON_QA?
16:01:06 <zbyszek> I changed it to ON_QA now.
16:01:46 <tyll> zbyszek: thank you
16:02:15 <tyll> #info change can be tested now
16:02:21 <tyll> #topic #1845 389-ds-base and freeipa on 32 bit arches
16:02:28 <tyll> .fesco 1845
16:02:30 <zodbot> tyll: Issue #1845: 389-ds-base and freeipa on 32 bit arches - fesco - Pagure - https://pagure.io/fesco/issue/1845
16:02:36 <tyll> https://pagure.io/fesco/issue/1845
16:03:22 <tyll> last meeting's decision: AGREED: FESCo requires that a Change be filed for F28, and involved packagers are otherwise free to sort out the issue (+6, 0, -0)
16:04:36 <nirik> so they pushed an update that dropped i686...
16:05:59 <nirik> this issue makes my teeth itch. They discovered testing that shows things don't work on i686, so they drop it... even with tools folks there wanting to track it down and fix it.
16:06:24 <maxamillion> I'm not a big fan of how that all went down eithehr
16:06:27 <maxamillion> either*
16:06:48 <bowlofeggs> lol @ teeth itch
16:06:55 <nirik> which may mean there's a tools bug that (silently) breaks other i686 stuff
16:07:12 <bowlofeggs> i'm also annoyed that they didn't file the change as requested
16:07:59 <bowlofeggs> when do we step in and say that the maintainer(s) is(are) acting irresponsibly?
16:08:12 <tyll> #info bowlofeggs requested the change proposal in https://bugzilla.redhat.com/show_bug.cgi?id=1530832#c8
16:08:22 <bowlofeggs> i'm not saying we've reached that point, but the question comes to mind
16:10:51 <tyll> how do we move on here? Would someone be willing to reach out to the maintainers to understand what is going on?
16:10:56 <sgallagh> bowlofeggs What would we do in that case? Take over the package?
16:11:01 <nirik> well, there's not much we can really do in the end... we can say no, or reassign packages, but ..
16:11:06 <sgallagh> I'm not about to attempt that with such a complicated project
16:11:28 <sgallagh> Our only recourse would be "block the release until they fix it"
16:11:34 <sgallagh> Which... hurts us more than them
16:11:39 <bowlofeggs> right
16:12:02 <bowlofeggs> tyll: i feel like i've bene trying to reach out to them wtihout a lot of success
16:12:26 <maxamillion> bowlofeggs: is there any history of that transaction that we can view? BZs, email, $other?
16:12:27 <bowlofeggs> we def don't have anyone banging on the door to maintain it
16:12:37 <bowlofeggs> maxamillion: just my comments on the BZs
16:12:52 <zbyszek> maxamillion: https://bugzilla.redhat.com/show_bug.cgi?id=1530832#c8 and https://bugzilla.redhat.com/show_bug.cgi?id=1530832#c9
16:12:58 <puiterwijk> bowlofeggs: well, we do have people willing to help look at the i686 problems (tools team) as for what I've heard...
16:13:01 <bowlofeggs> i haven't tried e-mailing them directly - perhaps that would be good
16:13:22 <bowlofeggs> puiterwijk: true. we could still kick this to the 686 sig
16:13:51 <maxamillion> zbyszek: no, I mean showing that someone was trying to reach out for help on the issue but received no feedback
16:14:02 <puiterwijk> bowlofeggs: I'm not sure the i686 sig is totally fair, but at least allowing the tools team to work on it might be possible? (no idea how it went down honestly. Not read anytime recently)
16:14:49 <nirik> maxamillion: well, glibc/gcc maintainers replied in the bug asking for reproducers or what they were seeing and... never really got details they could find the issue with
16:14:56 <zbyszek> maxamillion: that whole bug is about the tools team trying to help them diagnose the issue.
16:14:57 <bowlofeggs> maxamillion: i wouldn't say i got no feedback, but we also didn't get the change filed that we requested
16:15:22 <bowlofeggs> and yeah, they haven't really been working together with the tools team either by my estimation
16:15:32 <bowlofeggs> the tools team did seem pretty responsive i'd say
16:15:41 <maxamillion> bowlofeggs: right, I just wanted to make sure we weren't unfairly pointing fingers
16:16:09 <bowlofeggs> sure
16:17:01 <maxamillion> bowlofeggs: also wasn't sure if I was missing something
16:18:17 <nirik> I'm really not sure what we can do here.
16:18:22 <bowlofeggs> proposal: FESCo requires either a change for this, or 686 to be restored, as a final blocker
16:18:44 <zbyszek> bowlofeggs: +1
16:18:56 <nirik> perhaps we could go more lightweight than a change? release-note?
16:19:01 <bowlofeggs> but can we let them restore it at final rather than beta? then it'd miss out on beta testing...
16:19:08 <puiterwijk> I'm not sure that that hurts them a lot. As someone else just said, that hurts us more than them probably..
16:19:19 <bowlofeggs> well we don't need to hurt them
16:19:23 <bowlofeggs> we just need it to happen
16:19:31 <bowlofeggs> nirik: sure i could be fine with release note
16:19:40 <bowlofeggs> mostly we need to make sure the change is communicated
16:20:15 <bowlofeggs> i can also write comments on both BZs that just say "File a change for this"
16:20:18 <nirik> they might go for that more than a change perhaps...
16:20:26 <puiterwijk> Also, sure, it'll miss some testing, but I don't think anything i686 is blocking, so even if we required at beta, it might still not work, nad they probably won't prioritize it.
16:20:29 <bowlofeggs> or "release note"
16:20:35 <zbyszek> Yes, but a change at this level requires a Change page.
16:20:39 <bowlofeggs> puiterwijk: true
16:20:51 <zbyszek> It's not like writing three paras of text is such a big deal.
16:21:15 <sgallagh> zbyszek: Clearly, as we have just written several books worth on this topic...
16:21:18 <bowlofeggs> hell, can *i* write the change just to get it done?
16:21:19 <puiterwijk> bowlofeggs: I would personally not suggest doing anything around Beta with it. But I do think writing a change page should not be a problem
16:21:23 <bowlofeggs> even though i'm not the maintainer?
16:21:49 <sgallagh> bowlofeggs: If you want to write the Change and get them to sign off on it, I'd be good with that
16:22:13 <bowlofeggs> i don't mind it, though i'm not sure i'll get a sign off response
16:22:14 <nirik> sure, that would be fine.
16:22:24 <bowlofeggs> i just want ot see us stop discussing this one haha
16:22:50 <tyll> thank you very much bowlofeggs
16:23:01 <bowlofeggs> i'll write a comment to ask for it. if they dont' respnd in a week i'll write the chnage. no need for fesco to re-discuss this next week unless the change is tehre
16:23:23 <nirik> +1
16:23:25 <bowlofeggs> #action bowlofeggs will make sure a change is written NO MATTER WHAT HAPPENS
16:23:36 <tyll> #action bowlofeggs will ask them again for a change page and write one for himself if the maintainers do not provide one
16:23:48 <tyll> uh, double action again :-)
16:23:53 <maxamillion> bowlofeggs++
16:23:53 <zodbot> maxamillion: Karma for bowlofeggs changed to 16 (for the f27 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:23:58 <jsmith> bowlofeggs++
16:23:59 <zbyszek> bowlofeggs: now you're responsible for keeping the wiki running too
16:24:00 <sgallagh> tyll: Better than inaction!
16:24:04 <puiterwijk> tyll: bowlofeggs gets double the work :)
16:24:19 <tyll> ok, next neverending story
16:24:21 <tyll> #topic #1820 Adjust/Drop/Document batched updates policy
16:24:21 <puiterwijk> zbyszek: ooooh, that's a good one! Thanks for bring that on bowlofeggs!
16:24:34 <tyll> .fesco 1820
16:24:37 <tyll> https://pagure.io/fesco/issue/1820
16:24:38 <zodbot> tyll: Issue #1820: Adjust/Drop/Document batched updates policy - fesco - Pagure - https://pagure.io/fesco/issue/1820
16:24:55 <jsmith> Didn't see much discussion on this... other than complaints about the tooling
16:25:00 <bowlofeggs> i love the cookies
16:25:12 <bowlofeggs> yeah i don't know why i engaged so much in the discussion
16:25:38 <zbyszek> I'd be inclined to wait more, until the zchunk proposal matures a bit.
16:25:39 <jsmith> If kalev presented his ideas,  I missed them :-/
16:25:48 <jsmith> zbyszek: I'm fine with that :-)
16:26:10 <jsmith> I don't think there's any rush here...
16:26:13 <zbyszek> jsmith: https://pagure.io/fesco/issue/1820#comment-499724
16:26:18 <sgallagh> I don't really have much more to say than I've said in this meeting in the past: the current state is badly broken; either batched must be enforced or must be scrapped.
16:26:37 <sgallagh> I'd still like to see those UX improvements added and batching then enforced.
16:27:00 <jsmith> Did we ever get any answer back from releng regarding the possibilities of having another "firehose-updates" repo?
16:27:08 <zbyszek> Right, but I think anything will happen while we're trying to get F28 out of the door.
16:27:17 <jsmith> I seemed to remember some concerns regarding mirror storage increases, etc.
16:27:41 <jsmith> sgallagh: In that case, I'm inclined to "scrap" it in the short term
16:27:43 <nirik> well, theres push times, storage increases, etc.
16:27:56 <bowlofeggs> i haven't seen anyone strongly opposed to a firehose repo, though a "mild" opposition was voiced around the cost/benefit ratio
16:28:02 <nirik> but IMHO the problem is that 10 people will use it and it will be a waste of all those things.
16:28:07 <nirik> but I suppose we could try it and see
16:28:18 <zbyszek> There was also no feedback from QA to bowlofeggs' question
16:28:41 <bowlofeggs> QA might have been extremely busy this week since it was go/no-go week
16:28:45 <nirik> did we talk to qa?
16:28:48 <bowlofeggs> and probably next week too
16:28:52 <bowlofeggs> nirik: i posted on their list
16:29:06 <bowlofeggs> https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/thread/6CWTD5LGOPST5NP3YRNXOKNY5NEF7WR6/
16:29:28 <nirik> ok, cool. yes, likely this week and next are bad
16:29:42 <tyll> should we maybe postpone this until after the beta release?
16:29:52 <jsmith> I'm going to go out on a limb here....
16:29:52 <nirik> I'd be fine with that
16:29:56 <zbyszek> Me too.
16:30:10 <sgallagh> I think I agree with jsmith though; let's suspend batched until we have a plan in place.
16:30:11 <nirik> or perhaps it would make a good flock session... brainstorm updates process.
16:30:13 <jsmith> Proposal: Disable batching until we have a better plan in place (likely waiting on zchunk discussion
16:30:13 <bowlofeggs> yeah postponing is reasonable, though that's what we do every week
16:30:22 <bowlofeggs> sgallagh: suspending batched would be an extreme engineering effort
16:30:24 <sgallagh> The current operation is unpredictable
16:30:31 <sgallagh> bowlofeggs: Why is that?
16:30:36 <bowlofeggs> sgallagh: it would be just as much work as enforcing it
16:30:52 <bowlofeggs> sgallagh: it was not a small patch, and bodhi doesn't have a nice neat "state machine" - the code is littered all over the place
16:30:59 <sgallagh> bowlofeggs: You couldn't just hack up Bodhi to auto-submit for stable if it's submitted for batched?
16:31:23 <bowlofeggs> sgallagh: not without jsut as many bugs as we had when it was implemented - bodhi's code around its state machine is not clean at all
16:31:29 <sgallagh> ok
16:31:30 <nirik> or I suppose as a bandaid we could run the script that promotes right before pushes.
16:31:40 <bowlofeggs> nirik: yeah that could be accomplished more easily
16:31:45 <bowlofeggs> nirik: or run it daily
16:31:54 <bowlofeggs> via cron
16:31:59 <sgallagh> Yeah, that's more what I meant by "suspend batched"
16:32:11 <sgallagh> From the user perspective rather than eng/ops
16:32:26 <bowlofeggs> i don't see the zchunk as being a blocker for this though - i think it's orthogonal (and good)
16:32:43 <nirik> yeah, there's several things mixed in here.
16:32:53 <sgallagh> I still feel that batched is worthwhile if it's enforced, but I think enforcing it needs to wait on UX improvements
16:33:00 <nirik> but after all the discussion I think I agree that batching would be best on the client.
16:33:05 <bowlofeggs> sgallagh: i agree
16:33:16 <nirik> (but not sure how best to do that)
16:33:35 <bowlofeggs> the drpm generation problem can't be solved client side
16:33:45 <bowlofeggs> and i think that matters much more than the metadata download
16:34:02 <bowlofeggs> in my estimation, it causes more bytes
16:34:10 <bowlofeggs> especially if enforced batching is in place
16:34:35 <nirik> well, you get the same drpm effect if you batch on the server or on the client end no?
16:35:10 <bowlofeggs> nirik: no - a higher percentage of downloaded rpms are drpms for packages that rev frequently
16:35:25 <bowlofeggs> bodhi only generates drpms from one RPM to the very next one
16:35:28 * jsmith still thinks that without someone leading the charge, we're still just discussing this around in circles without any clear direction
16:35:38 <bowlofeggs> so if a client misses any in between, they download the whole RPM again
16:36:00 <zbyszek> bowlofeggs: bodhi could be changed to generate rpms in a different way
16:36:01 <bowlofeggs> we could also generate more DRPMs, but that's actually a big part of what makes bodhi composes slow today so it would go even slower
16:36:06 <nirik> I guess. That was not mentioned really as a goal here. ;)
16:36:25 <bowlofeggs> nirik: it was an unanticipated side effect i noted the week we did the experiment
16:36:35 <bowlofeggs> i wrote about it a bit in the ticket
16:36:45 <nirik> yes, I know.
16:37:31 <nirik> and perhaps it should be a goal... just that when we started total download size was not a goal.
16:37:48 <bowlofeggs> true
16:38:20 <tyll> another option would be to keep old drpms around a little bit longer, then users might have to download several drpms but they could still be less than the whole RPM
16:38:54 <jsmith> tyll: I'm not sure the logic in DNF knows to apply multiple deltas to get the end result
16:39:21 <nirik> it doesn't I am pretty sure.
16:39:44 <nirik> it only works from A to B if there's a drpm from A to B and B is the new version its' trying to update to
16:39:55 <tyll> maybe this could be added to dnf then
16:39:56 <bowlofeggs> perhaps we could get dnf to do that and that would solve it
16:40:08 <nirik> well, drpms are a tradeoff too
16:40:13 <bowlofeggs> but would would also need pungi to do it
16:40:22 <nirik> download size vs cpu and disk space.
16:40:26 <bowlofeggs> (it being "keep n old drpms"
16:40:30 <bowlofeggs> )
16:40:30 <puiterwijk> And it would bump up the storage requirements on both our infra and mirrors
16:40:36 <puiterwijk> Since currently we just drop old drpms
16:40:37 <bowlofeggs> yeah
16:40:56 <jsmith> Revised Proposal: Ask bodhi developers to try to "suspend" or "bypass" batching in the short term, continue discussion
16:41:02 <bowlofeggs> i still think that a high percent of updates just aren't important too
16:41:06 <bowlofeggs> even of my own updates, haha
16:41:23 <tyll> jsmith: +1
16:41:26 <zbyszek> jsmith: +1
16:41:31 <tyll> it has been too long for this discussion again
16:41:35 <nirik> +1
16:41:37 <bowlofeggs> jsmith: +0 - i don't think it's harmful to leave it as is, but am not opposed
16:42:05 <tyll> maybe we can also find someone to lead this and write up a collection of arguments and ideas
16:42:23 <zbyszek> tyll: I did that last time, I'd be happy to an update
16:42:35 <tyll> zbyszek: awesome, that would be great
16:42:46 <nirik> cool
16:42:58 <zbyszek> unless we want somebody else, to have a fresh perspective
16:43:23 <sgallagh> jsmith: +1
16:44:15 <tyll> zbyszek: did you do this in the comments? Maybe we can try a wiki page
16:44:33 <zbyszek> In a mail to fedora-devel
16:44:59 <zbyszek> I can make a wiki page
16:45:07 <tyll> zbyszek: wonderful
16:45:25 <jsmith> zbyszek: I'll volunteer to help review the wiki page for you :-)
16:45:26 <zbyszek> #action zbyszek to generate a wiki page with arguments and considerations, ask for feedback
16:45:29 <tyll> sgallagh: maxamillion jsmith dgilmore ?
16:45:38 <jsmith> tyll: I'm +1 to my own proposal :-)
16:45:42 <tyll> sgallagh: sorry, missed you
16:45:48 <sgallagh> tyll: I was about to say
16:45:49 <bowlofeggs> wiki is nice because we can review it before sending it to the mases
16:45:49 <tyll> jsmith: and I meant jwb
16:45:56 <bowlofeggs> to make sure all perspectives feel well represented
16:46:06 <puiterwijk> bowlofeggs: and you can edit it after sending :)
16:46:22 <bowlofeggs> well you can send to a specific commit :)
16:46:35 <bowlofeggs> but yeah :)
16:46:42 <zbyszek> puiterwijk: and you can edit it back if somebody edits it the wrong way ;)
16:46:50 <bowlofeggs> e-mail is harder because then people reply with "but you forgot A,B,C" :)
16:47:09 <tyll> #agree Ask bodhi developers to try to "suspend"  or "bypass" batching in the short term, continue discussion (+5, 1, -0)
16:47:09 <puiterwijk> zbyszek: right. I just meant that if you later realize more points for/against, you can add them :)
16:47:17 <puiterwijk> i.e. they don't get lost in the middle of an email thread
16:47:32 <tyll> #topic #1776 F28 System Wide Change: Deprecate TCP wrappers
16:47:37 <tyll> .fesco 1776
16:47:39 <zodbot> tyll: Issue #1776: F28 System Wide Change: Deprecate TCP wrappers - fesco - Pagure - https://pagure.io/fesco/issue/1776
16:47:39 <tyll> https://pagure.io/fesco/issue/1776
16:47:41 * puiterwijk was not considering malicious intent...
16:47:57 <zbyszek> puiterwijk: not malicious, just a guiding hand...
16:48:08 <puiterwijk> Heh
16:49:16 <zbyszek> Re 1776: I looked into this, and it seems that most of those packages *could* be easily edited and rebuilt using provenpackage privs and the pull requests that have been submitted. If we are fine with doing that while in beta freeze, I can do it.
16:49:38 <sgallagh> zbyszek: The only one of those that really concerned me was slapi-nis (needed by FreeIPA, a blocking package) and that one has been fixed.
16:49:51 <bowlofeggs> there seems to be recent discussion on https://bugzilla.redhat.com/show_bug.cgi?id=1495181
16:51:16 <zbyszek> bowlofeggs: not much movement really. Just confirmation that stuff is still blocked.
16:51:26 <tyll> zbyszek: great, I can merge PRs and issues rebuilds if we agree with it
16:52:55 <zbyszek> proposal: FESCo authorized use of pp privileges to apply patches and do rebuilds for those packages where the change is stalled
16:53:13 <zbyszek> *authorizes
16:53:21 <sgallagh> zbyszek: +1
16:53:25 <bowlofeggs> zbyszek: +1
16:53:26 <tyll> zbyszek: are there PRs or patches in Bugzilla?
16:53:32 <nirik> +1... and keep it for f28. We have been kicking this can down the road too long. ;)
16:54:02 <tyll> zbyszek: +1
16:54:21 <zbyszek> tyll: there were some PRs for some of the packages in src.fp.o, and some were linked from BZs
16:54:35 <zbyszek> (when I looked at this yesterday)
16:55:06 <tyll> jsmith: maxamillion jwb dgilmore ?
16:55:16 <jsmith> +1
16:55:33 * jsmith has to run.... sorry!
16:55:53 <maxamillion> +1
16:56:06 <tyll> #agree FESCo authorizes use of pp privileges to apply patches and do rebuils for those packages where the change is stalled (+7, 0, -0)
16:56:16 <tyll> #topic Next week's chair
16:56:24 <tyll> #undo
16:56:24 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x4a1b8590>
16:56:30 <nirik> note: next friday is a holiday in the us...
16:56:48 <tyll> #action till write use his PP powers to merge tcp wrapper patches
16:56:52 <tyll> #topic Next week's chair
16:57:02 <tyll> nirik: yes, in Germany as well
16:57:12 <tyll> proposal: Skip next week?
16:57:39 <nirik> yeah, we should skip.
16:57:41 <tyll> proposal: Do next meeting on April, 6th?
16:58:20 <tyll> #info next meeting will be on 2018-04-06 due to public holidays
16:58:28 <bowlofeggs> i will likely be absent on april 6 (spring break, woooo! [my wife is a teacher so i get spring break too])
16:58:35 <tyll> who will be the chair in two weeks?
16:58:40 <zbyszek> I can do it.
16:58:45 <tyll> zbyszek: thank you
16:58:52 <tyll> #action zbyszek will chair next meeting
16:58:57 <tyll> #topic Open Floor
16:59:04 <zbyszek> tyll: https://src.fedoraproject.org/rpms/yaz/pull-request/1
16:59:05 <sgallagh> zbyszek: Actually, I haven't done it in a while. Maybe I should take it?
16:59:13 <zbyszek> sgallagh: as you wish
16:59:16 <tyll> anything or I will close in a minute?
16:59:21 <sgallagh> I feel like you've been covering it an unfair amount of time :)
16:59:30 <tyll> #action sgallagh will chair instead of zbyszek
16:59:41 <sgallagh> I do have one item for Open Floor
16:59:42 <sgallagh> (Sorry)
16:59:52 <tyll> sgallagh: what is it?
17:00:09 <sgallagh> It's related to the Workstation Edition's suspend-by-default changes in F28
17:00:46 <sgallagh> I think it would probably be good for FESCo to keep an eye on it, as kparal discovered yesterday that this behavior leaks into other parts of Fedora if you install GNOME bits
17:00:52 * zbyszek has seen the "Computer will suspend very soon because of inactivity." message in VMs
17:00:56 <sgallagh> (For example, installing @gnome-desktop atop Server Edition)
17:01:16 <kparal> zbyszek: the VMs don't get suspended, but the notification still pops up. a bug
17:01:23 <zbyszek> kparal: yes
17:01:23 <sgallagh> The net result is a server that goes to sleep every 20 minutes that it isn't logged in at the console
17:01:55 <sgallagh> zbyszek: I've seen that happen on my laptop when I unlock it after a long absence... but it didn't suspend.
17:01:59 <sgallagh> So that's also weird
17:02:08 <bowlofeggs> oh that doesn't sound good
17:02:21 <zbyszek> Yes, I also saw it on my laptop immediately after dnf ugprade --releasever=28, but it didn't suspend.
17:02:51 <sgallagh> Anyway, I don't think there's any action for FESCo to take *yet*. But I figured I should make sure we're aware of the situation
17:03:16 <sgallagh> I talked with mclasen about it in the office this morning and he acknowledged that the Server going to sleep was definitely not appropriate.
17:03:37 <sgallagh> I'll stay on that and report back.
17:03:37 <tyll> sgallagh: thank you for the heads up
17:04:05 <tyll> ok, anything else?
17:04:15 <sgallagh> Nothing from me
17:04:38 <tyll> Thank you everyone for attending
17:04:41 <tyll> #endmeeting