15:00:39 <zbyszek> #startmeeting FESCO (2018-05-18)
15:00:39 <zodbot> Meeting started Fri May 18 15:00:39 2018 UTC.
15:00:39 <zodbot> This meeting is logged and archived in a public location.
15:00:39 <zodbot> The chair is zbyszek. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:39 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:39 <zodbot> The meeting name has been set to 'fesco_(2018-05-18)'
15:00:39 <zbyszek> #meetingname fesco
15:00:39 <zodbot> The meeting name has been set to 'fesco'
15:00:39 <zbyszek> #chair maxamillion dgilmore nirik jsmith sgallagh bowlofeggs tyll jwb zbyszek
15:00:39 <zodbot> Current chairs: bowlofeggs dgilmore jsmith jwb maxamillion nirik sgallagh tyll zbyszek
15:00:42 <zbyszek> #topic init process
15:00:47 <zbyszek> do we have quorum?
15:00:53 <jwb> i'm in a work meeting
15:00:53 <zbyszek> sgallagh is missing today
15:00:54 <maxamillion> .hello2
15:00:56 <nirik> morning
15:00:58 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
15:01:02 <zbyszek> bowlofeggs will be late
15:01:07 <zbyszek> .hello2
15:01:08 <zodbot> zbyszek: zbyszek 'Zbigniew Jędrzejewski-Szmek' <zbyszek@in.waw.pl>
15:02:32 <dgilmore> hola
15:02:57 <zbyszek> OK, let's start, assuming that jwb can vote
15:03:07 <zbyszek> #topic #1892 F29 Self Contained Change: MySQL 8
15:03:10 <zbyszek> .fesco 1892
15:03:11 <zodbot> zbyszek: Issue #1892: F29 Self Contained Change: MySQL 8 - fesco - Pagure - https://pagure.io/fesco/issue/1892
15:03:13 <zbyszek> https://pagure.io/fesco/issue/1892
15:04:23 <nirik> sgallagh was -1 on ticket
15:05:07 <zbyszek> OTOH, can we forbid maintainers of a package from updating the version?
15:05:28 <zbyszek> It might not be strictly backwards compatible, but version updates often are not.
15:05:34 <zbyszek> What is so special about mysql?
15:05:37 <dgilmore> all we can really do is guide them to make good choices
15:05:59 <nirik> yeah.
15:06:50 <dgilmore> given that mysql 8 is already built in rawhide
15:06:57 <dgilmore> its hard to say no
15:07:11 <jsmith> Sorry I'm late :-)
15:07:40 <maxamillion> I'd be interested to know how this has been handled in the past for database system software ... surely there've been non-trivial mysql/mariadb and postgresql upgrades in the past
15:07:52 <maxamillion> do we just throw them out there and let everyone deal with the pieces?
15:07:59 <jsmith> maxamillion: There have, expecially when we moved away from MySQL and towards MariaDB
15:08:34 <tyll_> .hello till
15:08:35 <zodbot> tyll_: till 'Till Maas' <opensource@till.name>
15:09:31 <dgilmore> I think 5.7 could be made a module
15:09:35 <dgilmore> as could 8
15:09:47 <nirik> they say in the change page a 5.7 module is planned
15:10:07 <dgilmore> maxamillion: for postgresql you have in the past had to run a manual process to migrate the data to a new format
15:10:32 <maxamillion> dgilmore: and did FESCo ever have any intervention with the packager when that was the case?
15:10:36 <nirik> that appears to be the case here as well (at least one report on the list said they needed to manually upgrade the db)
15:10:58 <nirik> maxamillion: to do what? I'm not sure that isn't the best choice...
15:11:57 <zbyszek> hmm, so what is the latest version of mysql in rawhide really?
15:12:13 <zbyszek> I see mysql-8.0-20180507133324.c7b355af (no dist tag), but later mysql-5.6-20180507181100.c7b355af
15:12:23 <zbyszek> https://koji.fedoraproject.org/koji/packageinfo?packageID=687
15:12:46 <zbyszek> (seems to be another package that lost %dist)
15:13:13 <nirik> 8.0.11
15:13:23 <nirik> it's community-mysql
15:13:37 <dgilmore> zbyszek: https://koji.fedoraproject.org/koji/buildinfo?buildID=1075447
15:13:49 <nirik> https://koji.fedoraproject.org/koji/packageinfo?packageID=15792
15:14:34 <zbyszek> Oh, thanks.
15:14:39 <nirik> proposal: wait for another week of discussion on list, ask maintainer to respond to issues with upgrades?
15:14:40 <jsmith> dgilmore: I like the idea of it being a module
15:14:46 <jsmith> nirik: +1
15:14:54 <zbyszek> nirik: +1
15:14:55 <dgilmore> nirik: +1
15:15:43 <zbyszek> jwb tyll maxamillion ?
15:16:33 <maxamillion> +1
15:17:46 <zbyszek> #agreed FESCo asks the maintainer to respond to the questions about upgrade issues. We'll discuss this next week (+5, 0, 0)
15:18:05 <zbyszek> Let's make a small swithc in the agenda
15:18:17 <zbyszek> and do the third ticket now
15:18:21 <zbyszek> #topic #1877 large number of packages FTBFS in F28
15:18:21 <zbyszek> .fesco 1877
15:18:21 <zbyszek> https://pagure.io/fesco/issue/1877
15:18:23 <zodbot> zbyszek: Issue #1877: large number of packages FTBFS in F28 - fesco - Pagure - https://pagure.io/fesco/issue/1877
15:19:00 <zbyszek> I proposed to close the ticket in https://pagure.io/fesco/issue/1877#comment-512628
15:19:01 * nirik liked tyll's last comment. :)
15:19:14 <nirik> oh, this is the other one
15:19:33 <nirik> +1 to the proposal
15:20:12 <zbyszek> I'm +1 to my own proposal
15:20:16 <zbyszek> sgallagh was +1 in the ticket
15:21:41 <bowlofeggs> .hello2
15:21:42 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:22:22 <bowlofeggs> +1 to zbyszek
15:22:31 <zbyszek> dgilmore, maxamillion, jwb, jsmith, tyll?
15:22:41 <jsmith> +1
15:22:45 <jsmith> Seems reasonable...
15:22:45 <maxamillion> +1
15:22:57 <maxamillion> sorry, I'm double booked and trying to multi-task ...
15:23:02 <bowlofeggs> (apologies for my tardiness, i had a slight scheduling overlap that required my physical presence away from $computer
15:23:05 <bowlofeggs> )
15:23:12 <tyll> I am not sure it is consistent
15:23:49 <dgilmore> +1
15:24:06 <zbyszek> tyll: what do you propose?
15:25:19 <tyll> zbyszek: first that we agree on a generic timeline goal, for when to retire/orphan pkgs when a) the maintainer does not react b) the maintainer reacts but fails to fix the pkg
15:25:53 <zbyszek> tyll: that's the other ticket
15:25:58 <tyll> oh
15:25:59 <tyll> sorry
15:26:01 <tyll> +1 then
15:26:05 <zbyszek> sorry, I wanted to close this one first
15:26:19 <zbyszek> #agreed Close this bug (+7, 0, 0)
15:26:19 <zbyszek> #info releng ticket 7469 and fesco ticket 1890 remain open to track related issues
15:26:23 <tyll> ah, we just need to make sure that ther are all bugs filed btw
15:26:35 <tyll> if we want it to count for the other deadline
15:27:01 <tyll> did not get to check it, yet, but I will work on it even when the ticket is closed :-)
15:27:18 <zbyszek> I think we'd need to do another mass rebuild, because we lost track what failed to build and why
15:27:39 <zbyszek> But yeah, I think it's reasonable to open ftbfs bugs for all packages
15:27:52 <zbyszek> Even outside of the mass rebuild
15:28:05 <zbyszek> #topic #1890 updating the FTBFS cleanup policy
15:28:05 <zbyszek> .fesco 1890
15:28:05 <zbyszek> https://pagure.io/fesco/issue/1890
15:28:07 <tyll> I can easily rebuild everything that failed to build the last time but I want to get the tooling for bug checking ready, first
15:28:08 <zodbot> zbyszek: Issue #1890: updating the FTBFS cleanup policy - fesco - Pagure - https://pagure.io/fesco/issue/1890
15:29:15 <nirik> can we put this on a fesco policy wiki page? I guess we should decide things first, but it definitely needs to be there someday
15:29:17 <zbyszek> tyll: I agree with your comments in your last comment
15:29:55 <jsmith> tyll: I like it :-)
15:29:58 <zbyszek> tyll: 6½ months = 6 months between mass rebuilds + 2 weeks
15:30:33 <dgilmore> my main question is who is going to do the work?
15:30:48 <zbyszek> a question though: could we reuse need-info for the weekly reminders
15:31:07 <zbyszek> Then we wouldn't need to create a new script, just set the field when the bug is created
15:31:21 <dgilmore> many of the steps that are caklled out as having to be done are currently manual only and done on a best effort basis
15:31:32 <dgilmore> just saying it has to be done will not make it so
15:31:46 <dgilmore> the only real way to ensure it is all done is to automate it
15:31:54 <dgilmore> but that needs work to happen
15:32:13 <zbyszek> dgilmore: I'd be happy to work a bit on this
15:32:17 <tyll> I am happy to work on it in general, but I cannot promise a timeline for when it will be rady
15:32:39 <zbyszek> I did some scripting for bugzilla for this, and it wasn't too onerous
15:32:57 <zbyszek> What exactly do we need?
15:33:09 <zbyszek> 1. get a list of ftbfs packages
15:33:13 <bowlofeggs> do it serverless
15:33:18 <maxamillion> bowlofeggs: ...
15:33:19 <bowlofeggs> hahaha
15:33:23 <zbyszek> 2. open tickets, set the right tags and so on
15:33:42 <maxamillion> ansible playbook would make sense, the infra team uses ansible, the releng team uses ansible, the fedora apps team uses ansible ... it's a common tool in the toolbox
15:34:05 <zbyszek> 3. do the retirement
15:34:15 * maxamillion does note his personal bias in favor of ansible so take that with whatever grain of salt you feel most comfortable with
15:34:42 <zbyszek> *orphaning
15:34:42 <dgilmore> zbyszek: we have scripts to file the bugs
15:34:43 * nirik likes very much the idea of automating as much of this as possible.
15:34:55 <nirik> the timing might be the most difficult part...
15:34:56 <dgilmore> zbyszek: we have scripts to do the pieces needed
15:35:04 <tyll> maxamillion: will you write a facts plugin for koji builds and Bugzilla bugs? ;-)
15:35:04 <dgilmore> we do not have anything automating it
15:35:19 <jsmith> tyll: That's just what I was going to ask :-)
15:35:33 <dgilmore> zbyszek: the scripts would need to know how to get auth tokens and run the scripts, and when to do so
15:35:41 <zbyszek> maxamillion: ansible makes it sound like this needs to be run by releng, but it would be nice if this could be run by any pp
15:36:02 <dgilmore> zbyszek: it probably needs to be run by releng
15:36:13 <dgilmore> zbyszek: bugs should all be filed by the releng usert
15:36:15 <dgilmore> user
15:36:36 <dgilmore> zbyszek: some of the actions needed to be run need special permissions
15:36:36 <tyll> zbyszek: proven packagers cannot orphan other persons pkgs
15:36:46 <tyll> zbyszek: they can only retire ;-)
15:36:59 <zbyszek> oops
15:37:10 <maxamillion> zbyszek: I think it needs to be releng for logistical purposes and because of permissions
15:37:34 <zbyszek> Well, so my proposal would agree on the rules, and work with releng to figure out what is missing to automatize this
15:38:28 <dgilmore> I am not saying we need to make any of it perfect, we do need to amke it good
15:38:52 <zbyszek> dgilmore: is "good" when a human needs to fire off a script by hand at the right time?
15:38:53 <tyll> Let's agree on a generic timeline goal, for when to retire/orphan pkgs when a) the maintainer does not react b) the maintainer reacts but fails to fix the pkg
15:39:04 <dgilmore> if we do not ensure that things happen automatically they are likely to get skipped at times
15:39:17 <tyll> we can discuss the technical issues in a smaller circle imho
15:39:24 <dgilmore> zbyszek: good is if it is remebered to be done
15:39:36 <dgilmore> zbyszek: in the case of the f28 mass rebuild many steps were skipped
15:39:46 <nirik> I think we can't get around having to run scripts, but if we just tell releng 'do this' it's likely not to get done due to all the other work releng has.
15:40:11 <nirik> scripts + docs + adding to FPM schedule I think we might be ok.
15:40:29 <zbyszek> I don't we have any better alternatives
15:40:39 <zbyszek> Doing it from a cron jobs sounds very risky
15:40:54 <nirik> we can't likely do cron jobs
15:40:57 <dgilmore> zbyszek: I do not think it should be based on cron jobs
15:41:09 <zbyszek> Yeah
15:41:09 <dgilmore> I think all releng cron jobs need to die in fire
15:41:10 <nirik> mass rebuilds do not occur at specific known times...
15:41:20 <dgilmore> as failures get ignored
15:41:29 <dgilmore> there needs to eb a framework for automation
15:41:50 <bowlofeggs> let's do it in java
15:41:51 <zbyszek> The upside is that if this retirement is done for a few releases, mass rebuilds will likely get quicker, because there will be lots less failing packages
15:41:59 <bowlofeggs> and xml
15:42:00 <nirik> we could perhaps enhance the mass rebuild tooling to emit fedmsgs and have the FTBFS stuff trigger on those
15:42:08 <maxamillion> bowlofeggs: you are why we can't have nice things ;)
15:42:10 <maxamillion> bowlofeggs: <3
15:42:12 <bowlofeggs> hahaha
15:42:17 <dgilmore> nirik: indeed
15:42:35 <nirik> or fold them into the same script... do the actions right after something fails.
15:42:57 <nirik> but all this is technical. :)
15:43:13 <maxamillion> nirik: what's the status of getting Ansible AWX deployed into the infra and wired up to FAS? ... I feel like this would be a good candidate for an AWX workflow
15:43:35 <nirik> as tyll noted, we should decide policy and let the people doing the work decide technical things.
15:43:36 <maxamillion> but yes ... technical details that can be hashed out outside of meeting time
15:43:38 <zbyszek> tyll: your approach of waiting 8 weeks in NEW state and orphaning, and if !NEW orphaning one week before branching sounds good
15:44:06 <nirik> maxamillion: it failed... we are going to work with them later this year to try it again when things are in a happier place for us and them.
15:44:39 <maxamillion> nirik: bummer ... noted
15:44:44 <zbyszek> Do y'all think there should be another version of the proposal, or can we approve "zbyszek's proposal + tyll's changes", and then hammer out any details later?
15:44:49 <maxamillion> nirik: what about just running fully supported Tower?
15:45:02 <maxamillion> actually, nvm ... off-topic, I'll follow up later
15:45:17 <nirik> zbyszek: I could be +1 to that and we can adjust from there if we want/need to.
15:45:24 <nirik> maxamillion: yeah, happy to talk outside meeting sometime.
15:46:11 <tyll> zbyszek:  !NEW orphaning one week before branchings means the broken pkgs will be branched, too
15:46:32 <tyll> maybe we could ask releng to not branch orphaned pkgs
15:46:33 <zbyszek> tyll: we can retire until right before release
15:46:45 <zbyszek> tyll: that would work too
15:47:12 <jsmith> tyll: Seems reasonable
15:47:36 <bowlofeggs> zbyszek: i'm +1 to your proposal + tyll's suggested changes
15:48:48 <jsmith> zbyszek: I'm +1 for "your proposal + tyll's changes"
15:49:08 <tyll> zbyszek: +1 to our combined proposal
15:49:15 <jsmith> zbyszek: I'm also perfectly find with hammering out the details over time.
15:49:25 <jsmith> zbyszek: No need to get every little detail perfect in this meeting
15:50:12 <dgilmore> there is many details to hammer out, but +1 to the idea
15:50:24 <dgilmore> with tyll's changes
15:50:43 <nirik> and it would be nice to have a clear wiki page describing all this. ;) or somewhere else
15:51:13 <dgilmore> nirik: perhaps bexelbie can tell us about doc options :)
15:51:36 <dgilmore> I hear he loves restructured text
15:51:44 <nirik> who doesn't? ;)
15:51:50 <zbyszek> #agreed The proposal from https://pagure.io/fesco/issue/1890#comment-512632 with changes in https://pagure.io/fesco/issue/1890#comment-512813 is approved. We'll hammer out any details later (+6, 0, 0)
15:52:08 <bowlofeggs> awesome
15:52:39 <maxamillion> go team
15:53:08 <zbyszek> #topic Next week's chair
15:53:26 <zbyszek> volunteers?
15:53:57 <jsmith> I can only cover the first 45 minutes...
15:54:04 <jsmith> ... but otherwise am happy to run it
15:55:26 <nirik> I guess I have not done it in a while... I could take next week
15:55:34 <zbyszek> That's be better.
15:55:40 <maxamillion> let me check
15:55:48 <zbyszek> jsmith: you'll have your chance in two weeks ;)
15:56:03 <zbyszek> #action nirik will chair next meeting
15:56:06 <maxamillion> nirik: I can take next week if you'd rather save your turn for another time :)
15:56:15 <zbyszek> #undo
15:56:15 <zodbot> Removing item from minutes: ACTION by zbyszek at 15:56:03 : nirik will chair next meeting
15:56:22 <nirik> maxamillion: doesn't matter, if you want it thats fine. ;)
15:56:30 <zbyszek> #action maxamillion will chair next meeting
15:56:34 <zbyszek> #topic Open Floor
15:56:57 <maxamillion> nirik: and so it is :)
15:57:31 <zbyszek> I'll close the meeting in 2 minutes if nobody speaks up
15:57:34 <dgilmore> elections are upon us
15:58:07 <dgilmore> #info nomination period for elections closes on MAy 22
15:58:18 <dgilmore> #undo
15:58:18 <zodbot> Removing item from minutes: INFO by dgilmore at 15:58:07 : nomination period for elections closes on MAy 22
15:58:21 <nirik> who's seats are up?
15:58:22 <dgilmore> #info nomination period for elections closes on May 22
15:58:27 <bowlofeggs> my seat is in danger
15:58:53 <bowlofeggs> especially after that vote i cast to murder all those helpless seals
15:59:00 <dgilmore> bowlofeggs, tyll, sgallagh and dgilmore all have seats up
15:59:00 <bowlofeggs> that was very unpopular with my voter base
15:59:24 <dgilmore> currently jforbes and sgallagh have nominated
15:59:27 <nirik> bowlofeggs: quick! bring in some folding chairs to pad the seat vote.
15:59:31 <bowlofeggs> hahaha
15:59:32 <maxamillion> wait, I thought I was up this time because I wasn't actually up last time
15:59:39 <dgilmore> maxamillion: that means you do not nominate this time
15:59:53 <maxamillion> I was told I was and I ended up on the ballot but my seat wasn't up
15:59:54 <bowlofeggs> maxamillion: this happened last summer too
15:59:55 <dgilmore> https://fedoraproject.org/wiki/Development/SteeringCommittee/Nominations
15:59:56 <maxamillion> dgilmore: ?
16:00:27 <dgilmore> maxamillion: it was a year ago the mess up happened
16:00:34 <maxamillion> oh man time flies
16:00:37 <maxamillion> alright, nvm me
16:00:56 <zbyszek> pfew, had me worried for a moment there
16:02:06 <zbyszek> bowlofeggs: do you plan to run again?
16:02:22 * dgilmore is not running
16:02:32 * dgilmore hopes tyll and bowlofeggs run again
16:02:43 * dgilmore also hopes that some new people run
16:02:52 <maxamillion> +1 to new people running
16:03:05 <jforbes> dgilmore: doh, stepping down? Thank you for serving as long as you have
16:03:12 <bowlofeggs> zbyszek: i plan to run again
16:03:23 <bowlofeggs> need to nominate and do the interview :)
16:03:26 <maxamillion> dgilmore: +1 - thank you for serving the Fedora Community for as long as you have
16:03:39 * bowlofeggs will miss dgilmore
16:03:50 <dgilmore> jforbes: indeed I am stepping down. between new job, council and school I am busy
16:04:02 <bowlofeggs> the last election was super tough
16:04:05 <jforbes> Makes sense
16:04:10 <bowlofeggs> there were only great candidates :)
16:04:19 <bowlofeggs> and tehre were more of them than there were seats
16:05:03 <bowlofeggs> hopefully we have that problem this time too :)
16:06:00 <zbyszek> OK, good luck to all candidates ;)
16:06:22 <zbyszek> 3
16:06:26 <zbyszek> 2
16:06:30 <zbyszek> 1
16:06:32 <zbyszek> #endmeeting