17:00:58 <mboddu> #startmeeting RELENG (2018-08-23)
17:00:58 <zodbot> Meeting started Thu Aug 23 17:00:58 2018 UTC.
17:00:58 <zodbot> This meeting is logged and archived in a public location.
17:00:58 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:58 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:00:58 <zodbot> The meeting name has been set to 'releng_(2018-08-23)'
17:00:58 <mboddu> #meetingname releng
17:00:58 <zodbot> The meeting name has been set to 'releng'
17:00:58 <mboddu> #chair nirik tyll sharkcz masta pbrobinson pingou puiterwijk maxamillion mboddu Kellin dustymabe
17:00:58 <mboddu> #topic init process
17:00:58 <zodbot> Current chairs: Kellin dustymabe masta maxamillion mboddu nirik pbrobinson pingou puiterwijk sharkcz tyll
17:01:07 <nirik> morning
17:01:33 <masta> morning
17:02:14 <puiterwijk> morning
17:04:24 <mboddu> So, I dont have much other than couple of open floor things
17:04:44 <mboddu> If anyone has anything, they can go first
17:04:49 <mboddu> #topic Open Floor
17:05:22 <masta> go for it mboddu
17:06:34 <mboddu> Okay
17:07:25 <mboddu> I had a talk with Kellin, he said he will be busy for the first couple of months with his new job, so someone has to pick up the push duty
17:07:39 <mboddu> Anyone of you know of someone who might be interested?
17:07:43 <mboddu> If not I will pick it up
17:07:48 <masta> we can just shorten the rotation
17:08:04 <masta> but I don't mind picking up the slack
17:09:30 <puiterwijk> Well, we've talked about this from time to time, but maybe we can discuss it again: making the push a cron job. And making push duty become a "you are responsible if something breaks, but otherwise, just stare at it"
17:10:17 <masta> yeah, I was just thinking the same thing... it's so trivial now, it used to eat hours of my day, now it's like ~ 45 seconds usually.
17:10:19 <nirik> I kinda like that idea...
17:10:27 <nirik> and then it's also more regular.
17:10:35 <mboddu> puiterwijk: Sure, we had a discussion about it couple of days back
17:11:01 <puiterwijk> Yeah. I know my push duties are kind of irregular... (Monday I almost forgot and I started it midnight UTC, the rest of the time I do it when I wake up, so yeah...)
17:11:23 <puiterwijk> I do do a push every day. Just not always at predictable times, which might be nice
17:11:39 <puiterwijk> mboddu: ah, cool. And did we enable it yet? :P
17:11:44 <mboddu> puiterwijk: https://taiga.fedorainfracloud.org/project/acarter-fedora-docker-atomic-tooling/us/1010 look at the last comment
17:12:01 <masta> same, I tend to forget to ask zodbot who's on .pushduty on Mondays
17:12:03 <mboddu> puiterwijk: Unfortunately no, but I guess we can
17:12:58 <puiterwijk> mboddu: so, the "what releases it's running on" and most of the freeze stuff could be just part of the cronjob definition
17:13:04 <puiterwijk> Also less prone to human error
17:13:18 <puiterwijk> So we just add a definition of what to push in the cronjob, and call it a day.
17:13:23 <mboddu> puiterwijk: Yup, I wanted to get the values from PDC (or its replacement)
17:13:36 <mboddu> But thats something we can add later
17:13:42 <puiterwijk> mboddu: Right. But that's going to be a while. Why not just do it manually in the cron definition for now?
17:13:42 <nirik> well, if you are pushing everything it's easy just don't say and bodhi does the right thing
17:14:09 <puiterwijk> Yep. And during freezes, just list all the other releases. And revert on unfreeze
17:14:29 <mboddu> Right, we can do that
17:14:32 <nirik> hum, can bodhi just do everything if you say only --testing?
17:14:39 <masta> what happens when we have to sign differently from normal?
17:14:49 <mboddu> And add a check, if bodhi is running, dont push it again
17:14:53 <puiterwijk> masta: signing is now all autosigned
17:15:12 <puiterwijk> mboddu: multiple pushes are actually not a problem anymore
17:15:46 <masta> oh, I guess in my mind sign & push are joined.... I meant to say push differently. Say when branched is pushed before the existing fedoras, or whatever
17:15:46 <bowlofeggs> i think bodhi-push does currently disallow multiple pushes, but we could change that imo
17:15:47 <puiterwijk> mboddu: or at least, with limitations.
17:15:55 <puiterwijk> bowlofeggs: yeah.
17:15:58 <mboddu> puiterwijk: What happens if couple of releases failed which are from different pushes?
17:16:12 <puiterwijk> mboddu: then the person on pushduty needs to look
17:16:35 <nirik> yeah we can just specify --request testing during freeze
17:16:41 <puiterwijk> mboddu: also, bodhi-push will then detect that. And we can make it deal otherwise with it
17:16:41 <bowlofeggs> FTR, I'm +1 to cron pushing ☺
17:17:19 <mboddu> bowlofeggs: Everyone is :), just figuring it out what are the issues we might have to face
17:17:27 <puiterwijk> bowlofeggs: I believe the one major thing we'd need to do in the code is add a -y flag to bodhi-push? Which would be trivial to add.
17:17:35 <bowlofeggs> puiterwijk: true
17:17:37 <nirik> we could have the cron run 'bodhi composes list' and exit if there were any
17:17:54 <puiterwijk> Yep. What KEvin says. Although bodhi-push could also do that itself
17:18:03 <nirik> or just 'echo y | bodhi-push ...'
17:18:07 <puiterwijk> nirik: we could, when we add the -y flag to bodhi-push, make it not do yes if there's a non-empty list
17:18:12 <bowlofeggs> bodhi push actually does already exit if there are any
17:18:13 <puiterwijk> Or that.
17:18:15 <mboddu> nirik: As per bowlofeggs, bodhi wont push if there is an exisiting push already, so we dont need it
17:18:19 <bowlofeggs> well, more that it prompts to you resume
17:18:22 <puiterwijk> bowlofeggs: no, it will ask if you want to resume
17:18:23 <nirik> great
17:18:23 <puiterwijk> Yeah
17:18:32 <puiterwijk> So actually...
17:18:44 <puiterwijk> bowlofeggs: I believe that on resume, it first asks resume: y/n, and then start: y/n, right?
17:18:55 <bowlofeggs> puiterwijk: yeah that sounds right to me
17:18:56 <puiterwijk> Since then we could do an printf "y\nn\n" | bodhi-push ...
17:18:57 <nirik> it asks a bunch of things.
17:19:05 <nirik> like y/n for each failed branch
17:19:06 <bowlofeggs> you guys use it a lot more than i do hahah
17:19:09 <puiterwijk> So that *if* there's a resume, the "shall we start" will be answered with no
17:19:12 <bowlofeggs> i only use it in staging to test
17:19:59 <puiterwijk> nirik: (echo "y"; yes | sed -e 's/y/n/') | bodhi-push  :D
17:20:14 <puiterwijk> i.e. answer y to a first question, n if there are any others
17:20:26 <nirik> seems easier to just exit when there are running pushes. ;)
17:20:28 <nirik> but sure
17:20:28 <puiterwijk> So that i8f there's anything running, it will then still cancel :)
17:20:50 <puiterwijk> nirik: yeah. That should be really simple to add in bodhi with a -y flag
17:20:53 * mboddu prefers just exiting out
17:21:07 <puiterwijk> bowlofeggs: maybe add a -y for "assume yes except for resume", and a "-f" for "also y if resume":
17:21:16 <puiterwijk> (just... for the future :) )
17:21:19 <nirik> sounds good
17:21:26 <puiterwijk> I always like a --force flag. Even if it doesn't do anything :D
17:21:52 <mboddu> puiterwijk: haha, I didn't know what it does, but now I know :D
17:22:00 <mboddu> "Nothing"
17:22:28 <puiterwijk> So should be simple enough
17:22:35 <mboddu> Seems like it
17:22:46 <puiterwijk> So. Shall we just bite the bullet and do it?
17:23:00 <mboddu> +1's Please
17:23:03 <mboddu> I am +1
17:23:20 <nirik> or since this iis bodhi a --awakening flag or perhaps a point break refrence, but not sure which.
17:23:29 <nirik> +1
17:23:56 <mboddu> Okay, I think masta is okay with this
17:24:00 <masta> in the mean time, I don't mind doing his signing
17:24:01 <bowlofeggs> yeah --force is lots of fun :)
17:24:08 <puiterwijk> --no-justice-limit ?
17:24:16 <mboddu> masta: "signing"?
17:24:17 <puiterwijk> masta: there should be no manual signing at all anymore :)
17:24:20 <masta> --no-really-yes
17:24:39 <bowlofeggs> --the-last-jedi
17:24:49 <bowlofeggs> --the-empire-did-nothing-wrong
17:24:51 <mboddu> --no-more-manual-bodhi-pushes
17:24:53 <puiterwijk> --make-your-getaay
17:25:06 * puiterwijk was actually looking at quotes of Point Break (never heard of it before just now actually)
17:25:18 <nirik> its so bad it's good. ;)
17:25:50 <mboddu> Anyone wants to work on it?
17:26:13 <masta> fyi +1
17:26:34 <masta> mboddu,  the cron job, or the bodhi enhancement?
17:26:35 <mboddu> If not I will take it
17:26:40 <mboddu> masta: Cron job
17:27:11 <masta> mboddu, you do it, but I'm sure we can all contribute with code reviews or whatever
17:27:12 <nirik> what time we do want to run it?
17:27:28 <puiterwijk> nirik: 00:00 UTC?
17:27:30 <masta> nirik, I'd say 00:01 utc
17:27:43 <bowlofeggs> note that bodhi-dequeue-stable i think runs at 03:00
17:27:58 <bowlofeggs> might want to run this just after that (or maybe in the same job for now?)
17:28:10 <nirik> yeah, after that would make sense.
17:28:23 <bowlofeggs> we can change when it runs of course - it was arbitrary anyway
17:28:27 <puiterwijk> Ah. Then 04:00
17:28:33 <bowlofeggs> imo, making it the same job could be a nice way to do it
17:28:44 <bowlofeggs> until/unless we actually want to reenable batching
17:28:47 <mboddu> nirik: I prefer 23:59 UTC, since freeze starts at 00:00 UTC
17:28:50 <bowlofeggs> that's a whole other can o worms
17:29:32 <puiterwijk> bowlofeggs: not now
17:29:33 <nirik> ok, then move the dequeue one to 23:00 and run the updates at 23:59
17:29:39 <puiterwijk> +1
17:29:56 <mboddu> +1
17:30:03 <masta> sure +1
17:31:21 <mboddu> #info bodhi push will be converted to a cron job which runs at 23:59 UTC and we will move the bodhi-dequeue-stable to 23:00 UTC
17:31:27 <puiterwijk> \o/
17:32:00 * dustymabe reading scrollback
17:32:15 <dustymabe> ok maybe I shouldn't read it all
17:32:23 <dustymabe> but I would like to ask.. can we still have someone on pushduty
17:32:26 <mboddu> puiterwijk: I know you like bodhi pushes, I hope you are not feeling nostalgic?
17:32:28 <puiterwijk> dustymabe: just the bodhi cron'ed pushing. Nothing else
17:32:32 <dustymabe> who is responsible for making sure it went successful
17:32:36 <puiterwijk> dustymabe: yes, we do. And they are responsible if things fall down
17:32:40 <dustymabe> +1
17:32:43 <dustymabe> woot!
17:32:53 <puiterwijk> mboddu: I like bodhi pushes. I just don't like needing humans to do what a machine can do :)
17:33:04 <bowlofeggs> the computer knows what's best
17:33:15 <puiterwijk> Well, at least the computer is best in consistent running of tasks
17:33:24 <mboddu> dustymabe: Yes, instead of pushing, now their responsibility is to just debug them if there is a failure
17:33:32 <puiterwijk> Hah.... Totally random thought I just had: we could put this cron job in Openshift!
17:33:35 * puiterwijk hides
17:34:10 <nirik> we could.
17:34:12 <dustymabe> thanks!
17:34:14 <nirik> they are pretty cool.
17:34:14 <masta> my only concern is with this automation we effectively reduce the people who know about push/signing issues to the current set, or something like that.
17:34:26 <puiterwijk> masta: that's where documentation comes in :)
17:34:53 <masta> indeed
17:35:48 * mboddu thinking of the second topic he wanted to bring up
17:35:53 <dustymabe> i know about push/signing issues but I usually can't fix any of them
17:35:54 <mboddu> But I cant remember now :(
17:36:04 <dustymabe> I try to chime in in the issue tracker when it's something I see I can help with
17:36:27 <puiterwijk> dustymabe: That's much appreciated. I do hope that in time we'll get things to auto-fix themselves
17:36:30 <nirik> I had one item: There was a post to the releng list about fedora-release / fedora-repos workflow...
17:36:48 <mboddu> nirik: Sure, go ahead
17:36:52 <nirik> I answered it when it was a bugzilla bug, but asked them to move it to the list. Dunno if anyone has thoughts on that or not.
17:38:11 <nirik> https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.org/thread/N3GNR3IXN7MLP2CYY2ALLZXOCX5QMYDI/
17:38:58 <nirik> for the last item we could add a PR template to ask people NOT to change release/version or add changelog entries.
17:40:23 <mboddu> nirik: I like all 4 of the changes
17:40:44 <nirik> I thought you and pbrobinson liked the sign off thing?
17:41:02 * dustymabe steps AFK
17:41:21 <mboddu> nirik: I do like it, but most of the people hate it, so ...
17:42:21 <nirik> it's another bar to contributors for no real reason. ;) I'd be happy to drop it... but if we are ok with those things can someone reply to the list and we can do them
17:42:39 <mboddu> nirik: Sure, I can do that
17:43:23 <mboddu> Also pbrobinson and I are planning to add some PR testing, so that would avoid some issues
17:44:16 <nirik> excellent. More tests++
17:44:22 <mboddu> #info We are okay with all the changes mentioned in https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.org/thread/N3GNR3IXN7MLP2CYY2ALLZXOCX5QMYDI/ and mboddu will reply to it
17:44:38 <mboddu> And I remembered what I wanted to say
17:44:47 <mboddu> nirik: Is that all?
17:45:08 <nirik> thats all I had
17:46:02 <mboddu> #info fedora base container release is done and everything should be up to date on registry.fp.o
17:46:04 <mboddu> nirik: ^
17:46:35 <mboddu> I have to make a PR to changes I made to the script while running the script, but other than that it should be all updated
17:46:58 <nirik> nice. so branched and rawhide should update 29/30 ones nightly now?
17:47:18 <mboddu> ^^ Because "I dont always test my code, but, when I do, I do it in production"
17:47:47 <mboddu> nirik: rawhide will be updated nightly, but not f29
17:48:45 <mboddu> nirik: I didnt want to release f29 nightly because, its not actually a released version
17:49:00 <nirik> ok, IMHO it would make sense to do both, but I don't care too much
17:49:33 <mboddu> I will check with dustymabe and cverna, if they think we should release it, then I will add it
17:50:03 <nirik> ok
17:50:23 <mboddu> Thats all I had
17:50:43 <nirik> just having a up to date rawhide container will be very usefull
17:51:21 <mboddu> Yes, and it will update everyday
17:52:54 <mboddu> I will 2 more min and if nothing I will end the meeting and give back 5 min :)
17:53:07 * cverna reads back
17:54:34 <cverna> mboddu: it is up to you mboddu, but since f29 is still in development maybe a nightly release make sense
17:55:02 <mboddu> cverna: Okay, if you guys are okay with it, then I will add it as part of the nightly script
17:55:08 <nirik> yeah I was thinking nightly until release, then move to whatever we do for stable releases (2 weeks?)
17:55:09 <cverna> mboddu++
17:55:09 * masta has step away
17:55:35 <mboddu> nirik: 2 weeks release after the fedora release, yes
17:55:37 <cverna> nirik: I would like to have a weekly release, but I need to look at loopabull for that
17:55:53 <cverna> so mboddu does not have to run stuff manually
17:56:22 * mboddu loves not running stuff manually :)
17:57:46 <mboddu> Okay, thanks everyone for joining
17:57:49 <mboddu> #endmeeting