17:02:09 <mboddu> #startmeeting RELENG (2017-10-05)
17:02:09 <zodbot> Meeting started Thu Oct  5 17:02:09 2017 UTC.  The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:09 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:02:09 <zodbot> The meeting name has been set to 'releng_(2017-10-05)'
17:02:09 <mboddu> #meetingname releng
17:02:09 <mboddu> #chair dgilmore nirik tyll sharkcz masta pbrobinson pingou puiterwijk maxamillion mboddu Kellin
17:02:09 <mboddu> #topic init process
17:02:09 <zodbot> The meeting name has been set to 'releng'
17:02:09 <zodbot> Current chairs: Kellin dgilmore masta maxamillion mboddu nirik pbrobinson pingou puiterwijk sharkcz tyll
17:02:18 <nirik> morning
17:02:26 <mboddu> .hello2
17:02:27 <zodbot> mboddu: Sorry, but you don't exist
17:02:36 <masta> hey gang
17:02:37 <mboddu> .hello mohanboddu
17:02:38 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
17:02:41 <bowlofeggs> .hell2
17:02:45 <bowlofeggs> .hello2
17:02:45 <bowlofeggs> haha
17:02:45 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <randy@electronsweatshop.com>
17:03:16 * mboddu wonders why my .hello didn't work but .hello worked
17:03:39 <mboddu> s/.hello didn't/.hello2 didn't/
17:03:43 <nirik> hello2 needs your irc nick to be the same as your fas name. if it's not it won't work
17:04:05 <mboddu> nirik: ahhh, okay, then I will use .hello
17:04:40 <maxamillion> .hello maxamillion
17:04:41 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
17:06:15 <mboddu> Okay, I dont have much for today, but lets get started with alt arch updates
17:06:30 <mboddu> #topic Alternative Architectures updates
17:06:36 * threebean waves
17:06:40 <mboddu> sharkcz: any updates?
17:06:42 <maxamillion> threebean: o/
17:06:52 <mboddu> puiterwijk, nirik : whats the status on s390x?
17:07:17 <nirik> s390x is great! the link to the builders... not so great. ;)
17:07:28 <sharkcz> mboddu: ppc (and aarch64) look both good, for s390x I got lorax fixed (in updates-testing)
17:07:30 <mboddu> nirik: haha, rofl
17:07:48 <nirik> various solutions are being discussed, not sure we need to rehash them here.
17:08:34 <sharkcz> mboddu: the tools should work in pungi, it works when run manually
17:08:38 <mboddu> #info ppc and aarch64 looking good and a new lorax update for s390x is submitted
17:09:11 <mboddu> sharkcz: what do you mean by running manually and what tools? You mean composes?
17:09:32 <mboddu> #undo
17:09:32 <zodbot> Removing item from minutes: INFO by mboddu at 17:08:38 : ppc and aarch64 looking good and a new lorax update for s390x is submitted
17:09:48 <mboddu> #info ppc and aarch64 are looking good and a new lorax update for s390x is submitted
17:09:56 <sharkcz> mboddu: right now lorax with the similar command line as pungi sends to koji
17:10:12 <sharkcz> so I get the install.img & co created
17:11:32 <mboddu> sharkcz: okay, but the problem is mount link to the builders, so how is lorax update going to fix it?
17:12:04 <sharkcz> mboddu: it isn't, it uses http to get to the repos
17:12:19 <sharkcz> but for testing it's good enough
17:12:35 <mboddu> nirik: Do you think it will be solved by final RC compose request?
17:12:42 <mboddu> sharkcz: oh okay
17:12:56 <nirik> no idea, but we will try
17:13:01 <mboddu> nirik, puiterwijk : ^^ What do you think of using http?
17:13:36 <nirik> it won't work.
17:14:11 <nirik> at least my understanding is that it needs local repos and needs to write results
17:14:23 <mboddu> nirik: is it pungi requirement on /mnt/koji?
17:14:57 <mboddu> nirik: ^^ what you said
17:14:58 <nirik> yes, it needs to: a) run on the arch it's composing for and b) read and write repos / files that are on /mnt/koji
17:15:16 <nirik> ask dgilmore for details. Thats just what I have been told.
17:16:54 <mboddu> nirik: yeah, I know about the pungi requirement of repo availability in /mnt/koji, but I am not sure if it has changed recently
17:17:01 <sharkcz> afaik the runroot task in koji initiated by pungi need more data from the /mnt/koji than only the repos
17:17:28 <nirik> I don't think it's changed.
17:19:14 <mboddu> sharkcz: Okay, I will check with dgilmore, but I dont think we can use http
17:20:00 <mboddu> #action mboddu will check with dgilmore if we can use http to access the repos for composing s390x.
17:20:42 <mboddu> #info infra is still figuring out how to make the connection to s390x builders stable
17:22:11 <mboddu> Okay, moving on
17:22:18 <mboddu> #topic Open Floor
17:22:35 <mboddu> I got one thing
17:23:54 <mboddu> nirik: https://blog.mozilla.org/blog/2017/09/26/firefox-quantum-beta-developer-edition/ not that its important that a new version of firefox is releasing but I am concerned about Rust
17:24:13 <mboddu> And we have a ticket for it - https://pagure.io/releng/issue/6889
17:24:42 <mboddu> nirik: When do you think we should try https://pagure.io/releng/issue/6889#comment-460010
17:25:58 <mboddu> The reason I brought it up now because, f27 beta is out and for me it feels like its the best time to test something since we will have some time now
17:25:59 <nirik> we already did that. ;)
17:26:17 <mboddu> nirik: I didn't know that, so, what was the result?
17:26:18 <nirik> it works fine in rawhide, but will break with mash. So, now we need to wait on the switching bodhi to pungi work
17:27:00 <mboddu> nirik: is it tracked somewhere?
17:27:20 <nirik> I don't know. I think maxamillion was looking at it now...
17:28:12 * mboddu wonders how he missed it 🤔
17:28:58 <mboddu> nirik: yes, maxamillion took over the bodhi to pungi work or he is in the transition
17:29:21 <nirik> we could update this ticket above with that info...
17:29:30 <maxamillion> o/
17:29:44 <maxamillion> yeah, I'm still fighting OSBS fires but I'll get to that ASAP
17:29:45 <maxamillion> sorry
17:29:47 <maxamillion> in two meetings
17:30:00 <mboddu> nirik: Can you do that and put any findings you remember while you tried it?
17:30:09 <nirik> sure
17:30:19 <mboddu> maxamillion: no problem :)
17:30:25 <threebean> can I raise https://pagure.io/releng/issue/7054 ?
17:30:51 <threebean> what all kinds of things do we need to line up for that?  an outage ticket probably for starters?
17:31:06 <mboddu> threebean: sure, but we need dgilmore for that, unless nirik knows how to upgrade koji
17:31:34 <nirik> we have a ansible playbook for that now. ;)
17:31:36 <sharkcz> mboddu: FF builds with the bundled rust pieces fine on all arches, don't think it's blocked the rust packaging changes
17:31:37 <threebean> http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/playbooks/manual/upgrade/koji.yml
17:32:26 <katecarcia> threebean was hoping someone on rel eng to do this upgrade as he is incredibly booked, but the upgrade blocks modularity
17:32:31 <nirik> yeah, outage ticket I guess... I am also planning on trying to finish moving builders to f26... but that can be another time/out of outage
17:32:48 <nirik> I can do it... early next week ok? or needed sooner?
17:32:52 <mboddu> nirik, threebean : Okay, I think it needs more than that, but I am more than happy if everything is in ansible playbook
17:32:52 <threebean> nirik: can you recommend time for the outage?  if we schedule it, I can help.
17:33:08 <nirik> and we can do stg today/tomorrow?
17:33:20 <threebean> next week works here, yeah.
17:33:26 <nirik> how about monday at 21UTC?
17:33:37 <threebean> fwiw, I'm free monday or tuesday evening to help.
17:33:41 <threebean> +1
17:33:45 <nirik> I think that is 5pm EDT? which means most east coast people are off home
17:34:23 <nirik> I might update/reboot other stuff then too perhaps...
17:34:32 <nirik> at least the koji db server
17:34:40 <threebean> a caveat - I won't be back at my laptop until ~23 UTC that day.
17:35:40 <mboddu> nirik: Can I follow what you are doing during the upgrade? So, I can do it next time if needed, assuming I have the permissions it needs
17:37:15 <nirik> sure thing. You can run the playbook if you like too... I don't care. ;)
17:38:07 <nirik> I can write an outage ticket up.
17:38:08 <mboddu> nirik: sure, thanks :)
17:38:17 <katecarcia> just confirming: koji upgrade with mboddu and nirik this coming monady at 21utc?
17:38:20 <nirik> threebean / mboddu: will you have time to update stg today/tomorrow?
17:38:38 <threebean> I could run it now, but iirc bowlofeggs is running some tests.
17:38:53 <nirik> yes, I think he is... so after his testing
17:38:58 <threebean> mboddu: if you're free tomorrow, we could walk through stg together then?
17:39:00 <mboddu> nirik: I am free after 3PM today
17:39:12 <mboddu> threebean: tomorrow sounds awesome actually
17:39:20 <threebean> superb!  :)
17:39:32 <bowlofeggs> yeah i'm using stg koji atm
17:39:45 <katecarcia> mboddu so stage upgrade tomorrow? prod monday?
17:40:01 <mboddu> katecarcia: yes
17:40:09 <katecarcia> awesomesauce!
17:40:24 <mboddu> nirik: so, when are you scheduling for the prod koji upgrade? 21 UTC Monday?
17:40:30 <nirik> yes
17:40:41 <mboddu> nirik: Okay
17:41:21 <mboddu> #info nirik threebean mboddu will upgrade stg koji on Fri Oct 6th and prod on Mon Oct 9th at 21:00 UTC
17:42:19 <maxamillion> +1
17:43:04 <mboddu> Okay, anybody has anything else to share?
17:44:11 <mboddu> sharkcz: Oh, I forgot to reply about your Firefox comment, thanks for the info on Firefox builds
17:44:24 <nirik> just to note final freeze is coming up fast.
17:44:36 <nirik> if anyone has work items to get done before then, they basically have to be next week
17:45:21 <nirik> oh, one thing...
17:45:21 <mboddu> nirik: 2 weeks seems a lot in this release cycle :D
17:45:45 <nirik> threebean: whats the status on the bodhi containers stuff? is that the release bowlofeggs is testing now? or still to come later?
17:45:53 <threebean> not quite.
17:46:00 <threebean> the containers stuff was *just* for their CI testing iirc.
17:46:12 <threebean> but he's cutting and deploying a release without modularity stuff today-ish.
17:46:41 <threebean> and the modularity stuff (is passing CI now due to the containers work) .. we're going to start the release process for that on Monday, last I checked.
17:47:04 <nirik> ok, is the idea to get that iin next week? or as soon as it's ready?
17:47:09 * threebean nods
17:47:13 <nirik> or wait until after final freeze?
17:47:19 <threebean> next week - we think it's ready now.  want to get it in before final.
17:47:30 <threebean> of course we'll be cautious and back it out if we smell anything burning.
17:47:33 <nirik> ok, cool.
17:48:06 <nirik> bowlofeggs: is there any messaging planned about the batched updates? we might want to do a devel-announce and some blog posts or something.
17:48:23 <nirik> thats the last thing I had, promise. ;)
17:48:34 <threebean> :)
17:48:36 <mboddu> nirik: hehe :)
17:48:38 <bowlofeggs> nirik: no real progress on running bodhi in containers
17:48:57 <bowlofeggs> nirik: but i started running bodhi's unit tests in containers, which i guess is "slightly" progress
17:49:18 <bowlofeggs> nirik: yes i am planning to write the devel list - i can do a blog post on my planet blog too
17:49:43 <nirik> great. I think devel-announce might be good (some maintainers possibly just read that and avoid devel)
17:50:23 <bowlofeggs> and yeah, i'm still planning to release the modularity thing next week
17:50:30 <bowlofeggs> cool, good point
17:50:57 <bowlofeggs> the modularity release will only have the modularity patch, to reverting will be real easy
17:51:00 <bowlofeggs> (that's on purpose :))
17:51:05 <threebean> +1
17:51:06 <bowlofeggs> no migrations
17:52:49 <mboddu> threebean, bowlofeggs : so, bodhi supporting containers will be rolled out today and support for modularity next week? Is that right?
17:53:17 <maxamillion> wait, wha?
17:53:26 <maxamillion> the container stuff got done?
17:53:33 <maxamillion> (if so, that's awesome!)
17:53:51 <threebean> I don't think that's quite right mboddu.  bowlofeggs, can you clarify?
17:55:02 <bowlofeggs> no no container stuff for bodhi
17:55:11 <bowlofeggs> today is just bodhi-2.11.0
17:55:24 <bowlofeggs> https://bodhi.stg.fedoraproject.org/docs/release_notes.html#id1
17:55:48 <bowlofeggs> i just wanted to separate the modularity patch from the "other stuff" that's been done on bodhi
17:55:52 <bowlofeggs> that's why there are two releases
17:56:12 <bowlofeggs> the modular mashing thing is very hard to test, so i view it as higher risk
17:56:31 <mboddu> bowlofeggs: when you say modularity patch, does that mean bodhi supporting modules?
17:56:50 <bowlofeggs> the only container related work that's been done on bodhi is to run its CI tests in containers, which doesn't really help us run bodhi in containers (and has nothing to do with shipping containers)
17:57:13 <bowlofeggs> like it's literally just running bodhi's unit tests in containers - it's a workaround for my jenkins not having fedora hosts
17:57:24 <bowlofeggs> mboddu: yes, specifically mashing them
17:57:25 <mboddu> bowlofeggs: Oh okay
17:58:36 <mboddu> bowlofeggs: So, bodhi can mash modules after the next week update then?
17:59:22 <nirik> well, pungi them... but yeah
17:59:29 <maxamillion> wait, run bodhi in containers or support containers release in bodhi?
17:59:43 <mboddu> nirik: yeah, bodhi calling pungi to mash modules
17:59:58 <mboddu> maxamillion: I think its just running bodhi in containers
18:00:09 * nirik thinks he started this confusion by typing container when he meant module eariler.
18:00:17 <nirik> there's no container changes.
18:00:32 <nirik> only some tests upstream running in containers. which has nothing to do with us. ;)
18:01:15 <maxamillion> mboddu: oh ok, my mistake
18:01:36 <maxamillion> we going to run bodhi in openshift?
18:01:47 <mboddu> nirik: ahhh, okay, and thats what confused me. "[13:45:45] <nirik> threebean: whats the status on the bodhi containers stuff?" I thought you are talking about bodhi support for containers
18:01:54 <bowlofeggs> mboddu: yes, via pungi
18:01:59 <maxamillion> mboddu: +1
18:02:05 <nirik> yeah, sorry, I meant modules there, not sure why I mistyped.
18:02:39 <maxamillion> nirik: words are hard yo
18:02:48 <mboddu> #info bodhi will support module mashing through pungi in the next week's update
18:02:57 <bowlofeggs> ok so to simplify: there has been no effort towards any container related things for bodhi that this channel would be interested in ☺
18:02:58 <mboddu> nirik, bowlofeggs : ^ is that right?
18:03:04 <bowlofeggs> it's just running bodhi's unit tests, that's all
18:03:05 <nirik> all the interesting word usements I structure.
18:03:16 <threebean> nirik++ wordsmithery.
18:03:34 <bowlofeggs> mboddu: yeah
18:03:37 <nirik> mboddu: hopefully yes there will be an update next week that does that
18:03:54 <mboddu> bowlofeggs: Okay, thats all I need, no more confusion
18:04:03 <mboddu> Anything else?
18:04:31 <mboddu> Okay, we went over the time, so I will end the meeting now
18:04:37 <mboddu> Thanks everyone for joining
18:04:40 <mboddu> #endmeeting