15:59:28 <mboddu> #startmeeting RELENG (2019-09-19)
15:59:28 <zodbot> Meeting started Wed Sep 18 15:59:28 2019 UTC.
15:59:28 <zodbot> This meeting is logged and archived in a public location.
15:59:28 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:59:28 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:59:28 <zodbot> The meeting name has been set to 'releng_(2019-09-19)'
15:59:28 <mboddu> #meetingname releng
15:59:28 <zodbot> The meeting name has been set to 'releng'
15:59:28 <mboddu> #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec
15:59:28 <mboddu> #topic init process
15:59:28 <zodbot> Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz
16:00:06 * nirik is on a call... but sort of here
16:01:26 * dustymabe waves
16:01:31 <dustymabe> .hello2
16:01:32 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:02:13 <mboddu> Since my 2 favorite people are here, lets start the meeting :P
16:02:27 <dustymabe> :)
16:02:35 <mboddu> dustymabe: Haha, you got it :)
16:02:48 <mboddu> #topic #8737 Please workaround koji bug 789
16:02:54 <mboddu> #link https://pagure.io/releng/issue/8737
16:03:33 <mboddu> nirik: So, how hard is it to break the x86_64 into two channels?
16:06:09 <mboddu> I am seeing 40 x86_64 and i386 builders, may be having 30 x86_64 and 10 i386 builders?
16:09:15 <nirik> I am not sure I like this plan.
16:09:57 <nirik> first we should actually test in stg to make sure it works. ;)
16:11:30 <mboddu> nirik: Sure, but I am guessing it should work
16:11:37 <nirik> so, lets do that first... I'd guess we would need 50/50 tho... or near it
16:13:16 <mboddu> nirik: Yeah, you might be right on 50/50
16:13:31 <nirik> pity we still need i686.
16:13:56 * mboddu hates multilib
16:14:18 <jednorozec> multilib is evil
16:14:21 <nirik> yep.
16:14:30 <nirik> sadly, we still need to deal with it. ;)
16:14:44 <mboddu> Right
16:15:10 <nirik> anyhow, proposal: test in stg, also ping koji folks to see if a fix might arrive. revist next week
16:15:16 <mboddu> Hopefully people will stop supporting i686 and start supporting x86_64 (for those who dont do it yet)
16:15:22 <mboddu> ack
16:16:10 <mboddu> #info We will test splitting the x86_64 and i386 builders into seperate channels in stage and also will ping koji folks about an update and then revisit it next week
16:17:50 <mboddu> #topic #8811 method for copying in ostree content to our ostree repos
16:17:56 <mboddu> #link https://pagure.io/releng/issue/8811
16:17:59 * dustymabe waves
16:18:33 <dustymabe> basically we need a fedmsg listener that will import ostree commits when we send a message
16:18:34 <mboddu> So, what are you thoughts on enabling write access to /mnt/koji/ for openshift apps?
16:18:48 <mboddu> nirik: ^
16:18:57 <nirik> I do not like the idea of exporting the entire volume rw for that
16:19:31 <nirik> could we perhaps move to a new volume with just ostree on it?
16:19:32 <mboddu> nirik: I agree, probably we can add rw to /mnt/koji/ostree
16:19:41 <mboddu> +1
16:19:53 <nirik> netapp does not let you export like that. it has to be a seperate volume to have a seperate export policy.
16:20:25 <nirik> I wouldn't think it would be too hard to move... might need a short outage tho?
16:21:25 <mboddu> dustymabe: How soon you want? Can infra plan it in next outage?
16:21:30 <nirik> dustymabe: would that work for you? and would there be a time we could do a short outage?
16:21:30 <mboddu> next planned outage*
16:21:34 <nirik> ha. jinx
16:21:55 <mboddu> Haha :)
16:22:17 <dustymabe> nirik: so basically if we move to a new volume
16:22:23 <dustymabe> then I can get rw in openshift ?
16:22:48 <nirik> yeah, I'd be ok with that... of course you still want to be careful...
16:23:18 <nirik> oh, and could it use fedora-messaging instead of fedmsg?
16:23:29 <dustymabe> yeah everything new uses fedora-messaging
16:23:34 <nirik> great, just checking
16:23:38 <dustymabe> :)
16:23:45 <dustymabe> so regarding moving to a separate mount
16:23:48 <nirik> anyhow, we can discuss schedule in ticket?
16:24:07 <dustymabe> there are a few things that are still using that dir
16:24:15 <dustymabe> like pungi composes for silverblue, etc..
16:24:33 <nirik> yep. we would need to also mount that volume on those machines that access it.
16:24:39 <dustymabe> we either need to update all of the consumers to point to a new location or we need to mount that new volume over the same mount point on all machines
16:24:57 <nirik> bodhi-backend01 for updates, raawhide/branched composers for composes, etc
16:25:14 <nirik> yep. I was planning on the second there. Just mount it on all those machines in the same place.
16:25:37 <mboddu> 2nd is the easiest to do
16:26:02 <dustymabe> so we do use two different directories
16:26:21 <dustymabe> one is the compose ostree repo and one is the prod ostree repo
16:26:24 <dustymabe> let me look at the file paths
16:26:42 <mboddu> compose ostree repo is /mnt/koji/compose/....
16:26:47 <dustymabe> /compose/ostree
16:26:56 <mboddu> prod ostree repo is /mnt/koji/ostree
16:26:59 <dustymabe> and /ostree
16:27:07 <dustymabe> yeah
16:27:32 <nirik> oh yeah. I was thinking just the compose one... but you would need both right?
16:27:36 <dustymabe> so it won't be super easy to put them both on one volume
16:27:45 <nirik> we could do 2 volumes...
16:27:45 <dustymabe> we might have to update some file paths
16:28:13 <nirik> but if we could put them on one we would get dedupe...
16:28:35 <dustymabe> nirik: that would be valuable
16:28:46 <nirik> yeah, they should mostly be the same most of the time right?
16:28:56 <dustymabe> nirik: ok let me think about this a bit and get back to you in the ticket
16:29:07 <dustymabe> two pieces of information for me for now
16:29:14 <dustymabe> 1. when is the next scheduled outage ?
16:29:27 <nirik> sure. side note: we would need to adjust the cloudfront setup if we move paths too.
16:29:34 <dustymabe> 2. can I get an NFS volume created today for testing everything out
16:29:46 <mboddu> dustymabe: Why do you need write access to compose ostree repo? Isn't updated by pungi composes?
16:29:52 <nirik> we haven't scheduled any currently... we could do this as it's own outage...
16:30:42 <nirik> I guess I can make you a stg one sure. :) Just file a ticket on it so I don't forget.
16:31:11 <nirik> or communishift should be similar... thats also nfs storage
16:32:03 <dustymabe> yeah I may be able to use that
16:32:17 <dustymabe> nirik: I'll update the ticket with our musings
16:32:20 <mboddu> dustymabe: Oh, right, coreos uses different compose process...
16:32:30 <dustymabe> mboddu: right
16:32:31 * mboddu needs more tea
16:32:45 <nirik> dustymabe: cool. Thanks
16:32:57 <dustymabe> nirik: mboddu I have to run to the fcos IRC meeting
16:33:08 <mboddu> dustymabe: Sure, thanks for joining
16:33:08 <dustymabe> I do still have a few outstanding items if you would please discuss: https://github.com/coreos/fedora-coreos-tracker/blob/master/Fedora-Requests.md#existing-requests-for-fedora-releng
16:34:01 * nirik doesn't know the agenda, we have like 20 tickets with meeting on them. ;)
16:34:26 * mboddu checks
16:35:03 <nirik> I might just try and manually fix the dist-repo thing now that we are out of freeze.
16:35:44 <mboddu> #info We decided to create a separate volume for just ostree stuff with rw access and dustymabe will test it in stg first
16:37:36 <mboddu> The other two are related to koji
16:37:44 <mboddu> nirik: Which dist-repo thing are you talking?
16:37:56 <mboddu> https://pagure.io/koji/issue/1630 or https://pagure.io/koji/issue/1637
16:38:12 <nirik> 1637
16:39:01 <mboddu> nirik: Let me know how that goes?
16:39:38 <nirik> yeah, I was gonna check the db in stg (where the perm exists) against prod and see what needs to be added for it. Might just be a insert into the permissions table.
16:42:02 <mboddu> nirik: I hope its as easy as that
16:42:51 <mboddu> nirik: "koji grant-permission dist-repo mohanboddu --new" should add the permission
16:43:31 <nirik> yeah, I don't think that worked, but can try again...
16:44:11 <nirik> but no need to do so now... any other meeting items? :O)
16:45:50 <mboddu> #topic #7555 scratch permission to build livecd for users
16:45:58 <mboddu> #link https://pagure.io/releng/issue/7555
16:46:05 <mboddu> So, I am testing it today
16:46:26 <mboddu> nirik: And we want to implement this sometime soon
16:46:49 <mboddu> (Having spins/labs having their own release cadence requires this)
16:47:22 <mboddu> We can give livemedia/image/appliance perms to certain users from their respective sigs/wgs
16:47:40 <nirik> I guess so... but still needs putting it in place and updating other things like websites/torrents?
16:48:07 <mboddu> nirik: Sure, thats a whole lot different problem
16:48:25 <mboddu> In my ideal world
16:50:01 <mboddu> The spin/lab maintainer will press a button which will build the image using base+updates repo and then they will test it and then push another button which will put it in /pub/somewhere/ (since its not directly related to the release compose) and updates the websites
16:50:13 <mboddu> And my ideal world also dont have torrents :)
16:52:07 <nirik> :)
16:52:12 * nirik has a plan for torrents.
16:53:11 <mboddu> nirik: Like...?
16:53:20 <nirik> get amazon to do them for us. ;)
16:53:35 <mboddu> Yeah, I think we discussed this sometime :D
16:53:35 <nirik> we already mirror to amazon, and it can do torrent on a s3 bucket.
16:54:01 <mboddu> Yeah, that should work
16:54:02 <nirik> just need to gather the torrent links somewhere and somehow and publish them somewhere
16:54:14 <mboddu> Yup
16:55:37 * nirik has another meeting in 5
16:55:44 <mboddu> nirik: So, what do think of creating a app that uses fedpkg/koji to build the image and then create the tickets where ever necessary to release the image
16:55:49 <mboddu> ?
16:55:57 <mboddu> Oh, I forgot about signing
16:56:21 <mboddu> ^^ That idea is something we can start with and then expand it later
16:56:55 <nirik> I think a better model might be to have a way for maintainer to fire off, then do everything with fedmsgs, and have a way for the maintainer at the end to send a 'release-this' message that also publishes, etc.
16:57:08 * nirik would really avoid tickets or manual actions unless we need to
16:57:55 <mboddu> nirik: Sure, thats the final goal for it
16:58:18 <mboddu> As is my ideal world :D
16:59:19 <mboddu> Some of things are dependent on other stuff, like signing, it isn't automated right now, when its automated then we can avoid manual signing
17:01:13 <mboddu> Okay, its getting over the scheduled time, we can plan how it should work later
17:01:25 <mboddu> Thanks everyone for joining
17:01:32 <mboddu> #endmeeting