17:01:15 <mboddu> #startmeeting RELENG (2018-06-21)
17:01:15 <zodbot> Meeting started Thu Jun 21 17:01:15 2018 UTC.
17:01:15 <zodbot> This meeting is logged and archived in a public location.
17:01:15 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:15 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:01:15 <zodbot> The meeting name has been set to 'releng_(2018-06-21)'
17:01:15 <mboddu> #meetingname releng
17:01:15 <zodbot> The meeting name has been set to 'releng'
17:01:15 <mboddu> #chair nirik tyll sharkcz masta pbrobinson pingou puiterwijk maxamillion mboddu Kellin dustymabe
17:01:15 <mboddu> #topic init process
17:01:15 <zodbot> Current chairs: Kellin dustymabe masta maxamillion mboddu nirik pbrobinson pingou puiterwijk sharkcz tyll
17:01:43 <nirik> morning
17:01:52 <sharkcz> heya, /me is here this week :-)
17:02:53 <mboddu> Hello guys
17:02:59 <mboddu> cverna: Are you around here today?
17:04:36 <nirik> kinda late for cverna...
17:04:38 <mboddu> Okay, lets get started
17:04:58 <mboddu> nirik: Yeah, but we are going to talk at 18:00 UTC later today, so I thought he will be around now
17:05:11 <nirik> oh, alright
17:05:44 <mboddu> since sharkcz is here, lets get started with alt arches
17:05:53 <mboddu> #topic Alternative Architectures updates
17:06:18 <mboddu> nirik, sharkcz : Any ideas on why s390x runroot tasks are getting hung?
17:06:46 <nirik> for some reason the network link is really really slow...
17:06:54 <nirik> I have no idea why, but was going to inquire.
17:07:32 <mboddu> nirik: Okay, thanks for tracking it
17:07:58 <sharkcz> slow network could explain that, it might be related to the quite high memory overcommit for the VMs, it should get better
17:08:20 <nirik> there have also been a lot of big jobs of late.
17:08:29 <sharkcz> after the resources from the secondary lpar are reused in the primary lpar
17:08:34 <nirik> like yesterday I looked and 7 of the builders were building openjdk
17:09:25 <mboddu> #info Network link between our infra site and s390x vms is really really slow. Kevin is looking into it
17:09:31 <sharkcz> would be good to have performance numbers for the lpar utilization
17:09:45 <mboddu> sharkcz: True, I am very excited about it, I hope it will happen soon
17:09:49 <nirik> yeah, I did some pinging internally...
17:11:00 <mboddu> Anyway, lets get moving
17:11:17 <mboddu> I dont have anything other than that, so I moving to open floor
17:11:27 <mboddu> #topic Open Floor
17:11:36 <mboddu> This is going to be short meeting
17:11:48 <Kellin> mboddu: we had the thing we tagged in stand up
17:12:29 <nirik> I was going to ask about 2 things. ;) want me to go or Kellin you want to go first?
17:12:47 <mboddu> nirik: You go ahead, I have to find that ticket
17:13:21 <nirik> 1) was there any progress on adding templates or other pagure customizations ?
17:13:42 <nirik> 2) did one of you want to try a non root test compose so we can figure out what needs tweaking if anything...
17:13:52 <Kellin> nirik: yes, we have templates for 3 tawsks
17:13:54 <Kellin> tasks...
17:14:12 <Kellin> and yes, where do you want to try the non-root compose, stage or prod?
17:14:18 <Kellin> (I prefer stage)
17:14:27 <mboddu> nirik: Yes, I added 4 templates, with 1 generic one
17:14:54 <nirik> oh, excellent. :) another new thing you could also leverage is the 'quick responses'
17:15:13 <nirik> Kellin: we can try stg, but I don't think it will give us a real valid test for prod...
17:16:35 <Kellin> Hmm, OK
17:18:23 * cverna is around now
17:18:48 <mboddu> cverna: Hello
17:19:11 <cverna> mboddu: o/
17:19:44 <mboddu> #info RelEng added templates to issues and we will look at adding more customizations using 'quick responses'
17:21:45 <mboddu> Kellin: Do you wanna try the rootless pungi runs?
17:22:31 <Kellin> mboddu: sure. that works for me
17:22:47 <mboddu> Kellin: Awesome
17:23:19 <mboddu> #info Kellin is going to work on testing the pungi runs without using root permissions
17:24:10 <mboddu> Now, this is different, go over meeting topics after open floor :D
17:24:16 <Kellin> heh
17:24:19 <nirik> meh, it happens.
17:24:21 <Kellin> wellw e have a lot of them
17:24:22 <mboddu> #topic #7534 push all arches base images info container registry.
17:24:26 <Kellin> we should take any spare time and keep clearing those out
17:24:38 <mboddu> #link https://pagure.io/releng/issue/7534
17:24:59 <mboddu> Kellin, nirik : :)
17:25:16 <mboddu> cverna: Okay, your turn :)
17:25:56 <cverna> ok
17:26:21 <mboddu> cverna: We are using https://pagure.io/releng/blob/master/f/scripts/sync-latest-container-base-image.sh#_79 to download just x86_64 images and pushing them to our registry
17:26:28 <cverna> so from what I understood we are pushing multi arch images to docker hub but not to our internal registry
17:27:06 <mboddu> So, if we just change that to download other arch images and push them, does our registry is capable to understand the arches and work with them properly?
17:27:16 <cverna> yes so we should add support to push other architecture
17:27:44 <cverna> mboddu: I believe so since we are using the plain docker registry
17:28:05 <cverna> mboddu: we can give it a try in stg and see how that goes
17:28:27 <mboddu> cverna: hehe, you read my mind, +1 trying it stg
17:29:37 <cverna> what we or I can do is open a PR on the script and test it on stg once we are happy with it merge it and do it in prod
17:30:25 <mboddu> cverna: I think we dont need to anything to that script, if we provide -s option it will do stage release
17:30:39 <mboddu> cverna: https://pagure.io/releng/blob/master/f/scripts/sync-latest-container-base-image.sh#_62
17:31:01 <cverna> mboddu: we need to get the other arch image from koji
17:31:10 <mboddu> cverna: Oh yeah, right
17:32:19 <cverna> mboddu: do you know how the images are pushed to the docker hub ?
17:33:02 <mboddu> #info RelEng and cverna will work together in testing this in stage by updating the sync-latest-container-base-image.sh to add support for other arches
17:33:34 <mboddu> cverna: If there is anything other than docker tag and docker push, then I dont know
17:35:27 <cverna> mboddu: so this is a manual process ? or is there a script ?
17:35:39 <mboddu> cverna: Thats the part of the script
17:35:49 <mboddu> cverna: https://pagure.io/releng/blob/master/f/scripts/sync-latest-container-base-image.sh#_88
17:36:29 <cverna> mboddu: oh ok I meant the images we are pushing on https://hub.docker.com/
17:37:14 <mboddu> cverna: Sorry, I overlooked at your first message
17:37:25 <cverna> no worries
17:37:40 <mboddu> cverna: I dont know that process, only puiterwijk and Adam M knows about it
17:38:02 <puiterwijk> cverna: the process for that is: ping me
17:38:11 <puiterwijk> (or Adam, but mostly me these days)
17:38:12 <mboddu> IIRC, there is a github repo with the scripts that does it
17:38:25 <puiterwijk> mboddu: yeah, but only me and Adam have access to do that
17:38:35 <puiterwijk> So just ping me if you want stuff updated on docker hub
17:38:40 <puiterwijk> I try to do it myself as well every so often
17:38:57 <cverna> puiterwijk: I was just wondering how the multi arch is done
17:39:18 <cverna> puiterwijk: since it is working on docker.io but not in our registry
17:39:21 <mboddu> puiterwijk: Just like keeping things to yourself, just like your yubikeys and not giving to us :P
17:39:29 <cverna> puiterwijk: I wanted to do it the same way
17:39:49 <puiterwijk> cverna: https://github.com/fedora-cloud/official-images/blob/master/library/fedora
17:39:56 <puiterwijk> The docker hub process is really manual
17:40:13 <puiterwijk> So doing it the same way is... not what we'd want
17:40:32 <puiterwijk> Basically, I push a repo with all the raw tarballs, then send them a PR for this, they merge it, and htat pushes to docker hub
17:41:34 <cverna> ok yeah not ideal
17:42:14 <puiterwijk> The Docker folks wrote bots to grab this and push tsuff
17:42:15 <puiterwijk> stuff
17:42:58 <cverna> what was not really clear to me was how to tag the other arch since I guess we want all the images to be name fedora
17:44:57 <mboddu> cverna: I think I can get some help with that, but no promises
17:45:46 <cverna> mboddu: sounds good :)
17:48:44 <mboddu> cverna: So, from what I understood, manifest lists is what we need to use, but I need more help
17:49:51 <mboddu> cverna: From docker docs - https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list-field-descriptions
17:49:57 <mboddu> hehe, docker docs :D
17:50:04 <mboddu> Anyway, lets move on to the next topic
17:50:49 <mboddu> #topic #6230 sign our docker images
17:50:57 <mboddu> #link https://pagure.io/releng/issue/6230
17:51:56 <mboddu> puiterwijk: If you are still around, can you tell us briefly about the container image signing?
17:52:08 <mboddu> nirik: ^ If you know anything that will help as well
17:52:24 <nirik> I don't really know much about it.
17:53:08 <mboddu> nirik: Okay
17:53:17 <puiterwijk> mboddu: I can later.
17:53:45 <puiterwijk> But basically, sigul supports signing, we have a place where we place signatures, but autosigning is only in the next robosignatury
17:53:48 <puiterwijk> robosignatory*
17:53:57 <puiterwijk> So we have the capability, we don't autosign at the moment
17:54:20 <puiterwijk> Do note that we are not using the docker signing system, but rather the RH signing system, as used by skopeo
17:54:21 <mboddu> puiterwijk: Okay, we can go over it when you have some free time :)
17:54:31 <puiterwijk> Well, I'd say that that's the summary
17:55:21 <mboddu> puiterwijk: Okay, just one quick question, when are they getting signed?
17:55:33 <puiterwijk> mboddu: as I said, right now, they don't.
17:55:45 <puiterwijk> With new robosig, they will be as soon as bodhi picks them up.
17:55:56 <puiterwijk> They'll be part of the standard autosigning-on-bodhi-submission
17:56:27 <mboddu> puiterwijk: Ohh, "we don't autosign at the moment" I thought you are signing them manually at some point
17:56:43 <mboddu> puiterwijk: Cool
17:56:48 <mboddu> Thanks for the info puiterwijk
17:56:50 <puiterwijk> mboddu: no, we don't sign at all
17:56:56 <puiterwijk> We *can*. but we don't yet
17:57:05 <mboddu> Got it, thanks :)
17:58:57 <mboddu> #info With upcoming new robosig, we will start supporting container signing and they will signed just like what we have with rpms today, containers will be signed when they are submitted to bodhi.
17:59:08 <mboddu> Anybody has anything else?
17:59:24 <mboddu> Oh we are out of time too
17:59:32 <mboddu> I thought this is going to be a short meeting
17:59:35 <Kellin> just we should get to the rest of our meting tagged stuff
17:59:48 <Kellin> so we should try, if we end early, to hit those and clear that backlog
17:59:54 <Kellin> it will help get rid of clutter
17:59:57 <mboddu> Kellin: +1
18:00:15 <mboddu> Anyway, thank you all for joining
18:00:20 <mboddu> #endmeeting