15:05:12 <jednorozec> #startmeeting RELENG (2020-08-04)
15:05:12 <zodbot> Meeting started Tue Aug  4 15:05:12 2020 UTC.
15:05:12 <zodbot> This meeting is logged and archived in a public location.
15:05:12 <zodbot> The chair is jednorozec. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:05:12 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:05:12 <zodbot> The meeting name has been set to 'releng_(2020-08-04)'
15:05:31 <jednorozec> #meetingname releng
15:05:31 <zodbot> The meeting name has been set to 'releng'
15:05:40 <jednorozec> #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec
15:05:40 <zodbot> Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz
15:05:57 <mboddu> Thanks jednorozec :)
15:06:10 <jednorozec> np
15:06:13 <nirik> morning
15:06:25 <mboddu> Morning nirik
15:06:32 <sharkcz> hi
15:07:11 <pingou> ó/
15:08:18 <mboddu> Hello everyone
15:09:14 <mboddu> #topic #9577 Fedora Media Writer 4.1.6 released
15:09:21 <mboddu> jednorozec: Whats the status on it??
15:09:58 <mboddu> Also nirik ^
15:10:13 <nirik> sigh. I haven't had time to get back to this.
15:10:28 <nirik> I need to find the key and then jednorozec was going to sign the windows side.
15:10:36 <jednorozec> mboddu, I have updated the SOP. But didnt test it
15:10:38 <nirik> and I need to dig out my mac to get the macos side done
15:10:45 <nirik> so, ENOTIME
15:10:52 <jednorozec> basicaly it is just different conversion of the certificate
15:11:29 <mboddu> jednorozec: Can you give it a try this week?
15:11:45 <mboddu> nirik: I understand and sorry I didn't have my mac with me :(
15:12:07 <jednorozec> mboddu, I need key to the certificate and then it should be just about running the commands and script.
15:12:09 <mboddu> Even though if I had, I dont remember the password and stuff :(
15:12:12 * nirik spent yesterday almost enturely on s390x builders.
15:12:52 <mboddu> jednorozec: Oh, I thought you already got the key
15:13:02 <jednorozec> mboddu, I have certificate
15:14:18 <jednorozec> Need key+certificate pair to generate PKCS12 certificate that is used to sign the binaries
15:16:59 <mboddu> nirik: ^ Is it blocked on internal or the cert provider?
15:17:27 <nirik> I need to find the key. I don't think I ever had it. I need to figure out how to get it.
15:17:32 <nirik> so yes.
15:19:55 <mboddu> nirik: And iirc, we might have to renew the apple developer program thingy (which renews the cert as well)?
15:20:34 <nirik> could be yeah
15:21:36 <mboddu> It will be interesting...
15:24:23 <mboddu> #topic #9451 rework koji channels / policy
15:24:29 <mboddu> #link https://pagure.io/releng/issue/9451
15:24:44 <mboddu> nirik: So, you think we should try it? with the koji update planned for tomorrow? :D
15:25:05 <nirik> uh, we can discuss it...
15:25:32 <mboddu> Okay
15:25:48 <mboddu> nirik: Now or tomorrow?
15:26:12 <nirik> either way. I'd have to try and recall what we were trying to do with the s390x stuff...
15:26:28 <nirik> and I guess we need to pull up stats and look at how utalized things are.
15:26:41 <nirik> koji list-channels
15:26:44 <nirik> gives you some of that
15:27:28 <nirik> https://koji.fedoraproject.org/koji/clusterhealth also has some
15:28:14 <nirik> we should probibly delete eclipse, images, maven and vm
15:29:20 <mboddu> And I dont know what testchannel is for
15:29:38 <nirik> oh yeah that can go too
15:30:16 <nirik> we had started to try and get it so s390x builders only did s390x tasks... no noarch and no srpmbuild... but I don't recall how we were going to do that exactly.
15:30:49 <nirik> (thats the build-arch and s390x-build ones)
15:31:53 <nirik> heavybuilder is for chromium. it has only 2 aarch64 builders in it (the ones with tons of cpus/memory)
15:32:45 <mboddu> Cool
15:33:13 <mboddu> I was looking at the koji hub policy, but nothing documented on how to handle srpm builds
15:33:44 <nirik> yeah, we came up with some clever way around that, but it's so clever I can't remember it now. ;)
15:33:53 <mboddu> There is a 'build_from_srpm' but not specific to srpm build
15:34:03 <mboddu> nirik: Haha, lol :D
15:36:25 <nirik> ah, you can match on method...
15:37:15 <mboddu> Ohhhh
15:37:42 <nirik> so we need a if buildSRPMFromSCM <channel with all the builders except s390x>
15:38:06 <mboddu> Right
15:38:14 <mboddu> Before we forget, I am updating the ticket
15:38:49 <nirik> so, might be we can just re-use createrepo channel.
15:39:08 <nirik> it's got all the x86 and ppc64le's
15:41:23 <nirik> -    method newRepo distRepo :: use createrepo
15:41:23 <nirik> +    method newRepo distRepo buildSRPMFromSCM :: use createrepo
15:41:30 <mboddu> It should also aarch64 builds
15:41:54 <nirik> hum?
15:43:02 <nirik> buildSRPMFromSCM is just the job at the start where it makes the src.rpm from git... it then passes that to all the archfull tasks to do the builds on the various arches
15:44:39 <nirik> but I guess there could be cases where if we restrict it, people have condtionals only for s390x or something, but thats against guidelines.
15:45:17 <mboddu> nirik: How come they have conditionals on srm.rpm generation?
15:45:36 <nirik> "
15:45:36 <nirik> Packages MUST NOT have Source: or Patch: tags which are conditionalized on architecture. For example, this is forbidden:
15:45:37 <nirik> "
15:45:50 <nirik> it's possible, just forbidden.
15:46:40 <nirik> so, I think it's safe to try that
15:46:55 <mboddu> nirik: In that case, it enforces the rule, which in my book is another +1 :d
15:47:11 <mboddu> Cool, lets try that
15:47:44 <mboddu> #info We will move buildSRPMFromSCM tasks to use createrepo channel which has x86_64, ppc64le and aarch64 builders
15:47:59 <mboddu> #topic PDC Backends
15:48:02 <mboddu> pingou:
15:48:08 <mboddu> You are up
15:48:20 <pingou> so we have currently 3 backends machines
15:48:29 <pingou> that are running pdc-updater + their audit cron job
15:48:40 <pingou> I could find 2 audit cron job sending to the releng-cron list
15:48:44 <pingou> 1 that backtraces
15:48:50 <pingou> the other is the import compose
15:48:55 <nirik> I think one yeah... doesn't know about flatpak's
15:49:12 <pingou> the pdc-updater handler (that consumes fedmsg) have been ported to toddlers/fedora-messaging
15:49:49 <pingou> I would like to: poweroff all three machines, git rm roles/pdc_backend, virsh destroy 02/03 and lvm_remove 02/03
15:50:07 <pingou> that leaves 01, shutdown but available to reboot if needed
15:50:32 <nirik> sure... on 01, do also a 'virsh autostart disable' so it doesn't come up if the virthost reboots.
15:50:41 <pingou> will do
15:50:42 <nirik> +1 from me
15:50:43 <mboddu> pingou: I would say poweroff, git rm roles/pdc_backend and wait for couple of days
15:50:49 <mboddu> And then nuke them
15:50:53 <pingou> wfm
15:51:04 <pingou> I'll autostart disable them all
15:51:22 <pingou> in case we want to reboot any hosts over the week-end or so
15:51:30 <pingou> and I'll nuke 02/03 sometime next week
15:51:34 <mboddu> Cool
15:51:39 <nirik> sounds good
15:51:52 <pingou> thanks
15:52:03 <mboddu> nirik: ^ So, do we get some resources when we nuke them? Wondering if we could add some more memory to odcs releng backend?
15:52:09 <mboddu> And thanks pingou for your help
15:52:11 <mboddu> pingou++
15:53:02 <pingou> sure thing :)
15:53:14 <nirik> mboddu: how much more do you need?
15:54:02 <mboddu> nirik: Eventually I want it to match rawhide/branched composers
15:54:23 <pingou> we want to keep the pdc_backend machines in the inventory of ansible until next week, right?
15:54:46 <nirik> pingou: I'd say no, because then nagios is going to annoy us, so remove them now?
15:54:50 <nirik> we can always revert that
15:55:13 <mboddu> pingou: When you rim the role, remove them from inventory as well
15:55:17 <pingou> ok
15:55:18 <mboddu> s/rim/rm/
15:55:40 <pingou> next week, we nuke the disk/images of all three or just 02/03?
15:56:03 <mboddu> pingou: I would say start with 02/03
15:56:11 <mboddu> Give sometime for nuking 01
15:56:23 * mboddu is being cautious
15:56:27 <nirik> frankly we could just leave the images around until we need space.
15:56:40 <mboddu> That works too
15:56:43 <nirik> just undefine the libvirt part.
15:56:44 <pingou> so no virsh undefine and lv_remove?
15:56:54 <pingou> ok just virsh undefine
15:56:54 <nirik> undefine yes, no lvremove
15:57:24 <pingou> still only for 02 and 03, right?
15:57:29 * pingou putting this down in the ticket
15:58:10 <nirik> I'm ok with dropping them all, but keeping the lv's around from them all. If we need anything we can mount the lv... and if we need one back we can make a new vm
15:58:36 <pingou> mboddu: wfm as well? ^
15:59:18 <mboddu> pingou: Yup, works for me
15:59:41 <mboddu> If we are not nuking them, we can bring them back up, just an extra step, thats all :)
15:59:42 <pingou> cool: https://pagure.io/fedora-infrastructure/issue/9181#comment-669119
15:59:51 <pingou> thanks for your help
16:00:00 <mboddu> Okay, couple of quick things
16:00:03 <mboddu> #topic Open Floor
16:00:12 <mboddu> #info ftbfs is filed for F33 mass rebuild
16:00:51 <mboddu> #info mass rebuild of modules is still running, should be done sometime today (hopefully)
16:01:13 <nirik> mbs got working again just in time. ;)
16:01:26 <pingou> \ó/
16:01:39 <mboddu> Haha :)
16:02:53 <mboddu> #info mbs is not working again
16:02:55 <nirik> I only see 1 left from releng
16:03:01 <mboddu> Any other updates before I close?
16:03:15 <nirik> no no, mbs _is_ working now?
16:03:25 <mboddu> #undo
16:03:25 <zodbot> Removing item from minutes: INFO by mboddu at 16:02:53 : mbs is not working again
16:03:38 <pingou> lol
16:04:03 <mboddu> Sorry, when you I read 'mbs' and 'working' together, my mind is defaulted to 'not working' :D
16:04:07 <nirik> ha
16:04:15 <jednorozec> haha
16:04:42 <mboddu> nirik: module builds work in a different way, they cant just submit and leave, they have to be submitted one after the other
16:04:50 <mboddu> Unlike the package builds
16:04:56 <nirik> thats... anoying
16:05:36 <mboddu> nirik: Its due to https://pagure.io/releng/issue/8664
16:06:13 <mboddu> Anyway, thats all I got
16:06:46 <mboddu> Its overtime, sorry guys
16:06:53 <mboddu> Lets catch up next week
16:06:58 <mboddu> Thanks everyone for joining
16:07:17 <mboddu> If I missed anything, we can discuss it in #fedora-releng channel
16:07:20 <mboddu> #endmeeting