16:01:08 <mboddu> #startmeeting RELENG (2020-12-08)
16:01:08 <zodbot> Meeting started Tue Dec  8 16:01:08 2020 UTC.
16:01:08 <zodbot> This meeting is logged and archived in a public location.
16:01:08 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:08 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:01:08 <zodbot> The meeting name has been set to 'releng_(2020-12-08)'
16:01:08 <mboddu> #meetingname releng
16:01:08 <zodbot> The meeting name has been set to 'releng'
16:01:09 <mboddu> #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec
16:01:09 <zodbot> Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz
16:01:09 <mboddu> #topic init process
16:01:22 <jednorozec> hello
16:01:40 <sharkcz> hi all
16:01:52 <nirik> morning
16:02:36 <pingou> ó/
16:03:26 * nirik may have to step out in about 45min...
16:06:44 <mboddu> Okay, lets get the party started
16:07:08 * mboddu is also in other meeting
16:07:15 <mboddu> #topic #9893 broken epel8-infra-stg distrepo
16:07:20 <mboddu> #link https://pagure.io/releng/issue/9893
16:07:28 <mboddu> I untagged bunch of old builds
16:07:43 <mboddu> But I dont know why https://pagure.io/fedora-infra/ansible/blob/master/f/roles/koji_hub/templates/koji-gc.conf.j2#_147 is not working
16:08:10 <nirik> because oscar doesn't have infra privs.
16:08:19 <mboddu> nirik: Good catch
16:08:37 <mboddu> Can I give it to oscar or is there any reason why we didn't give it?
16:08:59 <mboddu> Or nirik just did it
16:09:05 <mboddu> $ koji list-permissions --user oscar                                                          325ms  Tue 08 Dec 2020 04:05:57 PM UTC
16:09:06 <mboddu> admin
16:09:06 <mboddu> infra
16:09:06 <mboddu> autosign
16:09:39 <nirik> ok.
16:09:44 <nirik> we can see if that fixes it...
16:10:14 <nirik> Untagging build ipsilon-2.0.2-1.el7 from epel7-infra
16:10:14 <nirik> Warning: untag operation failed: policy violation (tag)
16:10:24 <nirik> (thats from the koji-gc output)
16:13:29 <mboddu> ^ after giving the permission or from before?
16:14:00 <nirik> no, from last night
16:14:54 <mboddu> Okay, thats good :)
16:18:24 <mboddu> #topic #9629 PDC: do we want to store the EPEL compose?
16:18:28 <mboddu> #link https://pagure.io/releng/issue/9629
16:18:45 <mboddu> Lets make a decision for PDC and post PDC
16:18:57 <mboddu> I feel like we can leave it as is for now
16:19:15 <mboddu> And when we have a PDC replacement, we can start tracking epel composes as well
16:19:18 * nirik doesn't feel strongly about it. If we aren't using it...
16:19:59 <pingou> it goes to the question of: what are we tracking composes for?
16:20:03 <pingou> what's the goal of this?
16:20:25 <jednorozec> that was going to be my question as-well
16:20:49 <jednorozec> what is the reason for storing compose info in PDC
16:21:21 <mboddu> I wanted to use it as a way to figure out whats changed between composes
16:21:30 <nirik> well, I know fedfind uses them? and openqa perhaps?
16:21:44 <mboddu> And QA uses them
16:21:50 <nirik> and dist-git uses them for eol hook
16:22:05 <pingou> not the composes (dist-git)
16:22:07 <mboddu> Not the composes
16:22:14 <pingou> and fedfind uses the rawhide composes only iirc
16:22:15 <nirik> ah true
16:22:18 <mboddu> It uses the component-branch-slas for it
16:22:37 <mboddu> pingou: Well, rawhide and branched
16:22:56 <mboddu> Whatever the compose QA has to test
16:23:16 <mboddu> Hence EPEL was never imported
16:26:17 <nirik> so, whats the thought here... try and get rid of pdc entirely? :)
16:26:37 <jednorozec> well we will replace PDC in 2021
16:26:50 <mboddu> ^ I sure hope so
16:26:59 <nirik> jednorozec: can I quote you on that at the end of next year? ;)
16:27:01 * pingou has it on his list
16:27:17 * jednorozec has different list but PDC is there
16:27:33 <nirik> well, I am just wondering if we could offload what it does to other places...
16:27:57 <nirik> but just a thought.
16:28:04 * mboddu wants to take a look at kinto or postgrest
16:28:39 <jednorozec> I think we can offload some stuff, but there has to be a place of truth for EOL and such
16:28:44 <mboddu> Anyway, so, lets leave it as is and then look at importing other composes when we have a replacement for PDC
16:28:48 <nirik> also adam went and started a release events thing (did everyone see that?)
16:28:59 <mboddu> Yup
16:29:01 <jednorozec> nirik, nope do you have link?
16:29:20 <nirik> https://pagure.io/releng/issue/9867
16:29:44 <mboddu> But that tracks the entire release EOL (which works for rpms in our usecase) but it cannot be used for modules
16:29:53 <pingou> nice book :)
16:30:03 <mboddu> pingou: Lol :)
16:30:05 <nirik> https://pagure.io/fedora-qa/releasestream/
16:31:33 <nirik> anyhow, sure, I am fine just not doing epel now
16:32:35 <pingou> +1 on closing the ticket and re-evaluate when we replace pdc
16:32:41 <jednorozec> +1
16:32:50 <nirik> sure
16:34:13 <jednorozec> mboddu, nirik pingou BTW I already wrote to amoloney and we should start scoping it for Q2
16:34:20 <jednorozec> PDC replacement
16:34:32 <mboddu> +1
16:34:33 <pingou> +1
16:34:38 <nirik> sure. Cool. ;)
16:35:06 <jednorozec> I will take the shame if its not replaced next year
16:35:11 <pingou> I think we should get either datagrepper/datanommer or PDC on Q2, they are kinda similar and both needs doing
16:36:38 <nirik> and FMN. :)
16:37:22 <mboddu> And everything else :)
16:38:47 <pingou> FMN is a beast on its own
16:38:59 <pingou> while the other two should more or less fit in a quarter
16:39:56 <mboddu> I am not sure about PDC
16:40:00 * jednorozec is not sure about datagrepper but PDC should not be too complex
16:40:18 <pingou> ^^
16:40:23 <mboddu> I think internally it took them more than a year, but maybe this time we know what we need
16:40:38 <pingou> it depends on the technical solution we go for I guess
16:40:42 <jednorozec> yeah but PDC is really complex and we don't use all the features...
16:40:44 <mboddu> Yup
16:41:05 <mboddu> Right, I totally agree, but I am just saying :)
16:41:09 <pingou> that's the nice part about the new solution, we'll get to drop everything we don't use
16:41:12 <jednorozec> BTW can I get nginx/apache logs for PDC?
16:41:35 * jednorozec is interested in what API endpoints we actually use and how often
16:41:43 <mboddu> jednorozec: Can you ssh into pdc-backend01.iad2.fp.o ?
16:42:33 <nirik> we don't have backends anymore. ;)
16:42:35 * jednorozec tries
16:42:37 <nirik> pdc-web01 :)
16:43:07 <jednorozec> ok thanks I will search the log there
16:46:19 <mboddu> Ah right, sorry, my bad
16:46:31 * mboddu juggling between meetings
16:46:33 <mboddu> Anyway
16:46:39 <mboddu> #topic Open Floor
16:46:45 <mboddu> Has anybody got anything to share?
16:46:58 <mboddu> #info nirik updated the builders (except arm) to F33 over the weekend
16:47:03 <jednorozec> mboddu, what do you think about the compose jenkins thing
16:47:21 <jednorozec> that jkaluza made
16:47:29 <mboddu> Although that broke container composes for some unknown reason :P
16:47:36 <mboddu> jednorozec: Right, I really liked the idea
16:47:41 <jednorozec> I was looking at it today and it looks good
16:47:49 <mboddu> I asked couple of questions to jkaluza, but he didn't respond yet
16:47:54 <jednorozec> and also we wont have to maintain another service
16:47:59 <nirik> Yeah, if anyone has ideas on the broken container builds I am all ears. ;(
16:48:18 <nirik> https://pagure.io/releng/issue/9890 has the traceback. ;(
16:48:29 <mboddu> nirik: At this point of time, I think it might be py3.9?
16:49:30 <nirik> I doubt it...
16:50:34 <nirik> might be a change in libvirt that we need to adjust for? but I am not sure at all... need someone more python savvy to trace it
16:50:34 <jednorozec> well I have seen such errors when the VM was not configured properly
16:50:59 <jednorozec> expecting network, but not geting it or storage and not getting it
16:51:31 <nirik> default network is there. ;(
16:53:42 <jednorozec> nirik, imagefactory plugins?
16:54:19 <nirik> checked them, fixed them... still broke
16:54:23 <pwhalen> we hit that error on the iot disk images today - https://koji.fedoraproject.org/koji/taskinfo?taskID=57046122
16:54:26 <nirik> something with oz perhaps
16:54:35 <nirik> pwhalen: so, likely the same thing...
16:55:19 <pwhalen> the disk images worked ok in the mainline compose, could help track it down
16:55:24 <jednorozec> nirik, can one ssh to the builder and runt the imagefactory by hand?
16:56:22 <nirik> pwhalen: all the containers in the main compose failed that way tho
16:56:27 <nirik> jednorozec: perhaps...
17:00:02 <nirik> I'll keep poking at it. :)
17:00:04 <mboddu> Seems like it is oz
17:01:09 <mboddu> Okay, its time
17:01:14 <mboddu> Thanks everyone for joining
17:01:17 <jednorozec> I would try to run the image build by hand
17:01:32 <mboddu> If anything else, please take it to the #fedora-releng channel
17:01:43 <mboddu> #endmeeting