15:01:13 <mboddu> #startmeeting RELENG (2020-03-31)
15:01:13 <zodbot> Meeting started Tue Mar 31 15:01:13 2020 UTC.
15:01:13 <zodbot> This meeting is logged and archived in a public location.
15:01:13 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:13 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:13 <zodbot> The meeting name has been set to 'releng_(2020-03-31)'
15:01:13 <mboddu> #meetingname releng
15:01:13 <zodbot> The meeting name has been set to 'releng'
15:01:13 <mboddu> #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec
15:01:13 <mboddu> #topic init process
15:01:13 <zodbot> Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz
15:01:41 <nirik> morning
15:01:52 <sharkcz> good afternoon :-)
15:02:37 <mboddu> Okay, lets get started
15:04:05 <mboddu> #topic #8957 Nonpackagers can review packages
15:04:11 <mboddu> #link https://pagure.io/releng/issue/8957
15:04:39 <mboddu> I was going through some old tickets and I found this
15:04:40 <nirik> so I guess we should open a fedscmadm bug and close this one in favor of that?
15:05:10 <mboddu> nirik: But is it really fedscm-admin bug?
15:05:11 <nirik> I actually thought for a while that we had being able to set fedora-review flag tied to bugzilla perms, but I guess not
15:05:46 <nirik> well, that seems the easiest place to solve it?
15:06:07 <sharkcz> this plan looks good
15:06:18 <mboddu> nirik: Okay, I thought it was bz perms bug
15:06:46 <nirik> well, we could try that angle, but I suspect fedscm-admin would be easier/quicker to fix.
15:06:56 <mboddu> Well, if everyone think its easier to fix in fedscm-admin, I will move it to fedscm-admin
15:07:17 <mboddu> Whats the fas group to check for?
15:07:41 <nirik> packager
15:08:17 <mboddu> nirik: Nope, not all packagers are reviewers
15:08:24 * mboddu is an example :D
15:08:42 <nirik> but you _could_ be... thats what grants you the power to approve reviews.
15:08:57 <sharkcz> all sponsored packagers are potential reviewers
15:10:05 <mboddu> nirik: Right, but in this case we need to check only for reviewer group, not the packager group
15:10:19 <mboddu> sharkcz: Yes, I thought there is a separate fas group for reviewers
15:10:22 * mboddu digs
15:10:47 <nirik> there is not any such group
15:10:52 <nirik> it's all packagers. ;)
15:11:10 <nirik> as soon as you are sponsored into packager you can review and approve reviews.
15:11:23 <sharkcz> right
15:11:28 <mboddu> .fasinfo eclipseo
15:11:29 <zodbot> mboddu: User: eclipseo, Name: Robert-André Mauchin, email: zebob.m@gmail.com, Creation: 2016-10-10, IRC Nick: eclipseo, Timezone: Europe/Paris, Locale: fr, GPG key ID: None, Status: active
15:11:33 <zodbot> mboddu: Approved Groups: +rust-sig provenpackager +packager @go-sig fedorabugs cla_done cla_fpca
15:12:17 <mboddu> Then why couldn't I able to review a package, it was like last year
15:12:25 <mboddu> Oh well, packager group it is then
15:12:40 <nirik> you should be able to... what prevented it? you couldn't set the flag?
15:14:15 <mboddu> nirik: I think so
15:14:31 <mboddu> Anyway, I will give another shot at it later
15:14:40 <mboddu> Not worth wasting time on it
15:15:16 <nirik> we still need to check for packager anyhow, because qa folks also get bugzilla privs...
15:16:34 <mboddu> Okay
15:16:41 <mboddu> While sharkcz is here
15:16:48 <mboddu> #topic Alternate Arch Updates
15:17:09 <mboddu> I know we lost ppc64le builders and couple of s390x during the last week
15:17:16 <mboddu> Anything else going on there?
15:17:36 <mboddu> And thanks nirik for going through the ppc64le builders fix
15:18:07 <sharkcz> not much, overall things are working, I just need to check the state of various images on ppc and s390x, sometime they fails
15:18:40 <sharkcz> like for missing /mnt/koji or some unknown reason on ppc, want to be sure we have all of them for GA
15:19:11 <mboddu> sharkcz: Right
15:19:37 <mboddu> On that note, we should also look at armhfp container failures
15:20:27 <sharkcz> yeah, I can talk to pwhalen
15:20:41 <nirik> There's still also sometimes src.rpm unpacking errors on s390x...
15:20:52 <nirik> which still we really haven't ever figured out.
15:20:55 <sharkcz> right
15:22:10 <sharkcz> I'll reply in the ticket, some debug output would be useful in the download path
15:23:08 <sharkcz> looks like only single large srpms have the problem
15:23:16 <nirik> it's not clear to me which download path it's taking.
15:24:02 <sharkcz> I think it's in the "requests.get()"
15:24:21 <sharkcz> if you already saw tkopecek's last reply
15:24:26 <nirik> I don't know if it's just a regular download or via xmlrpc or what...
15:24:30 <nirik> I did not
15:24:43 <nirik> I just woke up before this meeting. I have not read email yet. ;)
15:24:57 <sharkcz> np :-) I did read it :_)
15:26:57 <nirik> ok, I just did too. interesting...
15:27:11 * nirik still has no idea why it's failing.
15:27:47 <sharkcz> IMO the get() method downloads the file incomplete ...
15:28:03 <sharkcz> it could hit some timeouts
15:28:40 <nirik> yeah, but you would expect it to fail the build there then...
15:28:48 <nirik> or it's somehow thinking it's complete...
15:28:58 <sharkcz> ideally we should see expected and actual file size
15:29:11 <sharkcz> maybe there is a bug, so it thinks it's complete
15:29:47 <sharkcz> if expected and actual are the same, and still too short, then it would point to the cache
15:30:04 <nirik> yeah.
15:30:25 <sharkcz> but if they differ, it's "requests" problem
15:31:33 <nirik> the odd thing is that this only seems to hit src.rpms... I haven't seen any normal buildroot downloads hit it.
15:32:02 <sharkcz> because only the srpms are so large
15:32:56 <nirik> perhaps
15:32:57 <sharkcz> and the buildroot rpms might use a different method, mock->dnf->??
15:33:33 <sharkcz> I'll ask Tomas for adding some diagnostics and hope it will help to understand the issue
15:34:52 <nirik> ok. yeah, we can definitely do that
15:36:22 <mboddu> Thanks nirik and sharkcz
15:36:27 <mboddu> Moving on...
15:36:36 <sharkcz> ack
15:37:04 <mboddu> This is a not releng ticket but it needs some discussion/estimation
15:37:15 <mboddu> #topic #8690 Can't build module with dependency on module in RHEL - From fedora-infrastructure
15:37:21 <mboddu> #link https://pagure.io/fedora-infrastructure/issue/8690
15:37:47 <mboddu> So, pulling non-default rhel module content to generate epel8 buildroot
15:37:57 <nirik> I'm really not sure what the plan is
15:38:04 <mboddu> I have no idea how we can acheive this
15:38:10 <nirik> but it's not the epel8 buildroot.
15:38:10 <mboddu> achieve*
15:38:28 <nirik> it's the mbs buildroot or whatever it creates
15:38:52 <mboddu> Yeah, epel8 modular buildroot?
15:38:53 <nirik> but we don't have the rhel modules, they are only in an external flattened repo.
15:38:57 <mboddu> But, yes mbs stuff
15:39:08 <nirik> fedora mbs has no idea about rhel modules (I don't think?)
15:39:17 <mboddu> Nope, it doesn't
15:39:39 <nirik> so, yeah, no idea. We can ask tdawson who was going to work on this?
15:39:45 <mboddu> Both epel8 and epel8 modular uses the flattened rhel repo
15:40:38 <mboddu> tdawson is requesting for our help
15:40:40 <nirik> even if we enabled the modular repodata, it wouldn't help... mbs needs more than that I am pretty sure. it needs the modules in it's local db
15:41:08 <nirik> well, I think he had someone in mind, but I don't know who.
15:41:42 <mboddu> I think it was me (or smooge)
15:41:44 <tdawson> I'm requesting the help, and don't totally know the right way.   I'm wondering if merlinm might know better, since he helped to get the EPEL8 module infrastructure setup.
15:42:19 <nirik> I wonder if there's some place we could build epel modules in centos land...
15:42:51 <mboddu> repospanner :D
15:42:53 * mboddu hides
15:43:08 <nirik> another thing we could do is talk to koji developers... since they are taking over mbs... explain the use case and see if they can think of any way it might work?
15:43:27 <mboddu> nirik: I can schedule a meeting with them
15:44:26 <mboddu> I dont think it requires some discussion
15:44:26 <nirik> sure, or we could catch them on irc sometime... but yeah, perhaps a short meeting might be good.
15:44:38 <mboddu> I think*
15:45:22 <mboddu> Okay, I will invite nirik, tdawson, merlinm, mikem, anyone else?
15:46:01 <nirik> sounds good
15:46:36 <mboddu> Okay
15:46:53 <mboddu> #topic Open Floor
15:46:58 <mboddu> #undo
15:46:58 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x7fe0007e8ed0>
15:47:24 <mboddu> #info mboddu will schedule a meeting with koji folks to discuss how we can achieve this
15:47:28 <mboddu> #topic Open Floor
15:47:42 <nirik> lets see...
15:47:53 <mboddu> #info Next week Tuesday Fedora F32 Final Freeze starts
15:47:54 <nirik> One quick thing...
15:48:12 <nirik> pub/fedora-secondary/development/rawhide/Modular/ppc64 needs deleted. ;)
15:48:21 <nirik> oh, another quick thing...
15:48:26 <sharkcz> ack
15:48:38 <nirik> rawhide and f32 seems stuck... we should investigate.
15:49:17 <mboddu> nirik: I just did, I free'd the tasks as nothing was happening on the buildhw-10
15:49:28 <mboddu> Both of the f32 and f33 tasks are stuck in it
15:49:34 <nirik> great. :)
15:50:02 <mboddu> I looked at the builder and strace is not returning anything, seemed like stuck for some unknown reason, nothing in dmesg
15:50:15 <mboddu> I free'd the tasks and rebooted the box
15:50:28 <nirik> ok. cool.
15:50:31 <nirik> I nuked that ppc64 dir
15:50:38 <mboddu> nirik++
15:50:43 <mboddu> And the tasks are done now
15:52:01 <mboddu> Both are in checksum phase now
15:52:08 <mboddu> Anything else?
15:54:25 <mboddu> Okay, thanks for joining guys
15:54:31 <mboddu> I will give back 5 min
15:54:35 <mboddu> #endmeeting