15:00:16 <jednorozec> #startmeeting RELENG (2021-05-04)
15:00:16 <zodbot> Meeting started Tue May  4 15:00:16 2021 UTC.
15:00:16 <zodbot> This meeting is logged and archived in a public location.
15:00:16 <zodbot> The chair is jednorozec. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:16 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:16 <zodbot> The meeting name has been set to 'releng_(2021-05-04)'
15:00:16 <jednorozec> #meetingname releng
15:00:16 <jednorozec> #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec
15:00:16 <jednorozec> #topic init process
15:00:16 <zodbot> The meeting name has been set to 'releng'
15:00:16 <zodbot> Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz
15:01:11 <nirik> morning
15:01:37 <jednorozec> good morning sir
15:01:45 <pingou> ó/
15:04:14 <mboddu> I am sorta here
15:04:17 <mboddu> Also in another meeting
15:07:03 <jednorozec> looking  at the issues there are no now issues tagged with meeting from the last week.
15:07:05 <jednorozec> https://pagure.io/releng/issues?status=Open&tags=meeting&close_status=
15:07:49 <nirik> we can possibly close 9451?
15:11:15 <jednorozec> heh funny thing is that list-channels shows some channels with over 100% load
15:11:15 <nirik> mboddu: is there a SOP for 10099 ?
15:12:01 <jednorozec> I have not seen one in the releng sop
15:12:02 <nirik> jednorozec: yeah, I think that is because many builders are in a number of channels at the same time.
15:12:48 <nirik> .releng 7067
15:12:49 <zodbot> nirik: Issue #7067: Fix package ownership in koji - releng - Pagure.io - https://pagure.io/releng/issue/7067
15:12:51 <nirik> might be fixed soon
15:13:58 <jednorozec> nice
15:14:26 <nirik> .releng 8468
15:14:26 <zodbot> nirik: Issue #8468: fedora-scm-requests new repository procedure repeatedly keeps the repo owned by the releng person - releng - Pagure.io - https://pagure.io/releng/issue/8468
15:14:31 <nirik> can be closed ^
15:15:10 <jednorozec> closed
15:15:24 <jednorozec> that one is fixed for some time in fedscm_Admin
15:19:05 <nirik> there could be more to close, some of the old tickets have a lot of discussion...
15:20:23 <jednorozec> yeah I am not sure what to with discussions that are older than my participation in releng... Some of them may be obsolete but some may not.
15:21:11 <nirik> .releng 9773
15:21:12 <zodbot> nirik: Issue #9773: Add RAWHIDEFTBFS and RAWHIDEFTI aliases to trackers - releng - Pagure.io - https://pagure.io/releng/issue/9773
15:21:19 <nirik> this might be an easy one to knock out
15:22:02 <nirik> needs added to the branching sop.
15:22:10 <jednorozec> yup doc is needed
15:22:13 <jednorozec> I can fix that
15:22:50 <nirik> this next cycle, perhaps we should try and make a document with all the things we do listed clearly and then in f36 look at automating things with ansible or the like
15:23:25 <jednorozec> my idea is to have docs.fp.org page ready after f35 branching
15:23:41 <jednorozec> I have updated some steps in the current SOP last time
15:23:57 <jednorozec> but we should make it part fod doc.fp
15:25:05 <jednorozec> and automate as much as possible with ansible
15:25:09 <nirik> yeah. mobrien has been pushing for that too.
15:26:13 <nirik> not sure how releng and infra will be best organised there, but we can find some good way I am sure.
15:26:13 <jednorozec> I think it make sense to have one place for docs
15:26:46 <mobrien> jednorozec: I am writing a mail to the infra list on the subject at this very moment
15:27:37 <mobrien> Where do releng folks mostly hang out devel list?
15:28:42 <nirik> mobrien: there's a releng list, but it doesn't get much human traffic
15:28:53 <nirik> I think everyone in releng is on the infrastructure list
15:29:17 <mobrien> ok, I'll just send it to infra list so, thanks nirik
15:32:28 <jednorozec> what about this
15:32:33 <jednorozec> .releng 9240
15:32:33 <zodbot> jednorozec: Issue #9240: Update published Fedora 31 cloud images - releng - Pagure.io - https://pagure.io/releng/issue/9240
15:33:58 * nirik clicks
15:34:38 <nirik> so I guess we don't have a process to advertise a newer cloud image...
15:35:11 <jednorozec> nope
15:35:31 <nirik> we do build them nightly and upload them to amazon.
15:36:10 <jednorozec> so there are latest cloud images in the clouds?
15:36:15 <nirik> probibly wouldn't be too hard to also copy them to mirrors
15:36:25 <nirik> well, aws anyhow.
15:37:14 <nirik> fedimg.image.publish -- Fedora-Cloud-Base-33-20210504.0.aarch64 published in region, eu-west-2 (ami-0a0182d97e022d0ca, hvm, standard)
15:38:06 <nirik> looks like the 34 ones are failing or not working right
15:39:40 <nirik> perhaps we should try and go to the cloud sig meeting next time and make sure what we need to / want to do here
15:41:44 <jednorozec> #actionitem jednorozec will attend next clud-sig meeting to discuss 9240
15:41:55 <jednorozec> #action jednorozec will attend next clud-sig meeting to discuss 9240
15:41:58 <jednorozec> hm
15:42:33 <nirik> yeah, that doesn't output anything, but it worked. ;)
15:43:21 <jednorozec> Aha :) and which one did it?
15:45:05 <nirik> second one
15:45:10 <jednorozec> next cloud sig meeting is next Tuesday
15:45:29 <jednorozec> After this meeting
15:46:05 <nirik> cool
15:48:08 <jednorozec> .releng 10101
15:48:11 <zodbot> jednorozec: Issue #10101: Orphan sparkleshare because it is more than 8 weeks in NEW state and blocks F34FTBFS - releng - Pagure.io - https://pagure.io/releng/issue/10101
15:48:24 <jednorozec> some automation probably failed
15:48:50 <jednorozec> I will fix this after the meeting
15:48:58 <nirik> cool. Yeah, hadn't seen that one yet.
15:50:27 <mboddu> nirik: 10099, there is a doc, but its not in releng, it was in github somewhere under coreos org. I will add it to releng repo
15:50:54 <jednorozec> thanks mboddu
15:51:28 <nirik> cool.
15:52:37 <jednorozec> #topic Alternative Architectures updates
15:52:43 <jednorozec> sharkcz, any updates?
15:53:19 <sharkcz> jednorozec: perhaps the s390x compose builders lost /mnt/koji
15:53:44 <sharkcz> I haven't checked the details yet, but all images were dropped
15:54:39 <nirik> they did last night, I already fixed em. ;)
15:54:45 <nirik> should be ok for next run
15:54:53 <sharkcz> then everything is OK :-)
15:54:54 <sharkcz> thanks
15:55:12 <nirik> the s390x builders have continued to be a bottleneck. ;(
15:55:56 <sharkcz> you mean in general?
15:56:02 <nirik> mostly because people do say a PR on llvm... zuul submits a build, they do it on each branch, then they do real builds on each branch and suddenly 8 builders are building llvm
15:56:22 <sharkcz> I think we should open a ticket to get more capacity
15:56:25 <nirik> yesterday we had a time with 11 llvm's and 8 java's... that was basically everything
15:56:32 <sharkcz> because of these additional workloads
15:56:55 <nirik> yeah, I actually sent an email not long ago asking about plans to move us to the new mainframe...
15:57:00 <nirik> but I never got a reply. ;(
15:57:33 <nirik> we could carve up our kvm lpar differently if we wanted...
15:57:46 <nirik> do 2x builders but with 1/2 resources.
15:57:53 <nirik> I guess that won't work due to disk.
15:58:08 <sharkcz> ack, we need more capacity
15:58:29 <jednorozec> Whan can we do about it?
15:58:41 <sharkcz> I can check what Jira will be the right place this time ...
15:59:13 <nirik> sharkcz: ok. if you tell me where I can file. Actually... do we still have a issue ?
15:59:15 * nirik looks
16:01:07 <nirik> no, the last one was to move us to z13... and thats resolved.
16:01:31 <sharkcz> nirik: will let you know once I find it :-)
16:01:46 <nirik> sharkcz: ok, great.
16:02:03 <jednorozec> and we are over time.
16:02:05 <sharkcz> there was even an email thread internally about the lack ofresources/long queue
16:02:10 <nirik> I hope they can move us to new machine and give us more resources, but we will see
16:02:16 <sharkcz> yup
16:02:23 <nirik> sharkcz: yeah, I replied to that, but got no answer. ;(
16:02:49 <jednorozec> we need to make more noise
16:02:58 <sharkcz> seems so :-)
16:03:17 <nirik> there was also a memo list post internally. ;)
16:03:21 <jednorozec> Eventually people will just give us the resources so we stay quiet
16:04:18 <jednorozec> do we you have anything else to discuss?
16:04:35 <nirik> oh, one quick note:
16:04:48 <nirik> I updated koji on the hubs to 1.24.1... will do the builders soon.
16:04:58 <nirik> and we need to move all of them to f34 at some point.
16:06:08 <jednorozec> nirik, how much work is it?
16:06:48 <nirik> moving builders? well, not too bad, just time consuming. However, we are still blocked on armv7... newer than f32 kills kojid. ;(
16:06:54 <nirik> so we need to solve that
16:07:59 <jednorozec> I can look into it
16:08:14 <jednorozec> is there any ticket with info ?
16:08:39 <nirik> oh yes...
16:08:42 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=1920183
16:09:12 <nirik> buildvm-a32-01.stg is f34 and shows the problem if you do a stg scratch build of python3.8
16:11:01 <jednorozec> I will play with it
16:11:06 <jednorozec> thanks
16:11:22 <jednorozec> see you next week.
16:11:31 <jednorozec> #endmeeting