16:00:07 <jednorozec> #startmeeting RELENG (2023-01-10)
16:00:07 <zodbot> Meeting started Tue Jan 10 16:00:07 2023 UTC.
16:00:07 <zodbot> This meeting is logged and archived in a public location.
16:00:08 <zodbot> The chair is jednorozec. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:00:08 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:08 <zodbot> The meeting name has been set to 'releng_(2023-01-10)'
16:00:08 <jednorozec> #meetingname releng
16:00:08 <zodbot> The meeting name has been set to 'releng'
16:00:08 <jednorozec> #chair nirik sharkcz pbrobinson phsmoura dustymabe jednorozec
16:00:08 <jednorozec> #topic init process
16:00:08 <zodbot> Current chairs: dustymabe jednorozec nirik pbrobinson phsmoura sharkcz
16:00:20 <nirik> morning
16:00:38 <phsmoura> morning
16:00:41 <nirik> I have a few things... want to talk about mass rebuild improvements, and a few tickets.
16:01:03 <jednorozec> same here I have some mass rebuild notes and few tickets
16:01:13 <jednorozec> #topic Mass rebuild
16:01:31 <jednorozec> so we dont send out reminders about mass rebuilds
16:02:21 <jednorozec> until now we contacted people on change review requests tickets, but from the past experience I know that notifications from pagure tend to slip the radar
16:02:24 <nirik> it could be a rough one, as fesco landed a compiler flag change last minute... and it's not even all sorted out yet
16:02:42 <jednorozec> yeah I have noticed that one
16:02:54 <nirik> yeah. Ideally we get a ack from all changes that need it that they are ready
16:03:17 <nirik> would be good to update all those with a reminder this week
16:03:23 <jednorozec> I am thinking about adding a template for mass rebuild reminder email
16:03:42 <jednorozec> and to add to cc people that have opened ticket in releng tracker with massrebuild tag
16:04:05 <nirik> sounds good.
16:05:35 <jednorozec> ok, I will open PR and ping you for grammar
16:05:43 <nirik> +1
16:05:55 <nirik> I'd like to propose 2 changes to the scripting for mass rebuild...
16:06:02 <jednorozec> shoot
16:06:43 <nirik> 1. Add --depth 1 to the fedpkg clone... this will only checkout the latest thing (what we need) and not all the git history.. should make a lot of larger history packages much faster.
16:08:45 <nirik> 2. right now we use ssh and a weird bit of stuff to get that working. How about switching to https / token? so, change the fedpkg clone to use -a and setup a releng token for pushing.
16:09:12 <nirik> I am not 100% sure where the token goes, but we can figure that out.
16:09:25 <nirik> I think ~/.config/rpkg/fedpkg.conf or something
16:09:49 <nirik> that should simplify things a good deal
16:10:21 <nirik> I can try and make a PR (or review one if someone else wants to do it)
16:11:09 <jednorozec> so the 1, is nobrainer and should have been there from the begining
16:11:57 <jednorozec> I can look into 2.
16:12:08 <jednorozec> but I am not sure how it will make thing less complex
16:12:18 <jednorozec> it is basicaly one binary that we use for push
16:12:38 <jednorozec> but the truth is that we dont actually need that
16:12:55 <jednorozec> ah ok
16:13:09 <jednorozec> looking at the code there is like 20-50 lines of support code for the hack
16:14:10 <jednorozec> hm
16:14:45 <nirik> yeah, with the token there's just fedpkg push
16:15:14 <jednorozec> what is strange is that we use things like --allow-empty for the git executable
16:15:31 <jednorozec> I am not sure we will be able to do that with fedpkg, but also I am not sure if we need it
16:15:51 <jednorozec> or its just there as a part of legacy
16:15:52 <nirik> do we need to do anything for rpmautospec? I guess not, since we have had it for the last few...
16:16:45 <jednorozec> but yeah, I can look into this
16:17:05 <jednorozec> reducing complexity is my favorite type of work
16:18:50 <nirik> cool. ah, so the --allow-empty is there for rpmautospec I think
16:19:30 <nirik> because rpmdev-bumpspec doesn't do anything on those
16:19:44 <nirik> so there's no change to commit, but we need the commit/log message
16:20:09 <jednorozec> heh
16:20:13 <jednorozec> it was me who added it
16:20:26 <jednorozec> ah no
16:20:43 <jednorozec> I updated the cmd last but not with that switch https://pagure.io/releng/c/950ff66d09277039423964654db20d6a43415c6e
16:21:11 <jednorozec> anyway
16:21:16 <nirik> anyhow with --depth1 and https I think it will be faster
16:21:30 <jednorozec> yeah
16:21:31 <jednorozec> it should
16:21:55 <jednorozec> I will try to implement it tomorrow and we will see
16:22:17 <nirik> sounds good.
16:23:00 <jednorozec> lets move on
16:23:05 <jednorozec> #topic tickets
16:23:26 <nirik> I was going to mention https://pagure.io/releng/issue/11200 and see if you could do it? I don't know if we have a sop on it.
16:23:53 <jednorozec> sure its done already
16:23:59 <nirik> There's also just a ton of docs tickets now... would really be good to somehow move those forward
16:24:04 <nirik> ha. ok
16:24:08 <jednorozec> yeah
16:24:15 <jednorozec> I started to tag them with docs
16:24:34 <jednorozec> to have a track what I need to add to the new docs
16:24:36 <nirik> so, we can close 11200?
16:25:13 <jednorozec> I think so
16:25:17 <jednorozec> let me do that
16:25:41 <jednorozec> btw there is no SOP for it, since they provide script. You just need access to the keys
16:26:20 <nirik> ah, ok
16:27:05 <jednorozec> but when we are here
16:27:23 <jednorozec> we have a new addition to the team jnsamyak
16:27:33 <jednorozec> I am not sure if he is online since he is in India
16:27:59 <jnsamyak> Hello folks! 0/
16:28:07 <jednorozec> here you are :)
16:28:10 <nirik> ah cool. Welcome jnsamyak!
16:28:23 <jnsamyak> Just came online 10 mins back lost the track of time \0/
16:28:31 <jnsamyak> Helllo nirik :D
16:29:21 <jednorozec> nirik, I am thinking about guiding him through the release process with 38
16:29:53 <jednorozec> and using his youth and other superpowers to get some of the releng processes in shape
16:29:59 <nirik> sounds good. Always an adventure.
16:30:23 <jnsamyak> Seems pretty adventurous:P will love to start somewhere
16:30:33 <smooge> i had a general question if there is an open floor later on
16:30:43 <nirik> it's definitely a firehose... :)
16:31:01 <jednorozec> one of those might be automating work in ^^ that ticket
16:32:15 <jednorozec> I have this ticket
16:32:18 <jednorozec> .releng 11189
16:32:19 <zodbot> jednorozec: Issue #11189: add fedora-toolbox as an official Fedora release artifact - releng - Pagure.io - https://pagure.io/releng/issue/11189
16:32:43 <jednorozec> is this something we can do based on just this request?
16:35:57 <nirik> I'm not sure entirely what it would mean...
16:36:05 <nirik> I guess making it a blocking deliverable?
16:36:22 <jednorozec> that was my guess
16:37:11 <nirik> so, I think this needs fesco... https://docs.fedoraproject.org/en-US/releases/f38/blocking/
16:37:32 <nirik> and they need to make a change for that
16:37:47 <jednorozec> I see it on that page
16:37:55 <nirik> so, I think it's doable from our side, just needs approval.
16:38:05 <jednorozec> let me point there and leave a comment on the ticket
16:38:12 <nirik> and it might result in release delays when/if our container pipeline is messed up.
16:38:17 <nirik> sounds good
16:38:45 <jednorozec> that is all from me
16:39:17 <jednorozec> do you have any other tickets to discuss nirik ?
16:39:26 <jednorozec> or we can move on to open floor
16:39:57 <nirik> nope, oh, one more mass rebuild thing... I want to see about updating and rebooting all the builders before mass rebuild next week... I can do that sometime later this week when they aren't very busy.
16:40:46 <jednorozec> ack
16:40:53 <jednorozec> #topic OPen Floor
16:41:00 <jednorozec> smooge, the stage is yours
16:41:55 <smooge> thanks. so I noticed in the current 'FORTIFY' drama list that there is a good chance that someone felt that compiles for other arches may break a LOT on the rebuild
16:42:09 <smooge> sorry that doesn't make a lot of sense..
16:42:55 <smooge> basically some of the options may cause compilation failures for packages and I wasn't sure if anyone here had been made aware of it
16:42:56 <nirik> yeah, they are still discussing what to do on aarch64/s390x
16:43:03 <nirik> and thats poor...
16:43:19 <jednorozec> yeah
16:43:29 <jednorozec> I was just reading the list before this meeting
16:43:58 <smooge> it has been a very long time since I rememebred a change which could break everything happening in a mass rebuild. I think Dennis and Kevin had to refire everything from scratch
16:44:46 <smooge> so anyway I should have realized nirik would have read everything already and would have caught that =).
16:44:50 <jednorozec> I have lost all of my hair already
16:45:03 <jednorozec> what worse can happen?
16:45:27 <nirik> I'm really not happy with the statement someone had... "if lots of things start failing we can change it"
16:45:38 <smooge> I also wanted to make sure I didn;'t come across as 'This Cant Happen!!!' as some other people have been
16:46:20 <jednorozec> its always easy when someone else is "changing it"
16:46:22 <smooge> some people seem to have an idea that there is a goblin feeding in packages one at a time into the build system and another one who will tell the first to stop and fix things
16:47:02 <nirik> I'll raise these concerns at the fesco meeting...
16:47:06 <smooge> anyway, that was all. {EOF}
16:47:07 * mboddu_ waves
16:47:15 <nirik> not that likely that will change anything, but might make me feel better. ;)
16:48:03 <jednorozec> thanks guys
16:48:20 <mboddu_> I am not sure if we ever tried this, but can we test it out with a mass scratch build?
16:49:00 <nirik> perhaps. there's also a mass rebuild in copr script...
16:49:52 <mboddu> But copr might not have all of Fedora pkgs
16:50:02 <sharkcz> the rh tools team came up with a tool for doing and analyzing mass rebuilds and their results (mbp), i believe it was mentioned on fedora-devel list
16:50:19 <sharkcz> outside koji
16:50:20 <nirik> mboddu: it pulls from src.fp.o...
16:50:29 <nirik> sharkcz: yea, thats the one I am talking about I think
16:50:46 <sharkcz> yup, it likely is
16:51:07 <mboddu> Oh, that is interesting, I need to read about mbp
16:51:12 <nirik> https://gitlab.com/fedora/packager-tools/mass-prebuild
16:51:29 <nirik> but IMHO this is not for releng to do.
16:51:36 <nirik> it's for change owners to have done.
16:51:40 <sharkcz> right
16:51:49 <sharkcz> or for anyone else
16:51:57 <jednorozec> ideally not week before mass rebuild
16:51:59 * nirik has to go feed cats...
16:52:07 * mboddu agrees
16:53:03 * smooge goes to feed crows
16:53:34 * jednorozec goes to close the ducks and release the poodle
16:54:26 <jednorozec> thanks for your attention.
16:54:29 <jednorozec> #endmeeting