16:00:09 #startmeeting RELENG (2020-02-05) 16:00:09 Meeting started Wed Feb 5 16:00:09 2020 UTC. 16:00:09 This meeting is logged and archived in a public location. 16:00:09 The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:09 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:09 The meeting name has been set to 'releng_(2020-02-05)' 16:00:10 #meetingname releng 16:00:10 #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec 16:00:10 #topic init process 16:00:10 The meeting name has been set to 'releng' 16:00:10 Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz 16:00:15 morning 16:01:07 morning 16:01:24 nirik, smooge : Morning, how goes? 16:01:32 afternoon 16:01:39 morning all 16:01:43 or afternoon/evening 16:02:12 mboddu, it goes 16:03:10 Morning everyone 16:04:06 Lets get started 16:05:59 * cverna is kind of around 16:09:37 jrichardson: Do you wanna introduce yourself to the folks here? 16:09:52 sure 16:09:53 I guess only Dusty might not know about you 16:09:59 But go ahead 16:10:56 Hi all, my name is James, I'm a mature student at Waterford Institute of Tech and interning with Leigh Griffin on the AAA/Sustaining team here in Waterford 16:11:37 hoping to be useful soon :) EOM 16:12:12 Nice to see you here jrichardson 16:12:43 cverna, thanks, glad to be 16:13:13 Thanks jrichardson 16:13:21 Very happy that you are here to help :) 16:13:27 So, lets move to the tickets 16:13:45 sigh... laptop decided crashing was a good idea. 16:13:50 #topic #9057 Ability and permission to sign applications for download using Fedora Releng keys 16:13:59 #link https://pagure.io/releng/issue/9057 16:14:16 nirik: No problem, we just went through the intro of jrichardson 16:14:19 dustymabe: You are up 16:14:24 mboddu: I think this is mostly not an issue any longer 16:14:31 hi jrichardson! 16:14:34 we've worked with you guys and have a working relationship on this topic 16:14:44 but there is still work outstanding to automate more of the process I think 16:14:50 mboddu: would you agree? 16:15:29 dustymabe: Yes 16:15:30 nirik, hello! 16:16:55 mboddu: so we can probably stop discussing this in meetings for now 16:17:20 dustymabe: Okay, can we close the ticket? 16:17:22 Who is going to work on the automation ? 16:17:45 What needs to be done manually ? 16:18:17 cverna: Its scripted out, but someone with access to the keys has to run it. 16:18:30 cverna: we are tracking the automation in https://github.com/coreos/fedora-coreos-tracker/issues/335 16:18:45 Ok :) 16:18:46 right now we manually open a ticket and ask releng to run it like: https://pagure.io/releng/issue/9126 16:18:57 mboddu: we can probably close the ticket 16:18:58 +1 16:19:11 dustymabe: Do you have everything you need for the automation? 16:19:51 mboddu: I'm not 100% sure on that 16:20:01 but will definitely let you know if there are gaps/questions that need to be answered 16:20:33 dustymabe: Okay, please let us know and we would be happy to help 16:21:42 #info Current status is that we have a script to run but someone has to manually run it, dustymabe and team are working on automating it. For now we will close the ticket. 16:22:42 #topic #9153 Use an automated script to control checksums of compose images 16:22:49 #link https://pagure.io/releng/issue/9153 16:23:32 So, generally I have a script that I run before go/no-go for rc composes 16:23:54 But it would be nice to have it for nightly composes and that too automated 16:25:38 nirik: Do we have an bot user that can update wiki? 16:25:58 we do not currently no, but one could be made I suppose 16:26:47 nirik: So, I was thinking of updating the nightly.sh script to run the tests and update the wiki, wdyt? 16:28:01 which tests? 16:28:44 nirik: checksum tests 16:28:55 oh, sorry, missed the topic... 16:29:10 can we just add it to the report we email the list? 16:29:58 The whole point is updating the wiki, since they have to manually update the wiki 16:30:24 well, I think the question is still open. I like kparal's idea... 16:30:36 ie, once we test it remove it from qa matrix 16:31:03 so, I'd say lets discuss more in ticket? 16:31:12 * kparal is here 16:31:59 mboddu: I'd prefer to just remove it from the wiki and rely on your scripts to do the job 16:32:20 hey kparal 16:32:20 Also, now I think about it, when compose is just done, there wont be a wiki pointing to that compose, so we cant just update wiki 16:32:28 * mboddu waves at kparal 16:32:57 mboddu: our wiki is just a poor-man's tcms, really :) 16:33:32 haha :) 16:34:19 May be we should just do what nirik said and report it in the email 16:34:33 if your scripts blow up when the checksums don't match, we don't need to track the verification result. we'll know that it will be obvious when something goes wrong 16:35:01 ack 16:35:08 i.e. the compose will not be where it is supposed to be, or perhaps doomed or something, and everyone will know 16:35:49 it's not like it's going to happen frequently 16:36:04 kparal: I remember for one release, the 32 bit images failed that test, but we went ahead and released anyway, so, I am guessing we only test it for release blocking artifacts? 16:36:57 we also need to confirm that the images are all in the checksum files. 16:37:05 well, I don't remember that one, but we block the release just on release blocking artifacts. So I guess we didn't care about i386 checksums that much 16:38:34 I hope that this operation is so simple that if it breaks, it means a releng script broke somewhere (e.g. we added some new composes somewhere but forgot to add them in all places) or there's a serious problem in sha256sum or glibc or similar 16:38:39 or a hardware problem 16:39:05 * nirik nods 16:39:17 it's like signing packages, if it's broken, we definitely want to stop and look at it 16:39:58 nirik: yes we need to check that all images are present in the checksum file and our own testcase doesn't reflect that :) 16:40:20 kparal, nirik : That sounds good to me 16:43:05 so, leave ticket/discuss, move on? 16:43:19 #info RelEng will update the nightly script to run "sha256sum -c", "checkisomd5" tests and verify that the images are all in the checksum file and post the results to the compose email and if anyone of them fails, we should not sync the compose to the mirrors 16:44:45 If we want to discuss more, I will post this in the comments and ask for feedback 16:46:30 sure. 16:47:13 #undo 16:47:13 Removing item from minutes: INFO by mboddu at 16:43:19 : RelEng will update the nightly script to run "sha256sum -c", "checkisomd5" tests and verify that the images are all in the checksum file and post the results to the compose email and if anyone of them fails, we should not sync the compose to the mirrors 16:47:38 #info RelEng thinks we should update the nightly script to run "sha256sum -c", "checkisomd5" tests and verify that the images are all in the checksum file and post the results to the compose email and if anyone of them fails, we should not sync the compose to the mirrors. We will ask for comments in the ticket. 16:49:55 #topic #9154 F32 Change: new buildroot for CPU baseline update 16:50:02 #link https://pagure.io/releng/issue/9154 16:50:10 Thanks nirik for asking those questions 16:50:19 We haven't got any response from them 16:50:26 I will ping them in the ticket again 16:50:50 well, I just added the questions yesterday. ;) 16:50:57 I'm sure they will get to it. 16:51:03 hopefully my questions made sense 16:51:21 Those are the questions that I had and wanted to discuss 16:51:42 Well, you got more, but I had questions about the buildroot and the signing :) 16:52:29 So, last few min 16:52:33 #topic Open Floor 16:52:46 so, I had a few things... 16:52:52 Go ahead 16:53:21 Also, can you let us know whats the issue with the f31 updates composes as well? 16:53:38 1) we have been talking about doing some standups to go over/process tickets and backlog. I think this will be great... will try and keep the list in the loop as to when in case anyone wants to attend. 16:54:33 2) branching is next week. We should try and make sure we are all prepped for it. I'd like to look at signing everything for f33 this weekend so we are ready. Also, note that this time we are doing the 'no new updates in until we have a compose' thing... 16:55:04 I think the way we do that is to let packages land in some tag and disable autosign or bodhi processing them until we have a compose... 16:55:21 but we should figure the exact change. ;) 16:55:59 3) hopefully rawhide will compose later today with the mass rebuild in it. I guess mboddu you are gonna file the FTBFSes? 16:56:41 On 2. I am guessing we should not run updates composes until we have a branched compose 16:56:46 On 3. yes, I am on it 16:57:09 nope... we can't do that because bodhi isn't enabled for another few weeks. 16:57:46 also last time we had it like pre-gating rawhide... we should make it like gating rawhide this time. 16:58:20 Okay, I will look into it 16:58:42 so basically, branch, then disable probibly autosign so newly built packages go to some tag and sit until we have a branched compose 16:58:51 So, I got 2 things 16:58:55 should we open a ticket on this to coordinate? 16:59:02 might need to check with rawhide gating folks too. 16:59:03 Sure 16:59:10 ack 17:00:44 I think I had something else, but I can't remember now. ;) 17:01:34 nirik: Maybe, we could add fxx-staging tag and make autosign to move the builds to that tag and once we have a compose, we can use mass-tag.py to tag them into fxx 17:03:38 I think we don't need to add any new tag, just config autosign to not sign/move things... then when we have a compose we just retag everything and reconfigure autosign to process it again as normal 17:04:06 * nirik has another meeting... 17:05:58 Okay, we can discuss in that ticket 17:06:25 The only reason why I thought of adding another tag is not to lag the signing when we decide to set things again to normal 17:06:43 Thanks for joining everyone 17:07:10 If you have anything else to discuss, lets take it to #fedora-releng 17:07:15 #endmeeting