16:00:18 <mboddu> #startmeeting RELENG (2020-02-19) 16:00:18 <zodbot> Meeting started Wed Feb 19 16:00:18 2020 UTC. 16:00:18 <zodbot> This meeting is logged and archived in a public location. 16:00:18 <zodbot> The chair is mboddu. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:18 <zodbot> The meeting name has been set to 'releng_(2020-02-19)' 16:00:19 <mboddu> #meetingname releng 16:00:19 <mboddu> #chair nirik sharkcz pbrobinson pingou mboddu dustymabe ksinny jednorozec 16:00:19 <mboddu> #topic init process 16:00:19 <zodbot> The meeting name has been set to 'releng' 16:00:19 <zodbot> Current chairs: dustymabe jednorozec ksinny mboddu nirik pbrobinson pingou sharkcz 16:00:28 <mboddu> Thanks nirik for reminding it 16:00:32 <dustymabe> .hello2 16:00:33 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com> 16:00:33 <nirik> hey 16:00:35 * mboddu is still in Tue mode 16:00:44 <nirik> it's still monday to me. ;( 16:01:10 <mboddu> dustymabe: I have a question for you, but lets take it to releng 16:01:16 <mboddu> I mean #fedora-releng 16:01:22 <mboddu> So, lets get started 16:02:06 <dustymabe> mboddu: ok 16:03:17 <mboddu> #topic #9266 prune EOL content from OSTree repos 16:03:23 <mboddu> #link https://pagure.io/releng/issue/9266 16:03:46 <mboddu> dustymabe: So, finally pruning stuff :) 16:04:31 <dustymabe> mboddu: correct :) 16:04:48 <dustymabe> mboddu: we can do this in a few different ways 16:05:05 <dustymabe> we can either prune everything AFTER we do the migration 16:05:09 <dustymabe> or we can do it before 16:05:10 <mboddu> What does "--no-prune" will give us from the first script in that ticket? 16:05:23 <mboddu> Better do it after migration, right? 16:05:40 <dustymabe> mboddu: `--no-prune` is like `--dry-run` 16:05:48 <mboddu> Ah okay 16:05:56 <dustymabe> mboddu: yeah maybe after migration is better 16:06:06 <dustymabe> looks like we have 2T volume so it should all fit 16:07:13 <dustymabe> so maybe we should talk migration strategy 16:08:31 <mboddu> Okay 16:08:43 <dustymabe> what do you folks think about how we should pull off the migration? 16:08:54 <mboddu> dustymabe: 2T in cloudfront or somewhere else? 16:09:59 <dustymabe> mboddu: it's a 2T netapp volume 16:10:05 <nirik> I think like any other data move: 16:10:12 <dustymabe> see https://pagure.io/releng/issue/8811#comment-602395 16:10:48 <nirik> sync data, schedule time for no changes, do final sync, switch web/pointers to the new location 16:11:43 <dustymabe> nirik: so we'd want to do it at a time where no pungi composes are happening, correct? 16:11:49 <nirik> yeah 16:12:11 <mboddu> nirik: If we are only touching 27 and 28, we can do it at anytime 16:12:25 <dustymabe> mboddu: we're migrating the entire repo 16:12:45 <dustymabe> since pungi still makes silverblue, we have to care about pungi 16:12:46 <mboddu> Okay, then dont mind me, you guys are right 16:13:18 * mboddu thought we are only migrating eol'd content 16:13:53 <dustymabe> nirik: so should we enumerate what needs to happen 16:13:59 <dustymabe> 1. make backup of content 16:14:11 <dustymabe> ^^ I guess this can just be "leave the content in place" 16:14:12 <nirik> backup? 16:14:18 <nirik> ah yeah 16:14:39 <dustymabe> ok. so let's do it different 16:14:49 <dustymabe> 1. make sure no writers to ostree repos are running 16:15:01 <dustymabe> ^^ includes pungi and new-updates sync 16:15:19 <dustymabe> 2. copy content from production repos into new netapp volume 16:15:44 <dustymabe> 3. maybe mv prod repos to another directory (minimizes confusion) 16:16:19 <dustymabe> 4. set up /mnt/koji/ostree /mnt/koji/compose/ostree mounts on composer to point to new netapp volume 16:16:37 <dustymabe> 5. do we need to do anything to our webservers/cloudfront? 16:16:55 <nirik> I'd add a '0. make initial copy of content' which we can do anytime and will make step 2 much quicker 16:17:15 <nirik> as long as the new volumes are in the same path for webserver we should be fine for cloudfront 16:19:11 <dustymabe> nirik: i.e. an initial rsync and then a second rsync will take less time? 16:19:20 <nirik> yeah. 16:19:29 <dustymabe> ok sounds good to me 16:19:34 <dustymabe> anything else missing? 16:19:34 <nirik> since it will only need to copy the changes the second time 16:19:52 <nirik> when did we want to schedule it? 16:20:03 <dustymabe> tomorrow? 16:20:03 <mboddu> On that, next week is freeze time 16:20:32 <dustymabe> we can do the initial rsync today 16:20:40 <nirik> I suppose tomorrow might work... 16:22:03 <mboddu> #info We will work on migration first and then pruning, nirik will start working on migration tomorrow and then mboddu will work on pruning later 16:22:38 <dustymabe> mboddu: i should be able to handle pruning myself once we get things over into openshift 16:22:44 <nirik> I can't start anything today until this afternoon. I am in meetings all morning 16:23:11 <nirik> but I can look this afternoon and we can try and do the switch tomorrow after branched/rawhide finish? 16:23:30 <dustymabe> nirik: +1 i'll grab you in 3 or 4 hours 16:23:37 <dustymabe> if that's ok 16:24:11 <nirik> more like 4-5... but feel free to ping me and I'll let you know 16:24:25 <mboddu> #undo 16:24:25 <zodbot> Removing item from minutes: INFO by mboddu at 16:22:03 : We will work on migration first and then pruning, nirik will start working on migration tomorrow and then mboddu will work on pruning later 16:24:40 <mboddu> #info We will work on migration first and then pruning, nirik will start working on migration tomorrow and then dustymabe will work on pruning later 16:24:47 <mboddu> mboddu will help as needed :) 16:26:54 <dustymabe> +1 16:27:00 <dustymabe> ok i gotta go run my other meeting 16:27:03 <dustymabe> thanks mboddu and nirik 16:27:55 <mboddu> Thanks dustymabe 16:27:58 <mboddu> Okay, lets go back to old tickets 16:28:05 <mboddu> #topic #7400 Rethink how we handle networking config with createImage (oz / ImageFactory) 16:28:12 <mboddu> #link https://pagure.io/releng/issue/7400 16:30:26 * mboddu trying to find koji ticket 16:30:34 <nirik> was there one? 16:31:20 <mboddu> I dont think so 16:31:50 <mboddu> #info mboddu will create a upstream koji ticket and will check with mikem 16:31:54 <nirik> so we could file one I guess.. I don't have time to commit to do this one... 16:32:33 <nirik> sounds good 16:32:46 <mboddu> nirik: Understandable 16:34:27 <mboddu> #topic #7555 scratch permission to build livecd for users 16:34:34 <mboddu> #link https://pagure.io/releng/issue/7555 16:35:16 <nirik> so, not sure whats left here... 16:35:25 <nirik> are our compose images still scratch? 16:35:31 <mboddu> nirik: Yes 16:35:49 <mboddu> But can we give the users 'livemedia' permissions? 16:36:19 <nirik> hum, I wonder how that interacts with GC 16:37:24 <bcotton> fwiw (i haven't read through the entire ticket), but that change proposal was never submitted 16:38:34 <nirik> so, it looks like our cloud images at least are normal... but we never move them to updates tags so they get GCed 16:38:36 <mboddu> Sorry, they are not running using scratch 16:39:31 <nirik> and even our GA ones get GCed. ;( 16:40:02 <mboddu> But we import them into our compose, so..., thats still fine 16:40:23 <nirik> Wed Oct 23 17:08:41 2019: Fedora-Xfce-31-1.9 tagged into f31-updates-candidate by releng 16:40:23 <nirik> Sat Oct 26 12:15:33 2019: Fedora-Xfce-31-1.9 untagged from f31-updates-candidate by oscar 16:40:23 <nirik> Fri Jan 10 12:24:35 2020: Fedora-Xfce-31-1.9 tagged into trashcan by oscar 16:40:23 <nirik> Sat Feb 8 01:39:33 2020: Fedora-Xfce-31-1.9 untagged from trashcan by oscar 16:40:55 <nirik> so, we might want to start tagging some of them to keep em... but I guess we still have them on ftp / mirror 16:41:07 <mboddu> Exactly 16:41:30 <nirik> anyhow, I'd be open to granting some people perms to spin media... 16:41:33 <mboddu> bcotton: Yeah, I haven't seen the change as well 16:41:48 <nirik> this might go hand in hand with the idea of having a acutal list of maintainers. 16:41:53 * kalev is interested to spin Workstation test composes. 16:42:22 * Southern_Gentlem interested 16:42:32 <mboddu> nirik: Its good to give them to spins/labs maintainers, but what about people who want add new media? 16:42:47 <mboddu> s/who want/who want to/ 16:42:51 <nirik> sure, we could just also do it on a case by case basis? 16:44:16 <nirik> I can't recall... can you use a local ks for a scratch livemedia build? 16:44:56 <mboddu> I am not sure either, but I remember giving a ks file from my fork, so that should help 16:45:59 <nirik> yeah, looks like no, but I think it will accept any pagure.io git for them 16:46:26 <mboddu> So, how should we handle this? 16:46:30 <nirik> so, whats next steps here? 16:46:31 <nirik> ha. 16:47:02 <mboddu> Lets give it couple of folks who need it and see how they use it and get feedback and then announce it out for people who want it? 16:47:09 <nirik> how about we grant Southern_Gentlem and kalev perm and ask them to try it out and report back to us if it's useful/working? 16:47:21 <bcotton> +! 16:47:24 <nirik> nice. great minds think alike. ;) 16:47:27 <mboddu> Lol, exactly the same thing twice :) 16:47:27 <bcotton> +1 too 16:47:43 <mboddu> Southern_Gentlem and kalev : Are you okay with this? 16:47:47 <Southern_Gentlem> go with kalev first because i will need to work with someone to learn what i am doing 16:48:26 <kalev> mboddu: sure, works for me 16:48:53 <mboddu> Southern_Gentlem: Okay, but its simple 16:49:11 <mboddu> .fasinfo kalev 16:49:12 <zodbot> mboddu: User: kalev, Name: Kalev Lember, email: klember@redhat.com, Creation: 2009-02-25, IRC Nick: kalev, Timezone: Europe/Stockholm, Locale: en, GPG key ID: 8F62AF74, Status: active 16:49:13 <nirik> I am wondering if the limitations will make it not so useful, but we can see. 16:49:14 <zodbot> mboddu: Approved Groups: cla_fedora cla_done fedorabugs provenpackager cla_fpca gitspin-kickstarts qa +packager hosted-content @gnome-sig 16:49:25 <Southern_Gentlem> mboddu, i have been doing respins for years on my own equipment 16:49:35 <nirik> you do have to pass it a install tree... so it has to use a existing one of those... 16:49:48 <mboddu> nirik: install tree and the repo as well 16:50:03 <mboddu> Southern_Gentlem: Right, it should definitely help you a lot 16:50:38 <mboddu> kalev: You should have the permissions now 16:50:44 <kalev> nice, thanks! 16:50:45 <nirik> repo for ks yeah 16:51:24 <nirik> koji spin-livemedia --help 16:51:37 <nirik> make sure to use --scratch :) 16:51:50 <mboddu> kalev: Right 16:51:55 <kalev> ok, let me give it a try :) 16:53:32 <nirik> mboddu: 7min left... anything else? 16:53:52 <bcotton> o/ 16:55:24 <mboddu> #topic Open Floor 16:55:38 * bcotton has a thing 16:55:38 <nirik> freeze and bodhi enable is next tuesday? 16:55:45 <mboddu> nirik: So, I found the issue with stuck 32 compose, I will work with dustymabe on that 16:55:49 <bcotton> nirik: correct 16:55:50 <mboddu> nirik: yes 16:56:20 <nirik> I have something for that. can we change the time as we discussed a bunch last cycle? pretty please? 16:58:09 <mboddu> nirik: ? 16:58:15 <mboddu> Which one are you talking about? 16:58:43 <nirik> the freeze starting at 00:00 tuesday... this is confusing to everyone 16:59:06 <kalev> ah yes, can you just move it one minute earlier? no real changes, just makes it much more clear when it actually starts 16:59:37 <kalev> first test scratch build running: https://koji.fedoraproject.org/koji/taskinfo?taskID=41658761 16:59:41 <nirik> I was hoping to move to to something like tuesday at 14UTC (or 15, whatever we normally release on) 17:00:06 <mboddu> nirik: Ah, right, 14UTC seems okay 17:00:09 <mboddu> bcotton: ^ 17:00:20 <nirik> or later. or whatever. just not 00:00 17:00:43 <kalev> agreed, just as long as it's not 00:00 :) 17:00:44 <bcotton> WFM, so long as everyone has a shared understanding 17:00:55 <bcotton> i can update the schedule text to say 1400 UTC 17:01:18 <nirik> mboddu: can you be around at 14utc to enable bodhi? 17:01:26 <Southern_Gentlem> it would be more consistant across the board, people know we release around 1400 17:01:48 <mboddu> nirik: Yes, I would be around 17:02:02 <nirik> if everyone is ok with that, perhaps a email this week with a reminder/note about it to devel-announce would be good. 17:02:12 <nirik> and we should do the same for final, etc. 17:02:36 <bcotton> nirik++ 17:02:42 <nirik> so for final that would mean a updates push at 14 and then freezing. 17:03:24 <bcotton> #link https://pagure.io/fedora-project-schedule/issue/175 17:03:25 <mboddu> Right 17:03:33 <mboddu> Thanks bcotton 17:03:45 <mboddu> So, lets close this for today 17:03:49 * mboddu got to run 17:03:54 <mboddu> Thanks everyone for joining 17:03:55 <bcotton> i'll try to get the schedule updated by EOB tomorrow so that you can point to it when sending the remidner 17:04:12 <mboddu> bcotton: Okay, please ping me once its done 17:04:12 <nirik> bcotton++ 17:04:12 <zodbot> nirik: Karma for bcotton changed to 12 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:04:19 <bcotton> mboddu: ack 17:04:20 <nirik> thanks mboddu 17:04:25 <mboddu> bcotton++ nirik++ kalev++ Southern_Gentlem++ 17:04:26 <zodbot> mboddu: Karma for bcotton changed to 13 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:04:29 <zodbot> mboddu: Karma for jbwillia changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:04:39 <mboddu> #endmeeting