15:32:49 #startmeeting RELENG (2014-07-21) 15:32:49 Meeting started Mon Jul 21 15:32:49 2014 UTC. The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:32:49 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:32:54 #meetingname releng 15:32:54 The meeting name has been set to 'releng' 15:32:55 #chair dgilmore nirik tyll sharkcz bochecha masta 15:32:55 Current chairs: bochecha dgilmore masta nirik sharkcz tyll 15:32:56 #topic init process 15:33:10 * nirik is here, in another meeting and poking at alerts. ;) 15:33:14 * sharkcz is here 15:33:19 * masta is here 15:33:42 * jreznik is listening 15:33:51 * bochecha is here 15:34:23 #topic #5931 [Proposal] Move new branch and unretire requests to pkgdb2 15:34:28 https://fedorahosted.org/rel-eng/ticket/5931 15:34:39 pingou: ping 15:35:05 pong 15:35:36 * pingou finds the email 15:35:42 pingou: any updates> 15:35:44 ? 15:35:56 https://lists.fedoraproject.org/pipermail/rel-eng/2014-July/018135.html 15:36:14 so here I presented the different options we talked about at the last meeting 15:36:39 afaica, we have two options 15:36:44 afaics* 15:37:01 either we wait for the review-server (which hasn't been started yet) 15:37:20 or we can already start w/ bugzill/pkgdb2 knowing that the procedure might change in the future again 15:38:03 if we decide to go with the current setup 15:38:18 pingou: I guess the question is, with teh review app would we rather do it in pkgdb or the review app? 15:38:32 there is then the question, do we wait for bugzilla/fedmsg integration (procedure 2 on that email) or not? 15:38:43 I think thats getting near... 15:38:44 * dgilmore kinda guesses pkgdb, so adding epel branches etc is easier 15:39:07 dgilmore: in my mind, the reviewee would specify the branch(es) or interest when the review is created 15:39:10 pingou: I do not want packages added to pkgdb until reviewed by an admin 15:39:12 (w/ option to edit it after) 15:40:25 dgilmore: I liked the idea of adding the package (not the branch) automatically, because then it's the same process for the user to request branches for the first time or for new branch 15:40:36 pingou: so it sounds like you think being in the review app is best 15:40:49 and we can easily flag to the admin that this package was just added, has no branch and thus review should be checked 15:40:59 pingou: the issue is that once its added people will assume its in 15:41:16 fair point 15:41:17 pingou: its all to do with perception 15:41:29 true :) 15:41:36 even if it has no branches, and it has a « pending » status? 15:41:37 if we add it then people will try push dfor branches to be done automatically 15:41:42 bochecha: yes 15:41:59 because peolke will go its added, just make the branches automatically 15:42:10 dgilmore: do you like the procedure 1? the user ask for the package to be created w/ the branches at the same time 15:42:19 people are weird and jump to conclusions 15:43:13 pingou: I am okay with it. some of the checks we should be able to automates 15:43:17 automate 15:43:30 sure 15:43:36 checking that the people are packagers 15:43:38 etc 15:43:48 the question is more regarding the process 15:43:59 then we can see which point we can automate which shouldn't 15:44:00 adding a check that packages being added to epel are not in rhel 15:44:38 actually we allow packages in EPEL if they are not in all arches in RHEL, so it might be more complicated 15:45:00 pingou: until we get a review app I think procedure 1 is best 15:45:01 yeah. 15:45:04 in the review-app, we could simply have a button -> Create package request on pkgdb2 for branch: A, B, C 15:45:12 (once the review has been done) 15:45:13 tyll_: I kinda wish we didn't do that... but oh well, we did. ;) 15:45:32 pingou: procedure 2 id be okay with if we changed adding the package to building a queue for admin requests 15:46:24 dgilmore: so flag change broadcasted -> package request created on pkgdb2 -> cvsadmin... ? 15:46:54 tyll_: right. we really need some tooling for enforcement of that policy properlly. tyll_ but thats something we can check 15:47:10 assuming we track what arches get what packages 15:47:39 pingou: yeah, as long as the request created does not add the package 15:47:46 stupid question but there isn't an api we could query to know which packages are in which channel in rhel, right? 15:47:56 pingou: there is not 15:47:59 dgilmore: that's the idea yes, just like in procedure #1 15:48:16 pingou, there is the Yum API :x 15:48:21 pingou: right, if we did that request bit automatically id be okay with it 15:48:23 so the only way is to get a subscription and then use regular yum way ? 15:48:35 misc: pretty much yes 15:48:50 the api is to query the repo metadata of rhel7 or cent7 I guess 15:48:53 maybe we need something that provides an API 15:49:06 masta: rhel 5 6 and 7 15:49:11 dgilmore: cool, I'll update both procedure on the list then 15:49:13 we can not use CentOS 15:49:29 dgilmore: and a subscription with all access :/ 15:49:38 CentOS and RHEL ship differently, and the problem is all in rhel 15:49:51 sounds like a fun project... 15:49:54 misc: well to the content we care about 15:50:11 sounds like the pkgwat api for rhel basically 15:50:51 nirik: threebean there is no way we could set-up a packages application (reduced?) for the RHEL channel we've interest in? 15:51:08 (most likely a private instance I guess) 15:51:21 pingou: huh, dunno. I guess thats an idea... or some way to add it to regular pkgdb but keep it hidden? 15:51:43 nirik: better idea: internal pkgdb :) 15:52:00 then we have the pkgdb2 api :) 15:52:04 pingou, or just like koji has « external repos » for rhel, pkgdb could have « external repos »? 15:52:37 bochecha: not sure how it works in koji, so I can't answer that one :) 15:53:06 similiar idea but implementation would need to be completely different 15:53:42 dgilmore, for sure, but it could be nice to show « what we depend on » in pkgdb in a similar way to koji's external repos 15:54:14 the host pkgdb runs on can probably access the infrastructure RHEL repos to query for packages in RHEL 15:54:24 * pingou wants to keep pkgdb focused on what it does (do one thing, do it well) 15:54:42 pingou: right 15:54:56 this is a bit of a sidetrack 15:55:09 its a problem that would be really good to solve 15:55:17 but doent need dolved now 15:55:17 I'll summarize the two approaches we have on the list and we can start from there 15:55:24 pingou: okay thanks 15:55:36 sure 15:55:50 thanks pingou =) 15:55:51 #action pingou to summarise the two approaches on the list and start on implementation 15:56:05 masta: wait for it ;-) 15:56:17 #topic #5870 rawhide signing 15:56:23 https://fedorahosted.org/rel-eng/ticket/5870 15:56:39 tyll_: any changes here? 15:56:48 access to secondary signers seems to be working now 15:57:01 but I am lacking time to do something with it :-( 15:57:04 tyll_: awesome. I think I need to give you key access 15:57:07 okay 15:57:47 yes, I probably need access to both new keys - not sure, you might also need to revoke access the the f21 keys for me 15:57:48 yes. 15:57:58 I fixed this. was waiting for tyll_ to confirm. ;) 15:58:00 one note here - signing all rawhide on secondaries will increase the usage of the disk space 15:58:38 tyll_: right, will need to give you f22 key access 15:58:46 and secondary arch 15:58:51 sharkcz: that's a good point 15:59:01 sharkcz: how are you on space right now? 15:59:18 vtap-fedora-nfs01.storage.phx2.redhat.com:/vol/fedora_ppc/data 15:59:18 9.0T 6.8T 2.3T 76% /mnt/koji 15:59:25 vtap-fedora-nfs01.storage.phx2.redhat.com:/vol/fedora_s390/data 15:59:25 5.9T 5.3T 653G 90% /mnt/koji 15:59:25 sharkcz: we need to work on setting up cleaning up old signed copies 16:00:03 dgilmore: yeah, that was my backup idea, to remove the signed non-latest copies 16:00:11 yeah, we can always restore the signed package if we need them from the headers, right? 16:00:15 and primary is also getting low on space. 16:00:30 and we don't have space to grow. I will try and bug storage folks more. 16:00:49 masta: yes 16:01:03 we can always re write out signed copies 16:01:25 nirik: how much of primary is snapshots? 16:01:28 as part of the signing on rawhide, could we also remove the previous one? 16:01:38 nirik: we could 16:01:50 not sure how to tell... we have 10days of snapshots. 16:02:24 I know i deleted about 3 months work of rawhide and epel beta composes last week 16:02:31 worth 16:02:32 looks like 1.3tb in snapshots 16:02:40 I also cleaned up updates mash repos 16:02:51 there is about 1T i nightly image composes also 16:03:09 also, scratch builds aren't fully getting cleaned up right. 16:03:15 I should file a bug on that. 16:03:33 nirik: its a cronjob thats managed in puppet that does it 16:03:34 so at least, we probibly need to add some s390 space soon. 16:03:53 dgilmore: yeah, but there's a MANIFEST file thats not a link to the scratch space... so the dir never gets cleaned up 16:04:57 nirik: okay, thats likely a koji bug 16:04:57 https://kojipkgs.fedoraproject.org//work/tasks/7357/7067357/ 16:05:00 for example. 16:05:14 the rest of them were nuked because they were in scratch, but that wasn't. 16:05:27 not sure what makes that file... oz? 16:05:33 okay, ill see if i can fix that today, I need to pull in a couple of patches for docker images 16:05:46 need to update koji 16:05:47 want me to file a bug, or you got it? 16:06:23 I got it 16:07:02 cool. thanks. 16:08:02 anything else on rawhide siging? 16:08:19 * oddshocks pops in to lurk on the meeting 16:08:31 #topic #5914 Move fedmsg based blocking service to Fedora Infrastructure 16:08:42 there is nothing new :-( 16:09:17 tyll_: no problem. if there is something someone can do to get it moving please let us know 16:10:20 dgilmore: ok, but mainly it needs to be adjusted/re-written to be a proper service 16:10:49 tyll_: okay, is the existing code somewhere that people could work on and send you patches? 16:12:31 dgilmore: no, I can see if I get to publish it somewhere, but I am not yet sure how it needs regarding a license, because I copied some code from elsewhere 16:12:54 but it is also not much code 16:12:58 tyll_: okay. 16:14:26 alright lets move on 16:14:29 #topic Secondary Architectures updates 16:14:30 #topic Secondary Architectures update - ppc 16:14:37 masta: you're up 16:15:17 ppc is waiting for updated python2 in primary so it can be rebuilt on ppc64le with the correct libffi 16:15:32 okay thanks sharkcz 16:15:48 hey guys 16:15:52 there were problems with openssl (fixed now) and glibc (rhbz#1119769) 16:16:12 :( seems there has been a bunch of glibc issues recently 16:16:44 yeah, the openssl thing - they turned off sslv2,v3... I'm behind on signing, mashing, and pushing... I'm hoping to get that done this morning. 16:17:48 anything else? 16:17:59 #topic Secondary Architectures update - s390 16:18:13 sharkcz: you're up 16:18:24 gcc upstream has all the required fixes, I', waiting for an updated package 16:18:31 great 16:18:39 finially getting there :) 16:19:05 yeah :-) but all the bugs were general bugs, s390 was only used to show them 16:19:19 which is good 16:19:51 if only they showed up in other arches better 16:21:33 indeed 16:21:44 #topic Secondary Architectures update - arm 16:21:49 statistics: {'older': 221, 'local_only': 0, 'remote_only': 407, 'same': 14837, 'newer': 2, 'total_missing_builds': 366} 16:21:59 aarch64 is doing quite well 16:22:21 I know the virt guys are working on some issues to mnake virt work right 16:22:53 that will be nice 16:23:09 I did make a Fedora install tree last week 16:23:13 I need to test it 16:23:26 other things first though 16:23:43 there was/is an issue with smartd triggering kernel bugs 16:23:57 dgilmore: we might get you another board for crashing/testing 16:23:59 its a kernel regression, what wentupstream was incomplete 16:24:06 masta: I asked 16:24:24 we are getting a couple of aarch64 boxes in phx2 this week 16:24:43 so we will be able to do composes where koji and the storage is 16:24:51 dgilmore: we can use my board for the crashing right now, I can always jtag the thing back to life. 16:25:08 masta: no need for jtag 16:25:13 just need to do pxe installs 16:26:02 dgilmore: ok, easy enough. we can talk about that later. 16:26:17 okay 16:26:23 #topic next week 16:26:38 so next weeks meeting ill be on my way to Brno 16:26:48 I need someone to run the meeting next week 16:26:54 https://fedoraproject.org/wiki/ReleaseEngineering/Meeting_Process 16:26:57 is the process 16:27:34 I probibly could I guess... 16:27:47 nirik: thanks 16:27:56 I can also do the meeting.. 16:27:59 I may be in Brno in time 16:28:07 but I do not think i will be 16:28:15 masta: fine with me if you can... ;) 16:28:26 nirik: ok then =) 16:28:32 masta: its all yours 16:28:41 #action masta to run next weeks meeting 16:28:41 dgilmore: thanks for sharing the meeting process 16:29:32 #topic Open Floor 16:29:34 okay 16:29:39 so one item from me 16:29:52 right now we can not compose a whole tree 16:29:53 so I'm on package signing this week 16:30:11 still trying to pull in atomic tree composing 16:30:34 need to work out why i386 composes are failing 16:30:35 dgilmore: I'd like to expand my signing to aarch64, I'm guessing the secondary key would work there too, yes/no? 16:30:58 dgilmore: and we go to bodhi enabled branched now right? 16:31:12 or after tomorrow's compose? 16:31:29 masta: yes, i have branched being signed right now 16:31:38 nirik: yeah we do 16:31:55 I think we enable bodhi in the morning tomorrow 16:32:08 ok 16:32:22 right. 16:32:22 I have just a reminder for the 2 tickets I opened - the pkg sync issue with secondaries, and the pre-GA f20-updates-testing thing 16:32:35 masta: that means there will be different pushing for primary after tomorrow: 16:32:36 dgilmore the broken i386 means we haven't had any cloud images to test for a long time 16:32:54 I know you know that. Just mentioning it for the record. :) 16:32:56 push just f21 updates-testing, then 19/20, then later in the evening push f21 stable 16:33:23 nirik: ok, so right now I'm just doing f20 and f19 signing, but tomorrow we will do f21 (but not f21 updates), right? 16:33:40 mattdm: yeah. there is lots of broken bits still 16:33:43 :( 16:33:59 And the other open floor topic -- docker images 16:34:01 nirik: ok, I think I got that 16:34:04 masta: after we get bodhi enabled tomorrow, you will have to do them in that order. ;) 16:34:16 (not the atomic cloud images for running docker -- official docker base images) 16:34:30 nirik: yes sir, I better take a note of that to make sure it sinks in to my thick head. 16:34:31 since we have the go-ahead from legal the next steps are to figure out how to actually do it 16:34:37 nirik: well there will be no f21 stable pushes until after Alpha is out 16:34:42 masta: yeah, it's kinda confusing... 16:34:46 dgilmore: oh yeah, good point. 16:34:53 mattdm: im going to patch koji today for docker 16:34:56 so scratch that stable until after alpha 16:35:14 dgilmore cool! and then that takes a kickstart as input? 16:35:26 dgilmore: mattdm: i sent a revised patch for kickstart file 16:35:31 mattdm: yes, it uses anaconda and oz/imagefactory to make kit 16:35:34 make it 16:35:51 dgilmore ok awesome. should we add lsm5's kickstart to spin-kickstarts? 16:35:58 lsm5: where? can you send it to the spins list please 16:36:08 dgilmore: sure thing.. 16:36:12 masta: so, f21 just updates-testing then 19/20 normal. 16:36:14 mattdm: likely yes 16:36:55 okay, so then, we will have the tarball of / as an output. that needs to go to github somewhere and somehow (since that's part of upstream docker's workflow) 16:37:11 nirik: ok I'll have to adjust my signing process... it's mostly scripted.. so I'll share my process before I pull the trigger. catchup with you later on this. 16:37:14 mattdm: ive not looked at how that all works 16:37:23 mattdm: but something like that 16:37:50 dgilmore: mattdm btw, i'm collaborator on the unprefixed fedora image .. 16:38:06 dgilmore we need to push the binary image into github. then, when we want an update, someone (lsm5?) makes a pull-request against https://github.com/dotcloud/stackbrew/tree/master/library 16:38:18 lsm5: that needs to be all sorted out so that its in releng control 16:38:19 yup 16:38:26 dgilmore: ack 16:38:47 dgilmore ack. is there an existing releng-controlled github account? 16:38:57 lol 16:39:03 #action dgilmore get thinsg sorted out to upload docker images 16:39:07 mattdm: how about the fedora-cloud org on github? 16:39:21 note that here we are just using github as a conduit, not for development or even meaninful hosting 16:39:24 mattdm: there is not. Im guessing that legal will be oky with it 16:39:30 dgilmore: are you cool with using github? 16:39:42 mattdm: i personally choose to not use github at all 16:39:57 but I guess there is no choice 16:40:19 dgilmore yeah, understood. think of it as something like using amazon's web interface for managing images there 16:40:33 dgilmore: I'm working for you: http://209.132.184.222/progit ;-) 16:40:35 mattdm: right 16:41:35 pingou: :) 16:41:35 lsm5 if we can work out some non-github way for stackbrew to get images _directly from koji_ that might be even better 16:42:03 mattdm: from koji or some other location 16:42:19 dgilmore, lsm5 yes. ftp or whatever 16:42:22 mattdm: I have no idea how we do things like testing/stable either 16:42:24 or a git repo we host 16:42:43 something we will have to work out 16:42:47 mattdm: i'll check on that.. 16:43:03 and how we do things like keep f21 and rawhide images seperate 16:43:07 lsm5 thanks. that'll be more direct and seems better. 16:43:33 dgilmore right now, there is a top level "fedora" repo which cna have multiple images, which can be tagged with "f21" or "rawhide" or whatever 16:43:45 dgilmore: mattdm keeping images separate might be trouble, last time I checked, docker guys wanted it to be 1 image with different tags 16:43:49 in the future, we will have a fedora/ namespace we can use for fedora/f21, fedora/rawhide, fedora/whatever 16:44:09 lsm5 okay, or maybe we won't. :) 16:44:11 mattdm: okay, we need to work out how to do all of that 16:44:13 but at least we have the tags 16:44:36 maybe we need our own docker registry 16:44:59 just bevcause it really seems like docker is very shortsighted 16:45:07 dgilmore yes I think that's part of long term plans, but we also want to be visible in the public registry 16:45:09 no multiple arch support 16:45:22 seems maybe no real release support 16:45:37 anyway problems for another day 16:45:48 yep :) 16:46:03 there is a big player who wants to add multi arch support, fortunately, so it should change 16:46:06 anyway I have to go. just wanted to make sure that was moving forward. 16:46:12 anyone have anything else to discuss? 16:46:19 mattdm: cheers and thanks 16:46:26 if we need more help on anything docker related let me know and I'll try to find someone to pitch in 16:46:41 I just wanted to note that migration of distgit to ansible is pending review (nirik? dgilmore?) 16:46:56 once that's done, we can proceed with the move away from md5 :) 16:46:56 bochecha: where is the patches? 16:47:02 bochecha: yep. I've just had no time to work on it... 16:47:12 bochecha: next week seems possible. 16:47:20 https://fedorahosted.org/fedora-infrastructure/ticket/4452 16:47:24 infra should be frozen now 16:47:31 dgilmore, even staging? 16:47:33 dgilmore: this would be stg 16:47:37 bochecha: not staging 16:47:41 nirik, awesome :) 16:47:49 if stg goes ok, we could look at after alpha. 16:48:02 for prod 16:48:07 okay 16:48:12 also, I've started working on the script to copy/hardlink all the source files to the new path based on sha512 16:48:23 where should that script live? (so that I send the patches to the right place) 16:48:44 releng list I guess? 16:48:59 we could also just depend on netapp dedupe. ;) 16:49:15 bochecha: releng list 16:49:50 dgilmore, but which module/git ? 16:50:03 or do I just send the script itself? 16:50:17 bochecha: eithers fine 16:50:23 it would be in infrastructure with the ansible distgit stuff I assume (eventually) 16:50:31 probably releng or infra git repos 16:50:32 alright 16:50:40 or just straight into ansible 16:50:54 I'll keep working on that then, and send it for review when I have something ready :) 16:50:54 doesnt really matter too much 16:52:53 if there is nothing else ill close the meeting 16:53:39 nothing from me 16:57:19 #endmeeting