15:32:33 <dgilmore> #startmeeting RELENG (2014-10-06) 15:32:33 <zodbot> Meeting started Mon Oct 6 15:32:33 2014 UTC. The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:32:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:32:43 <dgilmore> #meetingname releng 15:32:43 <zodbot> The meeting name has been set to 'releng' 15:32:43 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson 15:32:43 <zodbot> Current chairs: bochecha dgilmore masta nirik pbrobinson sharkcz tyll 15:32:44 <dgilmore> #topic init process 15:32:50 <nirik> morning 15:32:52 <dgilmore> who is here 15:32:56 <bochecha> good evening 15:33:03 * sharkcz is here 15:33:06 * pbrobinson waves 15:33:07 * sgallagh lurks 15:34:01 <dgilmore> #topic #6014 multiple builds in primary koji are lackign arm buildroot info 15:34:29 <dgilmore> this seems to be some fallout from the koji db move/outage last week 15:34:36 <nirik> yeah ;( 15:34:44 <nirik> sorry about that. I tried to be carefull... not sure what happened. 15:34:45 <dgilmore> I think we should just bump and rebuild any effected builds 15:35:01 <dgilmore> going forward we should just be sure to switch the hubs off for any migration 15:35:01 <nirik> sounds like a good workaround 15:35:08 <pbrobinson> +1 15:35:15 <nirik> yeah. ;( 15:35:53 <sharkcz> I've already rebuilt gcc-python-plugin, the rest seem to fixable by a hack in koji-shadow 15:36:05 <dgilmore> sharkcz: i would rather just rebuild 15:36:15 <sharkcz> don't disagree 15:36:47 <dgilmore> #action all rebuild all effected packages 15:36:53 * nirik grabs more coffee. back in a sec. 15:37:35 <dgilmore> anything else on this? 15:37:53 <sharkcz> nope 15:38:00 <dgilmore> #topic #5886 need method for distributing urgent fixes... urgently 15:38:03 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5886 15:38:31 <dgilmore> so we really didnt discuss this during the week like we said we would 15:39:15 <tyll> Hi there 15:39:34 <dgilmore> I think we are in agreement that we need something 15:40:20 <tyll> iirc nirik prepared a wiki draft page, but I did not yet take a look 15:40:35 <nirik> yeah, I didn't do much more on it after that. 15:40:42 <dgilmore> https://fedoraproject.org/wiki/Urgent_updates_policy 15:40:54 <nirik> there' some questions there we could discuss... 15:41:10 <dgilmore> maybe we should leave this for the end. 15:41:19 <dgilmore> I really do not want to go over an hour 15:41:24 <nirik> alright. 15:41:35 <dgilmore> I have a crap load I need to get done before 3pm tomorrow 15:41:42 <tyll> #link https://fedoraproject.org/wiki/Urgent_updates_policy 15:42:15 <tyll> #info nothing new 15:42:52 <dgilmore> I am kinda skipping over soem tickets were afaik there is nothing new 15:43:00 <nirik> sounds fine 15:43:01 <dgilmore> #topic #5775 switch to partitioned image for EC2; remove editing of menu.lst from image processing 15:43:07 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5775 15:43:21 <dgilmore> so this is really out of our hands 15:43:36 <dgilmore> oddshock's upload service is supposed to be doing this 15:43:42 <dgilmore> we just need to deploy it 15:43:50 <dgilmore> nirik: any idea where that is at? 15:43:57 <dgilmore> we really need it sorted for Beta 15:44:02 <nirik> there was some issues with x8664 or something... 15:44:09 <nirik> he was working on it... not sure current status 15:44:16 <dgilmore> okay 15:44:41 <dgilmore> I know ill have to setup the fedora account credentials in ansible-private 15:44:50 * nirik nods 15:44:50 * masta looks in 15:45:00 <masta> hey guys 15:46:50 <nirik> hey masta 15:46:56 <dgilmore> #topic #5963 Orphaned vulnerable packages in EPEL 15:47:02 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5963 15:47:31 <dgilmore> I guess we should just go through and retire them all 15:47:53 <tyll> I can do this 15:47:57 <dgilmore> but we do need to be sure of breakages 15:48:14 <nirik> I think we need to notify before we do things. 15:48:22 <dgilmore> nirik: yeah 15:48:27 <nirik> There's been a number of folks posting to epel-devel about packages disappearing. 15:48:31 <tyll> there was a notification on epel-devel when the ticket was filed 15:48:34 <dgilmore> something like we do when retireing packages in fedora 15:48:49 <nirik> dgilmore: right 15:49:02 <dgilmore> nirik: they were all packages incorrectly retired afaik 15:49:19 <nirik> do we have a list of them? should we unretire them? 15:49:21 <dgilmore> and with process fixes they got retired right 15:49:37 <dgilmore> I do not know we have a list of them all 15:49:44 <dgilmore> but I am sure we could get it 15:49:44 <tyll> nirik: they were retired in pkgdb but not blocked in koji 15:50:08 <tyll> now they are also blocked since we now block every retired package when buildbranched/rawhide runs 15:50:38 <nirik> for example someone was asking today about libgeotiff 15:50:49 <nirik> tyll: but that wouldn't affect epel? 15:51:27 <tyll> nirik: yes, it does, it does this for all branches that we allow packages to be retired 15:51:58 <tyll> nirik: it is just in these scripts to make sure everything is blocked before they run 15:51:58 <nirik> ah... that seems not ideal 15:52:24 <tyll> otherwise we would just have a second (or more) cron jobs that do the same 15:52:47 <dgilmore> nirik: not sure how it seems not ideal 15:53:00 <dgilmore> its acting on the maintainer retiring the package in pkgdb 15:53:02 <nirik> well, I guess they are retired... 15:53:37 <dgilmore> we may need to make some changes for epel 15:53:41 <nirik> I guess the problem was that some epel packages were orphaned, then retired, then blocked, without folks getting a chance to see what would be broken or if they wanted to take them 15:54:03 <dgilmore> to untag all builds and unblock in -build incase it moved to rhel 15:54:34 <tyll> nirik: libgeotiff was retired before we got pkgdb2 15:54:40 <dgilmore> nirik: yeah a lot of that was due to the maintainers not talking about what they are doing 15:54:42 <nirik> ok. 15:54:57 <dgilmore> things were left in zombieland 15:55:00 <tyll> also retiring is not something that is a privileged operation, everybody can do this for orphaned packages afaik 15:55:09 <dgilmore> we went and killed the zombies 15:55:20 <nirik> anyhow, for this ticket we should do the cycle like we do for fedora... 'these are going to be retired, these deps will be borken' ... wait a week... etc 15:55:35 <dgilmore> nirik: yeah for this one we should 15:55:38 <tyll> dgilmore: the build untagging and unblocking in the build tag is already happening for EPEL 15:56:55 <dgilmore> tyll: okay awesome :) 15:58:03 <tyll> I might be able to report dependencies for these packages this week 15:58:52 <dgilmore> #action tyll to work on reporting dependncies to the list for the packages listed in this ticket and follow the fedora orphaned packages practices 15:58:53 <tyll> but maybe I already did it once for all orphaned packages in one EPEL release 15:59:11 <nirik> thanks. 15:59:15 <dgilmore> tyll: if you wanted to do all orphaned I wouldnt object 15:59:39 <dgilmore> anything else? 15:59:42 <nirik> that would be ok with me too. ;) 16:00:02 <dgilmore> #topic Secondary Architectures updates 16:00:03 <tyll> dgilmore: ok, this was also discussed recently in EPSCo 16:00:04 <dgilmore> #topic Secondary Architectures update - ppc 16:00:14 <dgilmore> tyll: okay 16:00:19 <pbrobinson> So we have a test compose out for ppc 16:00:40 <dgilmore> \0/ 16:00:44 <pbrobinson> awaiting IBM to feedback updates but intend to move to RC and possibly to Alpha this week 16:01:14 <pbrobinson> I now have working ability to sign secondary arches which should make things a little bit more straight forward 16:01:17 <dgilmore> we can only ship Release Tues-Thurs 16:01:25 <pbrobinson> OK 16:01:35 <dgilmore> ideally Tuesdays 16:01:38 <nirik> pbrobinson: did you get a chance to try those nfs mounts? 16:01:59 <pbrobinson> nirik: I was going to ask you which host/ip I should be using for the mount point :) 16:02:04 <nirik> let me know if they don't work. Sometimes the netapp is anoying and you have to flush it's export cache. 16:02:17 <nirik> ah, can get that for you. 16:02:42 <pbrobinson> nirik: yea, if you could ping me that I can test :) 16:03:59 <pbrobinson> I think there's a few issues with PPC but it's not looking too bad 16:04:13 <pbrobinson> sharkcz would know more on the package delta 16:04:44 <sharkcz> nothing major it seems - http://ppc.koji.fedoraproject.org/koji/serverstatus 16:05:15 <pbrobinson> sharkcz: cool, I'm doing a signing run as we speak 16:05:26 <sharkcz> pbrobinson: perfect :-) 16:05:38 <dgilmore> anything more ppc? 16:05:49 <pbrobinson> not from me 16:05:50 <sharkcz> no 16:05:52 <dgilmore> #topic Secondary Architectures update - s390 16:06:09 <dgilmore> sharkcz: hows the world according to s390 16:06:34 <sharkcz> I got first compose built with a hacked anaconda (broken deps in anaconda-gui), waiting for an official build 16:06:57 <dgilmore> okay, thats something 16:07:49 <sharkcz> packagewise we are 'older': 131, 'local_only': 13, 'remote_only': 995, 'same': 14618 - which I consider very well 16:07:59 <dgilmore> not too bad 16:08:44 <sharkcz> few perl modules still not rebuilt for the ABI, some staff needs vtk (same on ppc), and couple more 16:09:05 <sharkcz> so considered good :-) 16:09:21 <dgilmore> okay, so are you hoping to skip Alpha and ship Beta? 16:09:32 <sharkcz> yes 16:09:42 <dgilmore> okay 16:09:49 <dgilmore> anything else? 16:09:57 <sharkcz> nope 16:10:02 <dgilmore> #topic Secondary Architectures update - arm 16:10:12 <pbrobinson> so we're looking very good here as well 16:10:25 <pbrobinson> will likely do a RC tomorrow 16:10:34 <pbrobinson> once I'm up to date on package signing 16:10:49 <pbrobinson> and see if there was any fallout from people testing over the weekend 16:10:58 <pbrobinson> package wise it's all in pretty good shape 16:11:29 <nirik> BTW, I have all the arm*:builvm:buildhw:buildppc builders updated and rebooted... finally got the arm SOCs this weekend... 16:11:51 <pbrobinson> cool 16:12:01 <dgilmore> nirik: awesome 16:12:16 <nirik> and all the arm SOCs are working again... 16:12:26 <nirik> they sometimes get in a weird state that needs the chassis rebooted. 16:12:47 <dgilmore> :( 16:13:15 <nirik> (they see their 300GB drive as 8MB... which isn't nice :) 16:13:43 <nirik> we need to get the aarch64's ansibled sometime 16:14:12 <dgilmore> we need to install Alpha on them :) 16:14:19 <nirik> yeah. 16:14:58 <masta> I think they are on pxe, should be doable 16:15:25 <dgilmore> It is doable 16:15:47 <pbrobinson> nirik: would be happy to work with you on the aarch64 ones 16:16:04 <nirik> cool. :) 16:16:21 <dgilmore> we need to move all secondary arch boxes to ansible 16:16:28 <dgilmore> fully ansibilised 16:16:32 <dgilmore> with fas for auth 16:16:33 <nirik> yep. 16:16:36 <masta> yeah, me too... happy to get involved on the builders 16:16:37 <pbrobinson> nirik: and ppc64* 16:16:58 <nirik> yeah. :) 16:18:00 <nirik> what else did we have? ;) 16:18:15 <dgilmore> s390 ;) 16:19:21 <dgilmore> okay 16:19:24 <dgilmore> #topic Open Floor 16:19:35 <dgilmore> I have one item 16:19:41 <nirik> did we want to discuss the urgent updates thing a bit? or leave it for next/in ticket? 16:20:20 <dgilmore> I am going to Australia tomorrow evening for 2 months, with a short jaunt to FUDCon LATAM 16:20:24 <dgilmore> I would like to move the meeting two hours earlier 16:20:47 <nirik> thats pretty early for me, but I can try and make it. 16:21:02 <bochecha> I'd be happy with that time (in fact, happier than with the current time) 16:21:04 <sharkcz> works for me 16:21:47 <masta> That is a bit early, but not unreasonably so. works for me. 16:21:59 <dgilmore> as it is 2:20am in australia right now 16:22:32 <tyll> I cannot attend if it is earlier, becaue I have to work then 16:23:01 <dgilmore> tyll: okay. 16:23:01 <dgilmore> I am trying to avoidfd having to be up half the night 16:23:26 * pbrobinson is find with the change 16:23:33 <masta> dgilmore: seems like a hard problem with folks all over the globe. 16:23:55 * pbrobinson runs 16:24:19 <dgilmore> tyll: is an hour earlier possible at all? 16:24:42 <tyll> dgilmore: only 30 minutes 16:24:53 <masta> dgilmore: I'd be fine with 8am east coast Australia, but that would likely exclude our EMEA friends. 16:25:16 <tyll> maybe we can start 1 hour earlier with secondary arches 16:25:18 <dgilmore> masta: yeah that would be midnight or 1am in europe 16:25:39 <tyll> I usually have not much to discuss about them 16:25:39 <dgilmore> Lets try that 16:26:34 <dgilmore> I will switch the agenda order around 16:26:41 <dgilmore> start an hour earlier 16:28:10 <dgilmore> okay 16:28:32 <dgilmore> we are getting to the hour 16:28:44 <tyll> btw. there is daylight changing time soon here 16:28:52 <tyll> at the end of the month 16:28:54 <dgilmore> lets talk about how to implement the fast updates repo on list 16:29:07 <nirik> ok 16:29:15 <dgilmore> tyll: yeah, there is no daylight savings in Queensland where I will be 16:29:15 <tyll> but I think it goes in the wrong direction and makes it worse again 16:29:28 <dgilmore> the other states in Australia that follow it just started it 16:30:59 <dgilmore> so lets talk about meeting time also 16:31:06 <dgilmore> see what can be worked out 16:32:31 <dgilmore> anything else? 16:32:37 <dgilmore> if not ill close up shop 16:32:43 <tyll> starting 2014-10-26 I will be in UTC+1 instead of UTC+2 so I won't be able to attend then 16:33:53 <masta> I might need help fixing my f21-secondary key, wont' accept my passphrase. Just a bit of help, and I think I'll be good to help secondaries where it counts... signed packages before composes 16:34:18 <masta> oef 16:35:40 <dgilmore> e about things like that 16:35:49 <dgilmore> masta: you can ping me anytime about things like that 16:36:00 <dgilmore> #endmeeting