15:01:53 #startmeeting Infrastructure (2019-03-07) 15:01:53 Meeting started Thu Mar 7 15:01:53 2019 UTC. 15:01:53 This meeting is logged and archived in a public location. 15:01:53 The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:53 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01:53 The meeting name has been set to 'infrastructure_(2019-03-07)' 15:01:53 #meetingname infrastructure 15:01:53 The meeting name has been set to 'infrastructure' 15:01:53 #topic aloha 15:01:53 #chair nirik pingou puiterwijk relrod smooge tflink threebean cverna mizdebsk 15:01:53 Current chairs: cverna mizdebsk nirik pingou puiterwijk relrod smooge tflink threebean 15:02:09 .hello2 15:02:12 mizdebsk: mizdebsk 'Mikolaj Izdebski' 15:02:19 happy snowy morning everyone. 15:02:27 hi o/ 15:03:44 .hello2 15:03:45 pingou: pingou 'Pierre-YvesChibon' 15:04:41 .hello2 15:04:42 abompard: abompard 'Aurelien Bompard' 15:05:43 #topic New folks introductions 15:05:43 #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves 15:05:43 #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted 15:05:57 Hello 15:06:16 I just sent a greeting to the mailing list but wanted to see the meeting as well 15:06:34 hi and welcome frailtyy 15:06:46 frailtyy: welcome :) 15:07:11 hello 15:07:19 #topic announcements and information 15:07:19 #info Beta Freeze Begins 2019-03-05 15:07:19 #info Staging Koji sync planned for 2019-03-08 (ticket 7600) 15:07:19 #info Started working on fedora-messaging migration, status here https://github.com/orgs/fedora-infra/projects/2 15:07:20 #info Reminder: use good commit messages 15:07:36 #info smooge has given up bad commit messages for lent 15:08:04 #info trying to fix pkg02.stg.fedoraproject.org 15:08:19 any other announcements? 15:09:39 #topic Oncall 15:09:39 #info smooge is on call from 2019-02-28 -> 2019-03-07 15:09:40 #info mizdebsk is on call from 2019-03-07 -> 2019-03-14 15:09:40 #info bowlofeggs is on call from 2019-03-14 -> 2019-03-21 15:09:40 #info ?????? is on call from 2019-03-21 -> 2019-03-28 15:09:41 #info Summary of last week: (from smooge ) 15:10:05 So lots of little irqs last week mostly with trying to unbreak things 15:10:11 I can be back on the rotation anytime now... 15:10:17 ack, i'll be oncall starting today 15:10:19 .hello2 (sorry i'm late) 15:10:20 bowlofeggs: bowlofeggs 'Randy Barlow' 15:10:33 nirik, I will put you down for the last week 15:10:39 ok 15:11:37 #topic Monitoring discussion 15:11:38 #info https://nagios.fedoraproject.org/nagios 15:11:38 #info Go over existing out items and fix 15:11:49 this has improved a lot since last week 15:12:25 we have some machines that dropped disks. 15:12:31 yeah 15:12:36 no faf fedmsgs... (no idea why) 15:13:22 nirik: it might very well just be broken once more. Last I looked, it was out of disk space and refusing any ssubmissions 15:13:22 I guess we can re-enable autosign01... it's back up 15:14:22 it has disk space... no idea, can mail contact. 15:15:21 #topic Tickets discussion 15:15:22 #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket 15:15:37 no issues for this week 15:15:46 #topic hotfixes - kevin 15:16:21 I just wanted to mention hotfixes... I know we all sometimes just apply changes directly... is there something we could do to make hotfixes less anoying to deal with? 15:17:04 perhaps a role you pass the filename(s) you are hotfixing to? 15:17:51 The current process is: 15:18:08 we would need to pass it the local file(s) and the destination, so would be quite similar to just copying it at the right location directly 15:18:09 * add task(s) to cp files, add the orig files, add diff of hotfix to files, run playbook 15:18:39 the few I did in the last few weeks I tried to make sure they were also commented clearly in the file 15:19:08 there's also a patch ansible module I think... we could move to that perhaps, but might be less clear 15:19:13 * pingou guilty of not having done the two steps in the middle on his last 15:19:17 for application openShift solves hotfixing :) 15:19:27 i've been striving not to hotfix as much as possible myself 15:19:39 i'd rather make a real build with the fix, and even more a real release 15:19:57 cverna: indeed. 15:20:02 there are times i still do a hotfix when i just have to, but i've found that many times you really can take the time to make a release 15:20:28 and when i do a hotfix i work to make a release for it immdiately afterwards 15:20:34 i don't like the hotfix to be present for very long 15:20:49 it's a recipe for regressions 15:20:51 then of course there's AGPL... but not sure how many AGPL things we have left 15:21:02 fedora-packages? 15:21:07 not sure we have much more 15:21:15 yeah, packages, but not sure what else 15:21:53 anyhow, I guess perhaps the answer is: try not to, if you do use the process, move your apps to openshift. ;) 15:22:17 hyperkitty is AGPL I think 15:22:59 it should be easy to run an ad-hoc ansible command to list all AGPL-licensed rpm packages we have installed 15:23:00 abompard: I can only see GPLv3 15:23:01 nirik: that'd be the current approach anyway :) 15:23:34 * nirik nods. ok, just wanted to discuss it in case anyone had any better ideas than I. ;) 15:24:17 abompard: FYI, after this meeting in #epel, folks are working on the great python36 rebuild in epel7... might help out your upgrade stuff 15:24:46 nirik: ah cool 15:24:47 thanks for bringing it, I did not know that we have this process in place 15:24:47 thanks 15:25:21 ok next topic? 15:25:53 yep. move on 15:26:47 #topic Priorities for next week? 15:26:48 #info please put tickets needing to be focused on here 15:27:04 OK we are in a freeze and we have 120 ish open tickets 15:27:47 a full tank of gas and chicago is 700 miles away... hit it! 15:27:53 it would be cool if we could try to push a bit for this https://pagure.io/fedora-infrastructure/issue/7625 15:28:04 * nirik makes a bad blues brothers joke 15:28:16 it is not critical but that would allow us to move resultsDB to fedora-messaging in stg 15:28:28 nirik, I will send you to see the nun 15:28:30 we can put in a ticket... 15:30:14 thanks 15:30:55 just FYI... I got a master run completed before/around freeze time 15:31:07 there were some failed hosts tho. If I can find time I will look more at those. 15:31:52 so that's what changed back the hook in dist-git :) (I was suspecting as much) 15:31:54 ok any other tickets needing a foucs 15:32:34 #topic Apprentice Open office minutes 15:32:35 #info A time where apprentices may ask for help or look at problems. 15:33:01 https://paste.fedoraproject.org/paste/Qf1VIv3CKDGQz9cseL-CTA 15:33:09 pingou: yeah... likely so. 15:33:44 can we get ride of taiga in infracloud ? 15:33:50 or not yet 15:34:04 we should ask the owner 15:34:06 I think not yet, but we may want to advertise that it will go away 15:34:20 on the other side mizdebsk has a good point, it's not ours :) 15:34:21 yeah, I was waiting until we had the new one more in use... 15:34:27 mizdebsk: I am not sure we have an owner anymore 15:34:34 Kellin is/was the owner 15:34:39 Well, we could just make backups of all the projects, send those to the project owners and shut it down, worst case. 15:34:39 I think abompard took over as owner? 15:34:49 yes so we don't have an owner anymore 15:35:27 it was just for testing the auth plugin 15:35:33 ah, ok. 15:35:38 I don't need it anymore 15:35:42 abompard: you touched it last, so you own it now! :P 15:35:48 ^^ 15:35:50 puiterwijk: looks like it :-) 15:35:51 ha 15:35:53 * puiterwijk thinks he should be careful with that suggestion.... 15:36:06 abompard it is yoursssss 15:36:10 puiterwijk owns everything. 15:36:12 There's so much that would then immediately fall to me... oh wait, it does. 15:36:37 Anyway, I think we should start setting up a timeline for shutting taiga.fic.o down. 15:36:39 anyhow, lets wait until the new one is a bit more established, then look at retiring the old one 15:36:45 #topic Open Floor 15:36:46 It's easy enough for users to get backups and import to teams.fp.o 15:36:52 ok since we move it to that 15:37:09 there is also old cloud retirement, could be tied to that too, or related to it. 15:37:23 I think we do taiga first and then old-cloud 15:37:41 old cloud can have a power off date of July 4th 15:37:47 Hi! I'd like to experiment with setting up Matrix server for fedora - where do I start? Apply as an apprentice? 15:38:01 we should not set any date on it yet, unitl we have the new setup working 15:38:04 discussion at https://pagure.io/fedora-infrastructure/issue/7399 15:38:36 vrutkovs: well, I think we are all reluctant to run another service... 15:38:53 but could perhaps provide you resources to work on it yourself 15:40:03 we could make a cloud instance, but I hate to add things to our old cloud 15:40:23 I think this would be a good thing to look at after the new-cloud openshift is in place 15:40:30 nirik: right, I think I could manage it on my own, ideally in an openshift namespace 15:40:40 would there be concerns about running a communications system? 15:40:49 yes yes there are 15:41:22 gdpr my love 15:41:29 yeah, if you can wait until we have the community openshift going we could experiment there... 15:41:30 GDPR and such? There is a bot which requires consent, but a blessing from legal would be great 15:42:15 sure, there's no rush 15:42:36 except "all the kool kids like KDE already have their Matrix servers" :P 15:42:36 vrutkovs: well you also have to support the SAR requests and right to be forgotten 15:42:39 cool. I really hope to work on it, but then there's weeks like this one... where I don't get to anything on my list. ;( 15:42:51 and if fedora hosts that, we can't rely on community members to handle those requests 15:43:20 bowlofeggs: right, its an admin task - there are scripts to kick and erase the user 15:43:43 bowlofeggs, if community member doesn't handle it then we can handle it by removing the openshift project 15:44:04 heh true 15:44:34 comms systems are trickier than the other kinds of things we host 15:44:42 including mailing lists, because those are explicitly public 15:45:29 ok anything else? 15:45:31 well, we would want to make sure we sort that out before we run it as fedoraproject.org... but could run a test setup with fedorainfracloud.org I think. 15:46:36 I would like to run a zulip (https://zulipchat.com/) instance :) 15:46:42 certainly - for a prod we'd need DNS TXT entries and certificates 15:47:00 * cverna start the chat war 15:47:07 I would like people to pay for their own servers in AWS 15:47:15 there I have said it.. 15:47:18 smooge :D 15:47:38 bowlofeggs: you mean scaling? We could encourage people to use other instances and federate if the load becomes unbearable 15:47:56 vrutkovs: no, i mean with respect to law 15:47:57 however a test instance inside RH can handle ~600 chatrooms on a modest bare install 15:48:03 vrutkovs, there are network issues, more legal issues than gpdr 15:48:30 bowlofeggs: ah, true, I don't know much about that, I hope Matrix/KDE folks could help us with these 15:48:31 it is a quagmire 15:48:32 vrutkovs: if the server is in the US, it is also subject to US warrants, for example 15:48:53 including secret warrants 15:49:18 which doesn't really apply to mailing lists since they are completely public 15:49:33 vrutkovs, most of the community orgs usually don't think about dealing with them and say it isn't a problem until they get some sort of major problem 15:49:34 it's because matrix supports "private" chats 15:49:47 bowlofeggs: there's private mailing lists too... ;) 15:49:59 nirik: ah interesting,i didn't realize we hosted those 15:50:01 smooge: we could enforce using plain-stored chats, no e2e (although its possible) 15:50:09 not many, but a few 15:50:13 nirik: but, i would think in that case teh e-mail provider would be served a warrant and not us 15:50:18 for matrix, we would be served 15:50:25 ianal of course ☺ 15:50:38 in the end its all just entries in postgresq, I imagine it would be easy to provide matching entries 15:51:05 * nirik doesn't think we are going to solve this here... :) 15:51:12 yep 15:51:26 I think this is not something we can solve or debate to a solution 15:51:58 agree, lets comment in https://pagure.io/fedora-infrastructure/issue/7399 - I'll try reach out to Matrix folks about legal problems 15:52:02 so let us end this meeting and move on to problems we can deal with.. 15:52:22 thanks all 15:52:35 thanks for chairing smooge 15:52:55 thanks 15:53:13 thanks 15:53:14 thanks! 15:53:15 thanks for chairing smooge :) 15:55:27 #endmeeting