18:00:19 #startmeeting Infrastructure (2017-02-09) 18:00:19 Meeting started Thu Feb 9 18:00:19 2017 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:19 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:19 The meeting name has been set to 'infrastructure_(2017-02-09)' 18:00:19 #meetingname infrastructure 18:00:19 The meeting name has been set to 'infrastructure' 18:00:19 #topic aloha 18:00:20 #chair smooge relrod nirik abadger1999 lmacken dgilmore threebean pingou puiterwijk pbrobinson 18:00:20 Current chairs: abadger1999 dgilmore lmacken nirik pbrobinson pingou puiterwijk relrod smooge threebean 18:00:20 #topic New folks introductions 18:00:24 hey 18:00:25 morning everyone 18:00:28 hi 18:00:32 hi 18:00:54 .himynameis cverna 18:00:55 cverna: cverna 'Slim Shady' 18:01:03 * pingou here 18:01:04 .hello jcline 18:01:05 jcline: jcline 'Jeremy Cline' 18:01:14 * doteast here 18:01:54 Hello 18:02:02 here 18:02:19 any new folks like to give a short oneline intro? 18:03:07 alrighty, then lets dive on in... 18:03:25 #topic announcements and information 18:03:25 #info bodhi-2.4.0 beta deployed to stg, please test! - bowlofeggs 18:03:25 #info first mirrorlist container is in production now on proxy02 - kevin/patrick 18:03:25 #info mass rebuild hasn't yet started, but should be soon - releng folks 18:03:25 #info work on new nagios resumes 18:03:36 any other announcements or info? 18:04:06 I'm going to try to update our dist-git stg pagure instance 18:04:08 once 2.12 is out 18:04:11 .hello sayanchowdhury 18:04:12 sayan: sayanchowdhury 'Sayan Chowdhury' 18:04:16 #info new anitya in prod sometime in the next week 18:04:38 pingou: cool. ;) 18:04:53 :) 18:05:00 #info migration of badges from fedorahostted to staging pagure done 18:05:01 jcline: did the new newhotness go to prod? 18:05:22 sayan: oh, sweet! 18:05:27 nirik, not yet, first there was some logging configuration issues, then some dist-git url config issues which I fixed this morning 18:05:48 sayan: really? wow. Great news. 18:05:51 sayan++ 18:05:52 nirik: Karma for sayanchowdhury changed to 12 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:06:01 If things look happy in staging tomorrow I'll upgrade it in prod 18:06:18 sayan++ 18:06:18 cverna: Karma for sayanchowdhury changed to 13 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:06:30 nirik: pingou: we are deciding on the tags now should be done by next week i hope 18:06:32 sayan++ 18:06:40 :fireworks: 18:06:46 cookies :) 18:06:48 jcline: ok. Sounds good. 18:06:53 coookies :) 18:07:01 It would be nice to see the features in some demo 18:07:02 sayan++ 18:07:05 #info new newhotness update soon if stg looks good. 18:07:34 cverna++ vivek_++ for helping on the migration 18:07:34 sayan: Karma for cverna changed to 5 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:07:37 sayan: Karma for vivekanand1101 changed to 5 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:07:59 clime: features or? the-new-hotness/anitya? pagure? 18:08:00 sounds like a good team effort. ;) 18:08:03 s/or/of/ 18:08:16 * pingou migrated a few of his projects 18:08:25 pingou++ 18:08:29 pagure's features for the badges repo :) 18:08:38 we'll need some sort of script to go through all the projects on fedorahosted and see if they are on pagure 18:08:51 then we can see which moved elsewhere and then which are left 18:09:01 I mean, it seems to be quite an achievement to do that migration 18:09:25 pingou: yeah, it's hard. Lots of projects also moved other places... some of them didn't really update hosted, they just moved and left it there. 18:09:39 so I think it's going to be somewhat manual. 18:10:06 also I am 100% sure we will get people for at least the first months asking us where their project went when we take hosted down. ;) 18:10:20 nirik: you did a wiki page if I remenber to trac the status 18:10:42 oh, that reminds me. Currently the migration tool uses trac xmlrpc... could it be made to use sqlite db directly instead? 18:10:50 yes, I did make a wiki page, but it needs lots of updating. ;) 18:11:12 nirik : btw, don't we need to migrate this: https://fedorahosted.org/fedora-apac/ ? 18:11:16 https://fedoraproject.org/wiki/Infrastructure/Fedorahostedmigrations 18:12:08 .hello linuxmodder 18:12:09 linuxmodder: linuxmodder 'Corey W Sheldon' 18:12:11 this is a big table :) 18:12:16 sayan: well, owners of projects are supposed to migrate. I mean I would be happy to if they asked us too... 18:12:30 but we can't know everything 18:12:48 nirik: yeah there was no communications. I would start one then 18:13:07 sayan: that would be great. Or any other projects you see in the same situation. 18:13:50 #info everyone to check projects still on hosted and urge owners to migrate. 18:14:01 Cool. Anything else on status or announcements? 18:14:01 do we have a way to know who are the owners? 18:14:26 * pingou was kind of looking for a list of his projects to check all were migrated 18:14:27 we can look up TRAC_ADMIN 18:14:28 nirik, working on free-medai should be done by next week 18:14:42 linuxmodder: great! 18:14:57 we are mostly using it as membership refresh too 18:15:08 linuxmodder: I guess it will be setup the same way it is now? ie the form opens a private pagure ticket? 18:15:48 that is the hope yes not totally up to speed on that aspect been more housekeeping and new repos side myself 18:16:06 no worries. Hopefully it will all work. 😃 18:16:29 ha. The next topic was hosted migrations... which I think we already covered... 18:16:35 except... 18:16:35 we are also trying to get free-media more inline with ambassadors too 18:17:02 cverna: how hard would it be to change the pgimporter to import from a trac sqlite db instead of a trac xmlrpc network connection? 18:17:14 nirik, might need a few moments post mtg on some perm questions tho for respins.fic.o 18:17:34 sure 18:17:51 nirik: not sure, I have noidea how the trac db is setup 18:18:43 ok. The reason I ask is that after we sunset fedorahosted... people will come to us and want to migrate, but we will not have a trac instance up for them, just the data... 18:19:02 if they could import from the db that would be nice, instead of having to try and stand up a trac instance to read it. 18:20:05 yeah sounds like a better plan 18:20:15 * smooge looks forward to the "Hey no one ever told us..." emails 18:20:15 but anyhow, just wanted to get that out there. 18:20:19 Well, worst case the pgimport stuff could spin up a new container running trac that loads our database 18:20:25 smooge: yeah, there will be tons I am 100% sure. 18:20:38 people ignore notices and deadlines. 18:20:48 nirik: would it be an idea to start adding a huge banner to all fhosted pages? 18:21:08 puiterwijk, probably a good idea 18:21:16 puiterwijk: well, that would get users attention, not sure it would get owners attention... but sure. 18:21:19 smooge, there was talk of yet a another sunset article on mag befor ethe final die off 18:21:41 nirik: I'd hope that users would then notify or ask owners :) 18:21:44 +1 to both ideas 18:21:52 the banner and the fedora magazine article 18:21:55 'YOU HAVE COME TO THE END OF THE UNIVERSE. PLEASE CONTACT THE OWNER ON WHERE THEY ARE MOVING TO' 18:22:03 I've sent multiple announce emails and also I have emailed directly all people with TRAC_ADMIN on any trac's. 18:22:09 but you can't get everyone. ;) 18:22:19 nirik: I don't think I've seen any emails :D 18:22:36 * nirik mail💣s puiterwijk 18:22:53 I'll ping the mrkting folks again on that article this or next week with more visibilty 18:23:00 linuxmodder: thanks. 18:23:21 puiterwijk: you got adding a global banner on your list? or want me to do it? or ? 18:23:26 nirik: I got it 18:23:35 oh, one other thing we could do... 18:23:35 As long as it doesn't have to be pretty :) 18:23:46 but not sure how much pain it would be... 18:24:00 modify the ssh wrapper to print a banner. 18:24:05 So, I'd also suggest that we can keep hosted up for two weeks longer, but in read-only mode.. 18:24:22 nirik: ah, right. I think that'd be doable. I can look at that 18:24:59 puiterwijk: my plan was at sunset day: make everything read-only. Redirect everything to a big banner. We can open particular things to migrate, but otherwise they just go to the banner. 18:25:08 nirik, so the banner on all instances or just those not migrated 18:25:25 nirik: right. That sounds good 18:25:28 linuxmodder: I'd say all 18:25:33 well, all since we don't know for sure whats migrated. 18:26:16 ah and tehn what have the owners do soem backend disable of the banner once migration is confirmed? 18:26:38 No, I'd say we keep the banner on for all instances 18:26:40 well, I'd say just not bother since everything will go away at the end of the month anyhow. ;) 18:27:03 ok, any other hosted migration stuff? 18:27:29 #topic Moving SOPs to the proposed infra-docs sphinx project (https://docs.pagure.org/infra-docs/) 18:27:38 jcline: any further on this? or was this left from last week? 18:27:53 Left from last week, sorry 18:28:00 no worries. ;) 18:28:16 I've been writing documentation for pagure and I'm about ready to pick a date, though 18:28:17 #topic idea to make a packages git repo with all pkgs as submodules - kevin 18:28:22 jcline: great! 18:28:31 jcline++ thanks for all that doc 18:28:46 jcline++ 18:28:46 cverna: Karma for jcline changed to 11 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:28:47 so there was a suggestion from a user eariler this week I wanted to bring up and see what everyone thought of... 18:29:07 form the title, this feels: odd 18:29:08 right now all our packages repos are seperate. So if you want to keep them all up to date, you have to git pull in all of them, etc. 18:29:09 * smooge listens 18:29:20 (or download the daily checkout) 18:29:29 they suggested we make a big over repo that has all the git pkgs repos as submodules. 18:29:42 pingou: it's actually weekly I think 18:30:21 so with this overrepo you could just 'git pull' and get them all. However, my understanding of submodules is that we would have to update the main repo for every change in any submodules? 18:30:32 or can you just say HEAD on all? 18:30:49 git-seed is daily 18:31:24 ok. I thought it was weekly... 18:31:40 nirik: submodules are always a particular commit. They don't follow HEAD automatically 18:31:43 yeah, you are right, it's daily 18:32:05 puiterwijk: right, so we would need to update that on every commit to any submodule 18:32:20 Also, I'm not entirely sure that there's any use in this, since even if you follow the git master, you still only have the git content and not the actual package bits from lookaside 18:32:25 I'm afraid about the load on the server if someone does a git pull 18:32:46 yeah, there's a lot of repos. 18:32:47 for every new package, a submodule would have to be added (or removed on retirement)... 18:32:52 It would allow the user to easily pull the latest for each submodule, though. That doesn't seem terribly valuable though, if there are daily checkouts 18:33:01 Since I think their use was "I can work on packages offline by just downloading everything before I go offline" 18:33:09 pingou: would this help pagure in front of packages any? probibly not 18:33:16 nirik: nope 18:33:24 nirik: Pagure wouldn't make a difference here 18:33:43 ouch, someone wants some pain (d/l'ing all package repos?!?) 18:33:51 puiterwijk: want to download 17K+ git repo hjust to work on the airplane? 18:33:53 Oo 18:33:59 pingou: +1 18:34:06 pingou: that's what they said... 18:34:08 is this a common need? 18:34:21 kloczek: http://paste.fedoraproject.org/551887/48666524/ is the discussion before you joined. 18:34:24 stickster: I'd say it's a very rare need, and I don't think that it's of any use 18:34:32 pingou: 23k more like. 18:34:44 :0 18:34:44 Since, as said, you only get the spec files, without the actual content bits etc 18:34:47 for pkg in pkgdb-cli list --user= --nameonly; do fedpkg clone $pkg; done 18:34:52 it doesn't sound like something we'd want to rearrange all of dist-git to support, absent some real gain 18:35:22 pingou: right on, I was thinking pkgdb would get the needful here 18:35:35 pingou: well, thats going to be pretty super slow. 18:35:48 nirik: git submodules would also be pretty super slow 18:35:53 but the checkout seed is the win there 18:36:13 kloczek, feel free to chime in this is your idea after all 18:36:16 yeah, I'm not sure the submodules would increase the speed 18:36:30 potentially just reduce the number of commands ran 18:36:41 dunno. It should actually be possible to create this outside infrastructure... 18:36:45 nirik: yes 18:36:53 any idea on the actual increase of load tho pingou on the server 18:36:56 create the repo and modules and have a fedmsg script to update 18:37:45 linuxmodder: well, downloading the tarball is one large file, doing git pull on 23K git repo, is quite some more files 18:38:05 linuxmodder: so no idea how much more load this would cause, but my gut feeling is saying: more :) 18:38:14 I'm going to say lots 18:38:23 lots more :-D 18:38:29 fair enough 18:38:34 Especially if clients use the "smart protocol", since the server will do live compression and packing 18:39:04 yeah. It's an interesting idea, but I think the checkout seeds meet this need pretty well already for us. 18:39:14 dumb HTTP protocol would just mean tons upon tons of HTTP requests, but the smart protocols would also incur lots of CPU time on the server for the pulls 18:39:30 (and we have the smart HTTP protocol enabled) 18:39:33 "smart" 18:40:04 nirik: well, if you want to get a single repo, it's quite smart. But it does involve lots of server CPU overhead :(. And for 23k repos for a single user... 18:40:20 right. 18:40:22 so it compresses and packs server side tehn runs the sync ? 18:40:34 well, potentially, it's more than one person who would be interested 18:40:48 I mean, we've been asked for git-see 18:40:50 seed* 18:40:50 pingou: right. But it will do a recompress for every person downloading 18:40:56 and those people might also be interested in this 18:41:07 It does the compressing live 18:41:24 anyhow, shall we move on to the next topic? 18:41:25 I could see it if it did some sort of parse of which you had and only did those that way 18:41:30 You can see this when you do a git pull: "Remote: Compressing objects xx%" 18:41:32 puiterwijk: I'm not saying it'd improve the situation, merely saying that there might be more than one person interested 18:41:52 pingou: sure. Which makes it even less wanted in my opinion 18:41:57 fair ^^ 18:42:23 Guys I have some idea of simplification almost all spec files with libraries 18:42:25 Currently all those packages contains something like: 18:42:29 %post -p /sbin/ldconfig 18:42:31 %postun -p /sbin/ldconfig 18:42:35 lines like this can be dropped in ALL packages and can be replaced by add in glibc (which contain /sbin/ldconfig): 18:42:39 %transfiletriggerin -- %{_libdir}/lib*.so* -p /sbin/ldconfig 18:42:41 %transfiletriggerin -- %{_lib}/lib*.so* -p /sbin/ldconfig 18:42:49 kloczek: yep. But this isn't the place for that. ;) 18:42:50 this very simillar to solaris actions idea 18:42:54 kloczek: bring it up on the devel list 18:43:19 (and I think some folks were already doing that, but I could be misremembering, but the devel list should know) 18:43:31 #topic dist-git upstream progress status - clime 18:43:36 clime: take it away 18:43:38 "the devel list should know". Love that! 18:43:51 am I wrong? 18:43:52 I made a progress on dist-git upstream. 18:44:07 nirik: you're not. Just a great phrasing :) 18:44:07 Basically there is already a dist-git package in Fedora but is not used (much). 18:44:11 problem is that after introducing such policy it wold be good if smeone would do one massive change introducing something like this ina lla packages 18:44:37 What I did is that I brought up to date with current state of dist-git in Ansible 18:44:41 if it will be done separatelly by each packages maintainers it will be never introduced consictently 18:45:08 and then also put gitolite and pkgdb integration away from the package 18:45:17 so it is trimmed down a lot 18:45:18 the same can be done with for example info page 18:45:23 pages 18:45:42 which is good becase it can be integrated into COPR and then perehaps onto pkgs staging 18:45:54 kloczek: we are the infrastructure group. We run the servers. We don't dictacte the content... thats up to packagers and fesco (who you can find on the devel list) 18:46:04 I would like to continue on that. 18:46:20 instead adding/removing to info.dir each info document in %post/%postun this file can be automatically regenerated 18:46:25 clime: great. Perhaps coordinate with pingou to make sure it works with the pagure frontend? 18:46:57 nirik: yes, it should. It is very compatible with dist-git on pkgs 18:47:18 great. Would be good to get everyone using an upstream there 18:47:19 hopefully by next week we'll be able to test pagure on stg dist-git 18:47:22 I will try that out and talk to pingou if anything will pop up 18:47:27 and thus provide a way for clime to double-check that :) 18:47:44 sounds good. 18:47:53 clime: could the pkgdb and gitolite info be kept in a sub-package or so? 18:48:18 pingou: well, for the time being I would just deploy it by Ansible 18:48:19 few days ago I told as well on #fedora-admin about organizing one git repo linking all packages repositories 18:48:30 clime: fair 18:48:42 and maybe integrate it slowly over time 18:49:03 kloczek: that was the subject we were discussing when you joined :) 18:49:10 but I think it is good to make the package minimal 18:49:20 so that it can be used by more people. 18:49:31 pingou: yep :) 18:50:18 cool. Anything more on this for now then? 18:50:31 That was all 18:50:34 :) 18:50:43 thanks for the update clime 18:51:14 clime: oh, side note: The copr playbooks seem to have some idempotence issues now... was going to run them by you sometime, or look and fix them myself (but I keep not having time) 18:51:33 example of such linking is zabbix community repos https://github.com/monitoringartist/zabbix-community-repos 18:51:38 nirik: I'll look at it. 18:51:40 #topic Apprentice Open office hours 18:51:54 any apprentices with questions or comments or needing things to work on? 18:52:08 o/ 18:52:17 athos, hello 18:52:39 I am about to propose a project to GSoC and I'd like to check on it's feasibility with you guys :) 18:52:45 ok. 18:53:05 because fedora contains +23k packages probably it would be good to organize this like rpm directory tree with first level with only first character name of the pacage 18:53:37 athos, shoot 18:53:38 basically, I need a lot of static analysis data. So I want to run multiple static analyzers on a bunch of FLOSS projects, continuously. Debian started a similar project, called debile 18:54:06 the end game being ? metrics 18:54:13 debile in Polish means idiots (plural form :)) 18:54:24 same in french (singular form though) 18:54:31 kloczek, please continue on the devel mailing list 18:54:31 the Idea would be: run various static analyzers in our c/c++packages and have packagers check warnings introduced in the latest package versions (bugs?) 18:54:52 athos: this sounds to me like something that could be a taskotron job/check 18:54:58 linuxmodder: my final goal is to train a ranking algorithm to decrease the importance of false positives 18:55:25 then I'm with nirik on the taskatron bit 18:55:29 linuxmodder: ok thx 18:55:36 and to me seems viable for GSoC 18:55:48 athos: which tool do you have in mind? 18:56:01 it could be... but I would talk to QA folks and see if any would be willing to mentor... they may not have time, dunno. 18:56:05 question: would that be a good project for GSoC? even if not, would it be a nice project to include in the infra? 18:56:31 pingou: frama-C, cppcheck, flawfinder (there are many) 18:56:55 athos: it seems to me like an upstream project more than a Fedora one 18:56:56 please consider adding to the idea page for this year: https://fedoraproject.org/wiki/Summer_coding_ideas_for_2017 18:57:13 as in, upstream c/c++ dev or upstream of these tools might be interested in the results 18:57:26 and using Fedora's sources for this is surely something that is doable 18:57:29 * nirik still thinks it would be a nice taskotron check. ;) well at least that part of it to gather data for whatever 18:57:49 nirik: which would tell packagers about issues in the source code? 18:58:01 yeah. 18:58:05 could datagreper be pulled in for parsing even? 18:58:12 unless the packager is upstream, I think most people would just ignore that :s 18:58:21 could be yeah... 18:58:32 linuxmodder: https://infrastructure.fedoraproject.org/infra/db-dumps/ ? 18:58:58 athos, I'd say draft it mroe concretely and add as a possible on the ideas link I just gave 18:58:59 my final idea is to reduce the list of warnings (only warnings introduced in specific versions, ranked by probability of being true positives) 18:59:08 ok :) 18:59:16 I'd talk with QA folks 18:59:28 * nirik agrees with pingou that most of the fixes would be done by upstream... 18:59:30 I will :) 18:59:42 but I kinda feels this would be better placed under the umbrella of one of the upstream projects 18:59:45 and as someone mentioned ping the qa team ( I will be in GSoC this year but not able to do much on QA side) but maybe someone else in the sig can 19:00:20 pingou, maybe be the pilot for it and have it merged upstream 19:00:42 linuxmodder: would still be better to have this hosted by upstream, I don't think that's Fedora's task 19:00:57 ok, we are now over time... 19:01:00 #topic Open Floor 19:01:02 by upstream you mean, me or the static analyzers? 19:01:09 any quick open floor items? 19:01:14 athos, the projects use it 19:01:23 athos: the analyzers... since you are trying to improve them. 19:01:33 athos: yes 19:01:33 I'll ping you offline nirik with mine on account of time 19:02:10 ok, I'm always in #fedora-admin for questions. ;) 19:02:24 ok, will end the meeting in a minute or less unless something more comes up 19:03:02 bexelbie, ping 19:03:03 nb: Ping with data, please: https://fedoraproject.org/wiki/No_naked_pings 19:03:07 pm 19:03:07 thank you nirik 19:03:17 thanks for coming everyone. :) 19:03:19 thanks nirik 19:03:19 #endmeeting