14:01:44 #startmeeting PDC and Fedora 14:01:44 Meeting started Mon Jun 11 14:01:44 2018 UTC. 14:01:44 This meeting is logged and archived in a public location. 14:01:44 The chair is pingou. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:44 Useful Commands: #action #agreed #halp #info #idea #link #topic. 14:01:44 The meeting name has been set to 'pdc_and_fedora' 14:02:05 #chairs clime mboddu lsedlar threebean abompard 14:02:14 cverna: said he will be a few minutes late 14:02:18 .hello2 14:02:19 abompard: abompard 'Aurelien Bompard' 14:02:30 .hello ralph 14:02:31 threebean: ralph 'Ralph Bean' 14:02:33 threebean: do you know Ken's IRC nick? 14:02:56 pingou, ktdreyer I think 14:03:12 * threebean nods 14:03:23 * pingou pinged cqi on -releng 14:03:41 .hello mohanboddu 14:03:42 mboddu: mohanboddu 'Mohan Boddu' 14:03:48 someone know where he hangs so we can ping him? 14:04:26 pingou: I tried him on another channel, but he's currently marked as "away" 14:04:33 pingou, I just did on -admin 14:04:47 thanks to you both 14:06:05 So this is the email I've sent to the infra list a while back https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/JHKHWYU5XK7H2P2QZZCCQR4ZRCTY3OSB/ 14:06:11 when threebean and I started discussing this topic 14:06:29 if we want to use it as a base for our discussion today :) 14:06:42 was there an invite sent for this meeting? i had no idea we were having a meeting now 14:07:12 bowlofeggs: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/42J73LGGBXD7LWE4Y7LTDEOGUSIVBQYH/ 14:08:31 I propose we take some notes in http://etherpad.osuosl.org/pdc_fedora 14:08:53 hi clime 14:08:55 hi cqi 14:09:00 hi pingou 14:09:08 hello 14:09:24 cqi: I was just pointing to http://etherpad.osuosl.org/pdc_fedora 14:09:40 I propose we start by listing again what we are currently storing in PDC 14:10:04 then we go over the different solutions we have 14:10:43 I've put the 3 I know well 14:10:43 pingou: ah, i was expecting a confirmation/invite and didn't put it on my calendar 14:11:05 mboddu: I know PDC stores a bunch of things for releng, do you know them off hand? 14:11:30 pingou: Yes, I will add them to the etherpad in a min 14:11:35 thanks 14:11:38 * mboddu trying to find the doc where he listed them 14:13:18 * pingou waits for mboddu to find his doc :) 14:13:32 Brian (bex) Exelbierd - badges:5bc0e31 [*rules/anitya-map-01.yml] Fixing Anitya badge awarding https://infrastructure.fedoraproject.org/cgit/badges.git/commit/?id=5bc0e31 14:13:34 * cverna waves 14:14:02 lsedlar: do you know what is the goal for the composes metadata? 14:14:12 knowing what has ever been shipped when/where? 14:15:13 * threebean nods 14:15:15 pingou, internally it's used to track what's in the compose and thus what needs to be shipped; I think in Fedora there is no such use; so it's just a record of what was shipped and when 14:15:31 pingou: fedfind (adamwill) uses that data to track which composes live where. 14:15:50 lsedlar: is there a reason/use-case for storing them (other than, because we can? :)) 14:16:34 threebean: that's a very narrow subset of the data we currently store no? 14:16:36 pingou: Everyone added except 1 thing which I just added 14:16:59 pingou: right. I bet fedfind doesn't consult the massive list of rpms in each compose. 14:17:01 mboddu: the compose tracking? Doesn't that overlap with the metadata from composes that's just above? 14:17:22 pingou: Yeah, right, sorry 14:17:23 pingou, realistically I would say no; aside from possibility of figuring out when a particular package was shipped 14:17:28 mboddu: ok, just to be sure :) 14:17:29 * mboddu removes 14:18:07 github.issue.comment -- bexelbie commented on issue #364 on fedora-infra/tahrir https://github.com/fedora-infra/tahrir/issues/364#issuecomment-396259661 14:18:47 so that gives us 5 types of data 14:18:49 6 now :) 14:19:20 so threebean "dependency" data is no longer needed? 14:19:27 nothing is actively using it atm. 14:19:56 proposal: drop the dependency data for now (until something needs it) 14:20:03 cqi: did you see we are working on http://etherpad.osuosl.org/pdc_fedora ? 14:20:36 threebean: he is typing right now :) 14:20:40 :) 14:20:41 threebean, yes, I'm in that pad 14:21:15 cqi: are these two points, two new type of data or do they also fall on the release/life-circle tracking? 14:22:15 those look like where the dependencies are stored 14:22:35 pingou, freshmaker uses that two endpoints to get container-rpms dependency data 14:22:55 cqi: uses or ? 14:23:06 github.issue.labeled -- bowlofeggs added label Crash to issue #2431 on fedora-infra/bodhi https://github.com/fedora-infra/bodhi/issues/2431 14:23:07 github.issue.labeled -- bowlofeggs added label bugzilla to issue #2431 on fedora-infra/bodhi https://github.com/fedora-infra/bodhi/issues/2431 14:23:08 github.issue.opened -- bowlofeggs opened issue #2431 on fedora-infra/bodhi: Bodhi sends e-mails when it cannot modify private bugs https://github.com/fedora-infra/bodhi/issues/2431 14:23:18 pingou: they fall into "dependency data" 14:24:07 in packages we are using the "global-components" to get all the packages to index 14:24:08 threebean, +1 14:24:34 cverna: oh, good call. 14:25:22 threebean: all the modules data is moved to MBS, correct? 14:25:42 pingou: yes, or it will be if it's not already. 14:25:54 so that one has a new home 14:26:32 cverna: the composes metadata still has the question attached to it: what do we do with it? (keep/drop/keep a subset) 14:26:38 pingou: I am considering stream branches include release branches, or do you want me make it as a separate thing? 14:27:14 technically the critpath is also currently stored together with the branch information 14:27:58 mboddu: I wonder if we shouldn't split release and stream branches appart yes 14:28:05 * threebean nods 14:28:27 mprahl pointed out that when we first started the stream branches part, we thought that release branches would go away and "everything would be a module" in the future. 14:28:41 ...that's not true anymore. 14:29:04 and now we have this annoying requirement where we have to set the EOL on *all* the release branches for *all* the packages, in order to set the EOL for the release. 14:29:24 so - splitting release branches out into their own thing where they can be shared across components makes a lot of sense. 14:29:57 github.pull_request.synchronize -- pingou synchronized pull request #2392 on fedora-infra/bodhi https://github.com/fedora-infra/bodhi/pull/2392 14:30:00 pingou: I agree with threebean 14:30:37 mboddu: but i wonder if we couldn't integrate this in the release/life-circle tracking in some ways 14:30:56 but I'm going into a potentialy rat hole here 14:31:10 that's a technical detail that'll be up to the people working on the solution 14:31:39 pingou: We could but some times packages might get retired and will have a different EOL than the release 14:31:48 +1/+0/-0/-1 on dropping the dependency data? 14:32:05 mboddu: we don't retire on active releases :) 14:32:12 github.pull_request.opened -- vismay-golwala opened pull request #2432 on fedora-infra/bodhi: Sorted bugs by #RHBZ on update page https://github.com/fedora-infra/bodhi/pull/2432 14:32:14 github.issue.comment -- Algogator commented on issue #426 on fedora-infra/fedmsg https://github.com/fedora-infra/fedmsg/issues/426#issuecomment-396264572 14:32:37 pingou: Yes, only on branched and rawhide 14:33:29 ok so of our list of things that are stored we've split into 3 categories all but one 14:33:38 the "dependency data" 14:33:47 +0 to dropping dep data. it simplifies the current pdc problem. we have a new long-term problem about where freshmaker can learn about container dep data in fedora, but that's not an immediate problem for us. 14:33:54 cqi you said freshmaker uses this and threebean said it's not used 14:33:59 Brian (bex) Exelbierd - badges:d309871 [+pngs/events-linuxcon-2018.png +svgs/events-linuxcon-2018.svg] Adding Linuxcon 2018 badge https://infrastructure.fedoraproject.org/cgit/badges.git/commit/?id=d309871 14:34:11 pingou: it uses it in theory. we don't actually have a freshmaker instance in fedora atm. 14:34:23 ok 14:34:25 pingou, yes 14:34:34 threebean: but would it simplify freshmake to have this data? 14:34:39 * threebean nods 14:34:48 pingou, those endpoints were just freshmaker aimed to use 14:35:07 (as in: let's not retire something if we know we'll need it in a few months) 14:35:18 cqi: ok thanks :) 14:35:24 to get freshmaker to rebuild containers in fedora, we'll need a source for that data. koji has it, but not in an easily queryable way that freshmaker would need. 14:35:55 * ktdreyer is late, waves :) 14:36:06 threebean: so we need this info, just potentially not in the form it is today? 14:36:06 hi 14:36:09 hi ktdreyer 14:36:15 hi 14:36:18 pingou: sure, yes. :) 14:36:18 ktdreyer: we're both here and in http://etherpad.osuosl.org/pdc_fedora 14:36:25 wonderful, thank you pingou 14:37:00 threebean: which likely means: we'll need to reconsider/rework it anyway so let's not worry about saving the data we have 14:37:38 +1 14:37:54 that makes me think: +1 to drop but something to keep in mind we'll need to store at one point 14:38:17 lsedlar: cverna: abompard: mboddu: thoughts? 14:38:21 github.issue.comment -- jcline commented on issue #426 on fedora-infra/fedmsg https://github.com/fedora-infra/fedmsg/issues/426#issuecomment-396266727 14:38:50 if it's not used, then I would say drop it; the data is in koji, so it could be reconstructed in some more user-friendly way; +1 to drop from me 14:38:53 github.issue.comment -- jcline commented on issue #426 on fedora-infra/fedmsg https://github.com/fedora-infra/fedmsg/issues/426#issuecomment-396266727 14:39:09 * abompard has too little knowledge of all this to have an informed opinion. 14:39:25 I am +0 on this one 14:39:26 :) 14:39:35 abompard: use the +0 :P 14:39:42 alright, so let's drop it in its current form 14:39:52 but something to keep in mind we'll need to store 14:39:57 pingou: +1 on dropping it, but better add a note in etherpad 14:40:44 ok, so we're left with worrying about: 14:40:49 - Stream branches, branch ownership, retirement dates (EOL/SLE) 14:40:51 - release/life-circle tracking 14:40:53 - critpath 14:40:55 - composes metadata 14:40:57 - List of all packages 14:40:59 - dependency data in a new way (for freshmaker) 14:41:06 where can we store all of these :) 14:41:29 pingou: and what should we call it? :P 14:41:37 threebean has been mentioning that PDC was built as a collection of small django apps 14:41:45 fedata :) 14:41:54 cqi: FDC? :D 14:41:55 for the list of all packages, if this is only for packages , we could use the pagure_poc.json static file 14:42:08 pingou, cool :) 14:42:14 cverna: the new structure will need such a list anyway for the branch info 14:42:30 it should be pretty painless to only activate the django apps we need in PdC 14:42:45 abompard: activate/extract? 14:42:58 pingou: ok :) 14:43:09 well, deactivate the rest, however you want to call it 14:43:21 abompard: but does that save anything? You are still effectively running the PDC code 14:43:32 (And pdc-updater as well) 14:43:36 abompard: I meant extract as in: don't keep around code we don't use/want to maintain 14:43:38 abompard, threebean : How hard is it and can we still be able to maintain it? 14:43:47 pingou, first try could be remove dropped apps (data) from INSTALLED_APPS 14:43:54 Brian (bex) Exelbierd - badges:fbd279c [+pngs/fedora_podcast.png +svgs/fedora_podcast.svg] Adding Fedora Podcast badge https://infrastructure.fedoraproject.org/cgit/badges.git/commit/?id=fbd279c 14:44:43 (The trick will be to avoid maintaining any code that we don't really need.) 14:44:56 yeah you can remove the apps we don't want to maintain. I don't know how much code is PDC itself but probably not a lot since it's based on Django and DRF 14:45:40 some parts are pretty verbose, but those are not used and could be dropped 14:45:43 * mboddu prefers something new and better designed for our needs, but that needs more resources 14:45:46 I would say it's probably easier, given how little we actually need, to trim it down and do a very simple data app with a basic REST interface, we can re-use the RESTful source in PDC but I don't know that we need to keep the whole thing 14:46:10 abompard: your probably our most knownledgeable person in the team with django and PDC at this time 14:46:23 really? That does not bode well :-D 14:46:30 would you rather start from scratch or clean the existing? :) 14:46:44 * lsedlar will admit he wrote a significant chunk of PDC 14:46:51 (by team, I mean infra team) 14:47:02 oh look someone else who knows a lot about PDC :D 14:47:08 * pingou stares at lsedlar 14:47:20 lsedlar: how much would you say is PDC the-framework, without any apps? 14:47:20 * threebean pats lsedlar on the shoulder 14:47:22 lsedlar++ 14:47:22 ktdreyer: Karma for lsedlar changed to 3 (for the f28 release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:47:26 lsedlar: same question for you then :) 14:47:30 lsedlar++ 14:47:30 cqi: Karma for lsedlar changed to 4 (for the f28 release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:48:10 I have looked at REST libs for Flask but found nothing that compares to DRF 14:48:29 it's mostly framework, there are custom extensions to support bulk operations (which django does not by default), plus the HTML renderer is quite big (but not needed if the app is smaller) 14:49:11 plus there is some custom code to handle storing information about each change done through the API 14:49:41 you don't use DRF's HTML output? 14:49:58 abompard, it's customized 14:49:59 Oh right, I remember something extracting the docstrings for the HTML 14:50:12 since bodhi is the only thing that cares about critpath, we could just have bodhi store that info 14:50:38 it has a package model and could just have a critpath bool on there and a simple API for releng to edit it 14:51:10 as for rewrite vs. reduction of existing code: I think it really depends on how much the API should change: if the desired end-goal is similar to what's there now, it might be easier to just trim off the useless chunks; if it's significantly different, then starting with a new thing would be probably easier 14:52:01 if trimming it and maintaining what's left is a possibility that sounds like it could be advantageous 14:52:49 We could also start with a standard Django + DRF and import only the apps we need. 14:53:07 bowlofeggs: +1 to bring this to bodhi 14:54:15 abompard, true, that might be the best of both worlds 14:54:36 abompard: I feel that like the best choice, and not even really import apps, because from when I was working in there it seemed like there is a lot of unused or strangely named fields 14:55:37 abompard: from a releng perspective, one of the things I've been trying to do is settle/define nomenclature to reduce the number of communications issues, we'd probably glean a huge benefit when converting in doing the same with a small group - ensuring all our field names are defined, documented, etc. it would make it easier to use as well. 14:56:56 from seeing the discussion it feels to me that we could set on: 14:57:13 - Integrate the critpath flags into bodhi itself 14:57:38 - Start a new Django + DRF and import only the apps needed 14:57:45 mboddu: is it correct that bodhi is the only thing that cares about critpath packages? 14:57:58 i guess if something else cares, i could also provide a read API for that... 14:58:18 bowlofeggs: AFAIK, bodhi is the only place 14:58:20 pagure.issue.assigned.added -- gnaponie assigned ticket greenwave#220 to gnaponie https://pagure.io/greenwave/issue/220 14:58:26 of course, we'd also need releng's critpath script to be modified to talk to bodhi instead of pdc if we did this 14:58:52 mboddu: yeah in that case it makes sense to me that bodhi would do this anyway 14:59:02 it's silly to make bodhi depend on another service for data that only it uses 14:59:20 If we start anew, there is a case for going with Flask since we use it almost everywhere, but only if someone has a really good REST lib for Flask, which I haven't found. 14:59:25 that can only reduce reliability :/ 14:59:52 jcline, smyers: either of you know a good REST lib for flask? 14:59:53 abompard: flask-restful is nowhere as nice as DRF, but it is great for moving quickly 15:00:07 there is restful and restless in flask 15:00:11 bowlofeggs, not one I'd pick over using Django 15:00:15 abompard: pyramid actually has a niceish REST lib, though not as nice as what i've seen for django :) 15:00:26 yeah i think django is the best i've seen for that 15:00:28 Therei s restful which is *okay* 15:00:44 pretty much, yeah 15:01:11 https://github.com/miLibris/flask-rest-jsonapi but not many starts on gh 15:01:14 Honestly, Django is a solid framework and unless there's a super good reason not to use it, I would keep it 15:01:21 the pyramid one can autogenerate REST docs too which makes it easy to appease jcline without putting any effort into it 15:01:23 let's stick with django then 15:01:33 jcline: yes just like you said 15:01:39 i would not advocate for more pyramid tho 15:01:40 + drf would likely increase the amount of code we can re-use from PDC 15:01:46 * jcline <3s Django 15:01:57 django++ 15:02:04 DRF so good. Bringing in Django if flask is otherwise working might be a bit of a sledgehammer, but... 15:02:09 DRF is so good. 15:02:18 Yeah I'd go with django too. I was just mentionning flask to raise the argument. 15:02:31 smyers, in this case Django's already in use so \o/ 15:02:42 * smyers bangs the gavel 15:02:46 :) 15:03:04 imo, rewriting things just to be in a different framework isn't a good use of time 15:03:17 abompard: lsedlar: would you have the cycles to lead this effort? 15:03:25 thinking about frameworks when starting a new project is wise, but i don't think you get what you pay for if you spend the time to rewrite an app 15:03:27 we're at the top of the hour, so I'll have to drop 15:03:44 leading, probably not 15:03:47 haha 15:03:59 I can certainly help with django 15:04:01 so i guess i'll file some RFEs to get the package API set up for bodhi 15:04:07 it has package models, just needs an API to edit them 15:04:11 threebean: I seem to remember you or people in your team may be able to dedicate some time to help on this? 15:04:26 and that API can then be used to set the critpath bool, nice and easy 15:04:30 pingou, I would be willing to help with that, but not sure how much time I can put in it; PDC is no longer in my job description 15:04:35 maybe vgolwala would like to take care of it :) 15:04:36 heh, not on that one - (not immediately). we're busy with the mbs part, for one. 15:05:03 bowlofeggs: I was bout to say the same thing :) 15:05:05 Ok, so we have a solution but no-one to lead it :) 15:05:23 bowlofeggs: +1 on moving critpath stuff into bodhi, the only thing that I know of critpath is we store them in our pungi logs(dont know who uses it though) 15:05:25 pingou: later, tomorrow, can you write a description of the meeting and identify that as a gap? 15:05:34 let's stop the discussion here and I'll be chasing down people to look for a project lead :) 15:05:41 pingou++ 15:05:41 threebean: Karma for pingou changed to 5 (for the f28 release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:05:41 I can help setup the base Django + DRF code but I don't really know what we need to put into it 15:05:46 threebean: yup will do 15:05:54 I am also willing to help 15:05:55 #endmeeting