14:01:44 <pingou> #startmeeting PDC and Fedora
14:01:44 <zodbot> Meeting started Mon Jun 11 14:01:44 2018 UTC.
14:01:44 <zodbot> This meeting is logged and archived in a public location.
14:01:44 <zodbot> The chair is pingou. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:44 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:01:44 <zodbot> The meeting name has been set to 'pdc_and_fedora'
14:02:05 <pingou> #chairs clime mboddu lsedlar threebean abompard
14:02:14 <pingou> cverna: said he will be a few minutes late
14:02:18 <abompard> .hello2
14:02:19 <zodbot> abompard: abompard 'Aurelien Bompard' <aurelien@bompard.org>
14:02:30 <threebean> .hello ralph
14:02:31 <zodbot> threebean: ralph 'Ralph Bean' <rbean@redhat.com>
14:02:33 <pingou> threebean: do you know Ken's IRC nick?
14:02:56 <lsedlar> pingou, ktdreyer I think
14:03:12 * threebean nods
14:03:23 * pingou pinged cqi on -releng
14:03:41 <mboddu> .hello mohanboddu
14:03:42 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
14:03:48 <pingou> someone know where he hangs so we can ping him?
14:04:26 <threebean> pingou: I tried him on another channel, but he's currently marked as "away"
14:04:33 <lsedlar> pingou, I just did on -admin
14:04:47 <pingou> thanks to you both
14:06:05 <pingou> So this is the email I've sent to the infra list a while back https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/JHKHWYU5XK7H2P2QZZCCQR4ZRCTY3OSB/
14:06:11 <pingou> when threebean and I started discussing this topic
14:06:29 <pingou> if we want to use it as a base for our discussion today :)
14:06:42 <bowlofeggs> was there an invite sent for this meeting? i had no idea we were having a meeting now
14:07:12 <pingou> bowlofeggs: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/message/42J73LGGBXD7LWE4Y7LTDEOGUSIVBQYH/
14:08:31 <pingou> I propose we take some notes in http://etherpad.osuosl.org/pdc_fedora
14:08:53 <pingou> hi clime
14:08:55 <pingou> hi cqi
14:09:00 <clime> hi pingou
14:09:08 <cqi> hello
14:09:24 <pingou> cqi: I was just pointing to http://etherpad.osuosl.org/pdc_fedora
14:09:40 <pingou> I propose we start by listing again what we are currently storing in PDC
14:10:04 <pingou> then we go over the different solutions we have
14:10:43 <pingou> I've put the 3 I know well
14:10:43 <bowlofeggs> pingou: ah, i was expecting a confirmation/invite and didn't put it on my calendar
14:11:05 <pingou> mboddu: I know PDC stores a bunch of things for releng, do you know them off hand?
14:11:30 <mboddu> pingou: Yes, I will add them to the etherpad in a min
14:11:35 <pingou> thanks
14:11:38 * mboddu trying to find the doc where he listed them
14:13:18 * pingou waits for mboddu to find his doc :)
14:13:32 <zodbot> Brian (bex) Exelbierd - badges:5bc0e31 [*rules/anitya-map-01.yml] Fixing Anitya badge awarding https://infrastructure.fedoraproject.org/cgit/badges.git/commit/?id=5bc0e31
14:13:34 * cverna waves
14:14:02 <pingou> lsedlar: do you know what is the goal for the composes metadata?
14:14:12 <pingou> knowing what has ever been shipped when/where?
14:15:13 * threebean nods
14:15:15 <lsedlar> pingou, internally it's used to track what's in the compose and thus what needs to be shipped; I think in Fedora there is no such use; so it's just a record of what was shipped and when
14:15:31 <threebean> pingou: fedfind (adamwill) uses that data to track which composes live where.
14:15:50 <pingou> lsedlar: is there a reason/use-case for storing them (other than, because we can? :))
14:16:34 <pingou> threebean: that's a very narrow subset of the data we currently store no?
14:16:36 <mboddu> pingou: Everyone added except 1 thing which I just added
14:16:59 <threebean> pingou: right.  I bet fedfind doesn't consult the massive list of rpms in each compose.
14:17:01 <pingou> mboddu: the compose tracking? Doesn't that overlap with the metadata from composes that's just above?
14:17:22 <mboddu> pingou: Yeah, right, sorry
14:17:23 <lsedlar> pingou, realistically I would say no; aside from possibility of figuring out when a particular package was shipped
14:17:28 <pingou> mboddu: ok, just to be sure :)
14:17:29 * mboddu removes
14:18:07 <fm-apps> github.issue.comment -- bexelbie commented on issue #364 on fedora-infra/tahrir https://github.com/fedora-infra/tahrir/issues/364#issuecomment-396259661
14:18:47 <pingou> so that gives us 5 types of data
14:18:49 <pingou> 6 now :)
14:19:20 <pingou> so threebean "dependency" data is no longer needed?
14:19:27 <threebean> nothing is actively using it atm.
14:19:56 <pingou> proposal: drop the dependency data for now (until something needs it)
14:20:03 <threebean> cqi: did you see we are working on http://etherpad.osuosl.org/pdc_fedora ?
14:20:36 <pingou> threebean: he is typing right now :)
14:20:40 <threebean> :)
14:20:41 <cqi> threebean, yes, I'm in that pad
14:21:15 <pingou> cqi: are these two points, two new type of data or do they also fall on the release/life-circle tracking?
14:22:15 <lsedlar> those look like where the dependencies are stored
14:22:35 <cqi> pingou, freshmaker uses that two endpoints to get container-rpms dependency data
14:22:55 <pingou> cqi: uses or ?
14:23:06 <fm-apps> github.issue.labeled -- bowlofeggs added label Crash to issue #2431 on fedora-infra/bodhi https://github.com/fedora-infra/bodhi/issues/2431
14:23:07 <fm-apps> github.issue.labeled -- bowlofeggs added label bugzilla to issue #2431 on fedora-infra/bodhi https://github.com/fedora-infra/bodhi/issues/2431
14:23:08 <fm-apps> github.issue.opened -- bowlofeggs opened issue #2431 on fedora-infra/bodhi: Bodhi sends e-mails when it cannot modify private bugs https://github.com/fedora-infra/bodhi/issues/2431
14:23:18 <threebean> pingou: they fall into "dependency data"
14:24:07 <cverna> in packages we are using the "global-components" to get all the packages to index
14:24:08 <cqi> threebean, +1
14:24:34 <threebean> cverna: oh, good call.
14:25:22 <pingou> threebean: all the modules data is moved to MBS, correct?
14:25:42 <threebean> pingou: yes, or it will be if it's not already.
14:25:54 <pingou> so that one has a new home
14:26:32 <pingou> cverna: the composes metadata still has the question attached to it: what do we do with it? (keep/drop/keep a subset)
14:26:38 <mboddu> pingou: I am considering stream branches include release branches, or do you want me make it as a separate thing?
14:27:14 <lsedlar> technically the critpath is also currently stored together with the branch information
14:27:58 <pingou> mboddu: I wonder if we shouldn't split release and stream branches appart yes
14:28:05 * threebean nods
14:28:27 <threebean> mprahl pointed out that when we first started the stream branches part, we thought that release branches would go away and "everything would be a module" in the future.
14:28:41 <threebean> ...that's not true anymore.
14:29:04 <threebean> and now we have this annoying requirement where we have to set the EOL on *all* the release branches for *all* the packages, in order to set the EOL for the release.
14:29:24 <threebean> so - splitting release branches out into their own thing where they can be shared across components makes a lot of sense.
14:29:57 <fm-apps> github.pull_request.synchronize -- pingou synchronized pull request #2392 on fedora-infra/bodhi https://github.com/fedora-infra/bodhi/pull/2392
14:30:00 <mboddu> pingou: I agree with threebean
14:30:37 <pingou> mboddu: but i wonder if we couldn't integrate this in the release/life-circle tracking in some ways
14:30:56 <pingou> but I'm going into a potentialy rat hole here
14:31:10 <pingou> that's a technical detail that'll be up to the people working on the solution
14:31:39 <mboddu> pingou: We could but some times packages might get retired and will have a different EOL than the release
14:31:48 <pingou> +1/+0/-0/-1 on dropping the dependency data?
14:32:05 <pingou> mboddu: we don't retire on active releases :)
14:32:12 <fm-apps> github.pull_request.opened -- vismay-golwala opened pull request #2432 on fedora-infra/bodhi: Sorted bugs by #RHBZ on update page https://github.com/fedora-infra/bodhi/pull/2432
14:32:14 <fm-apps> github.issue.comment -- Algogator commented on issue #426 on fedora-infra/fedmsg https://github.com/fedora-infra/fedmsg/issues/426#issuecomment-396264572
14:32:37 <mboddu> pingou: Yes, only on branched and rawhide
14:33:29 <pingou> ok so of our list of things that are stored we've split into 3 categories all but one
14:33:38 <pingou> the "dependency data"
14:33:47 <threebean> +0 to dropping dep data.  it simplifies the current pdc problem.  we have a new long-term problem about where freshmaker can learn about container dep data in fedora, but that's not an immediate problem for us.
14:33:54 <pingou> cqi you said freshmaker uses this and threebean said it's not used
14:33:59 <zodbot> Brian (bex) Exelbierd - badges:d309871 [+pngs/events-linuxcon-2018.png +svgs/events-linuxcon-2018.svg] Adding Linuxcon 2018 badge https://infrastructure.fedoraproject.org/cgit/badges.git/commit/?id=d309871
14:34:11 <threebean> pingou: it uses it in theory.  we don't actually have a freshmaker instance in fedora atm.
14:34:23 <pingou> ok
14:34:25 <cqi> pingou, yes
14:34:34 <pingou> threebean: but would it simplify freshmake to have this data?
14:34:39 * threebean nods
14:34:48 <cqi> pingou, those endpoints were just freshmaker aimed to use
14:35:07 <pingou> (as in: let's not retire something if we know we'll need it in a few months)
14:35:18 <pingou> cqi: ok thanks :)
14:35:24 <threebean> to get freshmaker to rebuild containers in fedora, we'll need a source for that data.  koji has it, but not in an easily queryable way that freshmaker would need.
14:35:55 * ktdreyer is late, waves :)
14:36:06 <pingou> threebean: so we need this info, just potentially not in the form it is today?
14:36:06 <cqi> hi
14:36:09 <pingou> hi ktdreyer
14:36:15 <ktdreyer> hi
14:36:18 <threebean> pingou: sure, yes.  :)
14:36:18 <pingou> ktdreyer: we're both here and in http://etherpad.osuosl.org/pdc_fedora
14:36:25 <ktdreyer> wonderful, thank you pingou
14:37:00 <pingou> threebean: which likely means: we'll need to reconsider/rework it anyway so let's not worry about saving the data we have
14:37:38 <threebean> +1
14:37:54 <pingou> that makes me think: +1 to drop but something to keep in mind we'll need to store at one point
14:38:17 <pingou> lsedlar: cverna: abompard: mboddu: thoughts?
14:38:21 <fm-apps> github.issue.comment -- jcline commented on issue #426 on fedora-infra/fedmsg https://github.com/fedora-infra/fedmsg/issues/426#issuecomment-396266727
14:38:50 <lsedlar> if it's not used, then I would say drop it; the data is in koji, so it could be reconstructed in some more user-friendly way; +1 to drop from me
14:38:53 <fm-apps> github.issue.comment -- jcline commented on issue #426 on fedora-infra/fedmsg https://github.com/fedora-infra/fedmsg/issues/426#issuecomment-396266727
14:39:09 * abompard has too little knowledge of all this to have an informed opinion.
14:39:25 <cverna> I am +0 on this one
14:39:26 <cverna> :)
14:39:35 <cverna> abompard: use the +0 :P
14:39:42 <pingou> alright, so let's drop it in its current form
14:39:52 <pingou> but something to keep in mind we'll need to store
14:39:57 <mboddu> pingou:  +1 on dropping it, but better add a note in etherpad
14:40:44 <pingou> ok, so we're left with worrying about:
14:40:49 <pingou> - Stream branches, branch ownership, retirement dates (EOL/SLE)
14:40:51 <pingou> -  release/life-circle tracking
14:40:53 <pingou> - critpath
14:40:55 <pingou> - composes metadata
14:40:57 <pingou> - List of all packages
14:40:59 <pingou> - dependency data in a new way (for freshmaker)
14:41:06 <pingou> where can we store all of these :)
14:41:29 <mboddu> pingou: and what should we call it? :P
14:41:37 <pingou> threebean has been mentioning that PDC was built as a collection of small django apps
14:41:45 <cqi> fedata :)
14:41:54 <pingou> cqi: FDC? :D
14:41:55 <cverna> for the list of all packages, if this is only for packages , we could use the pagure_poc.json static file
14:42:08 <cqi> pingou, cool :)
14:42:14 <pingou> cverna: the new structure will need such a list anyway for the branch info
14:42:30 <abompard> it should be pretty painless to only activate the django apps we need in PdC
14:42:45 <pingou> abompard: activate/extract?
14:42:58 <cverna> pingou: ok :)
14:43:09 <abompard> well, deactivate the rest, however you want to call it
14:43:21 <otaylor> abompard: but does that save anything? You are still effectively running the PDC code
14:43:32 <otaylor> (And pdc-updater as well)
14:43:36 <pingou> abompard: I meant extract as in: don't keep around code we don't use/want to maintain
14:43:38 <mboddu> abompard, threebean : How hard is it and can we still be able to maintain it?
14:43:47 <cqi> pingou, first try could be remove dropped apps (data) from INSTALLED_APPS
14:43:54 <zodbot> Brian (bex) Exelbierd - badges:fbd279c [+pngs/fedora_podcast.png +svgs/fedora_podcast.svg] Adding Fedora Podcast badge https://infrastructure.fedoraproject.org/cgit/badges.git/commit/?id=fbd279c
14:44:43 <threebean> (The trick will be to avoid maintaining any code that we don't really need.)
14:44:56 <abompard> yeah you can remove the apps we don't want to maintain. I don't know how much code is PDC itself but probably not a lot since it's based on Django and DRF
14:45:40 <lsedlar> some parts are pretty verbose, but those are not used and could be dropped
14:45:43 * mboddu prefers something new and better designed for our needs, but that needs more resources
14:45:46 <Kellin> I would say it's probably easier, given how little we actually need, to trim it down and do a very simple data app with a basic REST interface, we can re-use the RESTful source in PDC but I don't know that we need to keep the whole thing
14:46:10 <pingou> abompard: your probably our most knownledgeable person in the team with django and PDC at this time
14:46:23 <abompard> really? That does not bode well :-D
14:46:30 <pingou> would you rather start from scratch or clean the existing? :)
14:46:44 * lsedlar will admit he wrote a significant chunk of PDC
14:46:51 <pingou> (by team, I mean infra team)
14:47:02 <pingou> oh look someone else who knows a lot about PDC :D
14:47:08 * pingou stares at lsedlar
14:47:20 <abompard> lsedlar: how much would you say is PDC the-framework, without any apps?
14:47:20 * threebean pats lsedlar on the shoulder
14:47:22 <ktdreyer> lsedlar++
14:47:22 <zodbot> ktdreyer: Karma for lsedlar changed to 3 (for the f28 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
14:47:26 <pingou> lsedlar: same question for you then :)
14:47:30 <cqi> lsedlar++
14:47:30 <zodbot> cqi: Karma for lsedlar changed to 4 (for the f28 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
14:48:10 <abompard> I have looked at REST libs for Flask but found nothing that compares to DRF
14:48:29 <lsedlar> it's mostly framework, there are custom extensions to support bulk operations (which django does not by default), plus the HTML renderer is quite big (but not needed if the app is smaller)
14:49:11 <lsedlar> plus there is some custom code to handle storing information about each change done through the API
14:49:41 <abompard> you don't use DRF's HTML output?
14:49:58 <lsedlar> abompard, it's customized
14:49:59 <abompard> Oh right, I remember something extracting the docstrings for the HTML
14:50:12 <bowlofeggs> since bodhi is the only thing that cares about critpath, we could just have bodhi store that info
14:50:38 <bowlofeggs> it has a package model and could just have a critpath bool on there and a simple API for releng to edit it
14:51:10 <lsedlar> as for rewrite vs. reduction of existing code: I think it really depends on how much the API should change: if the desired end-goal is similar to what's there now, it might be easier to just trim off the useless chunks; if it's significantly different, then starting with a new thing would be probably easier
14:52:01 <bowlofeggs> if trimming it and maintaining what's left is a possibility that sounds like it could be advantageous
14:52:49 <abompard> We could also start with a standard Django + DRF and import only the apps we need.
14:53:07 <pingou> bowlofeggs: +1 to bring this to bodhi
14:54:15 <lsedlar> abompard, true, that might be the best of both worlds
14:54:36 <Kellin> abompard: I feel that like the best choice, and not even really import apps, because from when I was working in there it seemed like there is a lot of unused or strangely named fields
14:55:37 <Kellin> abompard: from a releng perspective, one of the things I've been trying to do is settle/define nomenclature to reduce the number of communications issues, we'd probably glean a huge benefit when converting in doing the same with a small group - ensuring all our field names are defined, documented, etc.  it would make it easier to use as well.
14:56:56 <pingou> from seeing the discussion it feels to me that we could set on:
14:57:13 <pingou> - Integrate the critpath flags into bodhi itself
14:57:38 <pingou> - Start a new Django + DRF and import only the apps needed
14:57:45 <bowlofeggs> mboddu: is it correct that bodhi is the only thing that cares about critpath packages?
14:57:58 <bowlofeggs> i guess if something else cares, i could also provide a read API for that...
14:58:18 <mboddu> bowlofeggs: AFAIK, bodhi is the only place
14:58:20 <fm-apps> pagure.issue.assigned.added -- gnaponie assigned ticket greenwave#220 to gnaponie https://pagure.io/greenwave/issue/220
14:58:26 <bowlofeggs> of course, we'd also need releng's critpath script to be modified to talk to bodhi instead of pdc if we did this
14:58:52 <bowlofeggs> mboddu: yeah in that case it makes sense to me that bodhi would do this anyway
14:59:02 <bowlofeggs> it's silly to make bodhi depend on another service for data that only it uses
14:59:20 <abompard> If we start anew, there is a case for going with Flask since we use it almost everywhere, but only if someone has a really good REST lib for Flask, which I haven't found.
14:59:25 <bowlofeggs> that can only reduce reliability :/
14:59:52 <bowlofeggs> jcline, smyers: either of you know a good REST lib for flask?
14:59:53 <ktdreyer> abompard: flask-restful is nowhere as nice as DRF, but it is great for moving quickly
15:00:07 <pingou> there is restful and restless in flask
15:00:11 <jcline> bowlofeggs, not one I'd pick over using Django
15:00:15 <bowlofeggs> abompard: pyramid actually has a niceish REST lib, though not as nice as what i've seen for django :)
15:00:26 <bowlofeggs> yeah i think django is the best i've seen for that
15:00:28 <jcline> Therei s restful which is *okay*
15:00:44 <abompard> pretty much, yeah
15:01:11 <pingou> https://github.com/miLibris/flask-rest-jsonapi but not many starts on gh
15:01:14 <jcline> Honestly, Django is a solid framework and unless there's a super good reason not to use it, I would keep it
15:01:21 <bowlofeggs> the pyramid one can autogenerate REST docs too which makes it easy to appease jcline without putting any effort into it
15:01:23 <cverna> let's stick with django then
15:01:33 <cverna> jcline: yes just like you said
15:01:39 <bowlofeggs> i would not advocate for more pyramid tho
15:01:40 <pingou> + drf would likely increase the amount of code we can re-use from PDC
15:01:46 * jcline <3s Django
15:01:57 <bowlofeggs> django++
15:02:04 <smyers> DRF so good. Bringing in Django if flask is otherwise working might be a bit of a sledgehammer, but...
15:02:09 <smyers> DRF is so good.
15:02:18 <abompard> Yeah I'd go with django too. I was just mentionning flask to raise the argument.
15:02:31 <jcline> smyers, in this case Django's already in use so \o/
15:02:42 * smyers bangs the gavel
15:02:46 <bowlofeggs> :)
15:03:04 <bowlofeggs> imo, rewriting things just to be in a different framework isn't a good use of time
15:03:17 <pingou> abompard: lsedlar: would you have the cycles to lead this effort?
15:03:25 <bowlofeggs> thinking about frameworks when starting a new project is wise, but i don't think you get what you pay for if you spend the time to rewrite an app
15:03:27 <pingou> we're at the top of the hour, so I'll have to drop
15:03:44 <abompard> leading, probably not
15:03:47 <bowlofeggs> haha
15:03:59 <abompard> I can certainly help with django
15:04:01 <bowlofeggs> so i guess i'll file some RFEs to get the package API set up for bodhi
15:04:07 <bowlofeggs> it has package models, just needs an API to edit them
15:04:11 <pingou> threebean: I seem to remember you or people in your team may be able to dedicate some time to help on this?
15:04:26 <bowlofeggs> and that API can then be used to set the critpath bool, nice and easy
15:04:30 <lsedlar> pingou, I would be willing to help with that, but not sure how much time I can put in it; PDC is no longer in my job description
15:04:35 <bowlofeggs> maybe vgolwala would like to take care of it :)
15:04:36 <threebean> heh, not on that one - (not immediately).  we're busy with the mbs part, for one.
15:05:03 <cverna> bowlofeggs: I was bout to say the same thing :)
15:05:05 <pingou> Ok, so we have a solution but no-one to lead it :)
15:05:23 <mboddu> bowlofeggs: +1 on moving critpath stuff into bodhi, the only thing that I know of critpath is we store them in our pungi logs(dont know who uses it though)
15:05:25 <threebean> pingou: later, tomorrow, can you write a description of the meeting and identify that as a gap?
15:05:34 <pingou> let's stop the discussion here and I'll be chasing down people to look for a project lead :)
15:05:41 <threebean> pingou++
15:05:41 <zodbot> threebean: Karma for pingou changed to 5 (for the f28 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:05:41 <abompard> I can help setup the base Django + DRF code but I don't really know what we need to put into it
15:05:46 <pingou> threebean: yup will do
15:05:54 <Kellin> I am also willing to help
15:05:55 <pingou> #endmeeting