18:31:00 <cverna> #startmeeting bugzilla sync
18:31:00 <zodbot> Meeting started Mon May  6 18:31:00 2019 UTC.
18:31:00 <zodbot> This meeting is logged and archived in a public location.
18:31:00 <zodbot> The chair is cverna. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:31:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:31:00 <zodbot> The meeting name has been set to 'bugzilla_sync'
18:31:10 <cverna> #chair pingou nirik
18:31:11 <zodbot> Current chairs: cverna nirik pingou
18:31:18 <nirik> Morning all. Reminder link: https://pagure.io/fedora-infrastructure/issue/7690
18:31:37 <pingou> So the principal of the script is to take poc and cc from dist-git and sync them to bugzilla
18:31:45 <pingou> it's a one way sync dist-git -> bugzilla
18:31:46 <nirik> so, how shall we approach this? do we want to re-write completely? or just try and fix things on the existing script or?
18:32:03 * nirik nods.
18:32:19 <pingou> if there is not a bugzilla account corresponding to the email set in FAS for this user, it sends a reminder that this should be fixed
18:32:30 * mboddu is here
18:32:33 <pingou> there is a list of overrides, but it's kinda painful to deal with
18:32:42 <cverna> #chair mboddu
18:32:42 <zodbot> Current chairs: cverna mboddu nirik pingou
18:32:54 <pingou> for groups, the script queries the FAS field corresponding to mailing-list
18:33:00 <nirik> also the script doesn't work a lot of the time because it hits a timeout or error collecting info
18:33:24 <pingou> we also allow overrides in the releng repo
18:33:27 <pingou> for epel vs Fedora
18:33:43 <nirik> for which it does a http get for each file/package in the repo...
18:33:47 * bcotton lurks
18:33:54 <pingou> so two sources of info: dist-git + the override repo
18:34:00 <pingou> and FAS
18:34:04 <pingou> so that's 3 sources
18:34:09 <pingou> and one target: bugzilla
18:34:32 <pingou> we may be able to drop the override repo with pagure-dist-git
18:34:45 <nirik> that would be a nice savings.
18:35:04 <nirik> note a new corner case came in in the last few days:
18:35:08 <nirik> https://pagure.io/fedora-infrastructure/issue/7755
18:35:10 <pingou> we could then just tweak the script to take the adjust db model into account
18:35:28 <nirik> (basically python maintaienrs want to set poc to a redhat list... that has no fas account or group associated with it)
18:36:10 <pingou> nirik: the override repo would currently allow this
18:36:18 <pingou> as long as there is a bugzilla account associated
18:36:18 <nirik> nope... it takes fas names. ;(
18:36:22 <pingou> arf
18:36:36 <nirik> as far as I know...
18:36:38 <pingou> so we'd need it to accept fas username or email
18:36:49 <cverna> it would be easy for the python maintainer to create a dummy fas account for that ? no ?
18:36:55 <pingou> this kinda opens a can of worms I'm not too happy with
18:37:11 <pingou> how to prevent abuse? If we allow any email addresses to be entered
18:37:29 <pingou> cverna: I'm not sure FAS allows for - in usernames
18:37:32 <nirik> well, no abuse until they make the bugzilla account.
18:37:42 <pingou> true there
18:38:06 <mboddu> I would like to avoid overrides, if they want it they have to create a BZ account
18:38:08 <nirik> but thinking about that... I wonder if one way to 'fix' the people who don't have accounts, is to just have the script create them... and make them login with saml2
18:38:28 <pingou> but we may want to send some sort of notifications: foo added the email foo@bar.com as a POC/CC for the package: BAR
18:38:49 <pingou> mboddu: overrides are used for Fedora vs EPEL
18:38:51 <pingou> :)
18:39:05 <nirik> since we don't have per branch owners anymore.
18:39:13 <pingou> yup
18:39:27 <nirik> is that likely to change? I know it's a oft requested feature....
18:39:49 <pingou> nirik: I often hear about it, but I'm yet to see a ticket for it open :)
18:39:53 <nirik> well, there's also the other overrides...
18:39:55 <cverna> I think that could be part of this work, it would bring a lot of value
18:40:00 <mboddu> pingou: Right, sorry
18:40:12 <nirik> the 'I don't want to use the email I have in fas for bugs' override.
18:40:15 <pingou> cverna: extends the scope quite a bit :s
18:40:50 <cverna> yes but it does make the UI nicer, and less requests on the infra/releng team
18:40:59 <nirik> https://pagure.io/pagure/issue/3416 ?
18:41:11 <cverna> ie people manages the bugzilla stuff in src.fp.o
18:41:22 <nirik> oh, thats flags nevermind
18:41:35 <pingou> nirik: look at #46 :)
18:41:42 <nirik> ah, ok.
18:41:46 <nirik> wow. thats old.
18:41:49 <pingou> yup
18:41:55 <pingou> and not much weighted in
18:42:01 <pingou> and see who opened it ;-)
18:42:37 <pingou> cverna: adding the overrides (not per branch) in pagure-dist-git will reduce the requests on infra/releng w/o expending the scope that much
18:42:38 <mboddu> I think we should change/create the script keeping that in mind
18:42:42 <nirik> well, I think it would be great, but also agree it makes this a lot larger.
18:43:06 <pingou> we don't sync to bugzilla per branch anyway
18:43:15 <pingou> we sync per product: Fedora vs Fedora EPEL
18:43:21 <cverna> ok
18:43:27 <pingou> so adding branch ACLs to pagure won't actually help with the sync
18:43:28 <nirik> (and modules and containers and )
18:43:41 <pingou> but these are different namespaces ;-)
18:43:56 <nirik> so. I think we have 2 broad things here:
18:43:59 <cverna> and flatpak maybe soon
18:44:12 <nirik> 1) robustness/inefficent/not sure it's working.
18:44:18 <pingou> adding per branch ACLs brings the question: which branch has the priority for EPEL? :) (historically, epel7, then 6, then 5 then...)
18:44:51 <nirik> 2) rfes: reassign bugs, close retired to new bugs, possibly support sub categories
18:46:45 <nirik> I think moving the stuff in the overrides repo to a plugin is a high pri
18:46:54 <cverna> +1
18:46:56 <nirik> as that would get us a much faster/better way to get those and help users a lot
18:47:04 <mboddu> ack
18:47:14 <bcotton> i'm going to stop lurking for a moment to suggest that one part of this could be a "here's my BZ email" field in FAS. that'd be handy for me when I go to create Change tracker bugs, too beause sometimes i have to hunt for it. i wonder if there are additional workflows that would benefit from adding that
18:47:27 <nirik> bcotton: thats planned for the fas replacement. ;)
18:47:49 <bcotton> nirik++
18:47:52 <pingou> nirik: +1, it's planned but not in this scope :)
18:48:11 * bcotton resumes lurking
18:48:16 <nirik> yeah, I think we wait on the fas replacement for that, but agree it would be nice...
18:48:44 <pingou> and would solve the override: I'm using a different email than the one in FAS issue we have
18:49:01 <pingou> so as output:
18:49:12 <nirik> so, right now we stick those in a fas hotfix.
18:49:17 <pingou> - adjust pagure-dist-git to record overrides for Fedora and Fedora EPEL on the rpms namespace
18:49:20 <nirik> could we instead just move them to this script?
18:49:31 <pingou> out of it, into a config file I'd think
18:49:40 <nirik> sure, but out of the stupid hotfix. ;)
18:49:44 <pingou> +1
18:50:12 <nirik> another rfe one is to manage other things we currently don't sync at all in this script...
18:50:23 <pingou> - adjust the cron job to take into account the overrides stored in the dist-git DB when outputing the JSON files used as input to the sync script
18:50:33 <pingou> nirik: such as?
18:51:11 <nirik> 'distribution' the various spins 'Xfce spin' have components... but we mange them manually, which means... 'poorly'
18:51:33 <pingou> oh
18:51:39 <nirik> ie, most of the spins don't have components, but some do.
18:52:32 <nirik> back to your list... we would need to also do the release monitoring stuff as a plugin in order to drop that override repo right?
18:53:03 <pingou> that one is already done :)
18:53:12 <nirik> \o/ :)
18:53:13 <cverna> \o/
18:53:20 <pingou> reviewed by cverna :)
18:54:08 <pingou> - adjust the script to take into account specific components: for examples the different spins (xfce, lxde...)
18:54:33 <nirik> this will need to also be in a config file... but thats a much better place for it than nowhere.
18:54:50 <pingou> - adjust the script to take the overrides "I want a different email than the one in FAS" from a configuration file
18:54:54 <pingou> +1
18:55:08 <nirik> - add error handling.
18:55:22 <nirik> right now if anything fails anywhere it just tracebacks.
18:55:35 <pingou> what is the expected behavior then?
18:55:38 <nirik> it could retry things or skip a entry if it's broken
18:55:39 <pingou> keep going or ?
18:55:45 <nirik> depends on where I guess.
18:55:59 <nirik> if it's fetching data from pagure/fas, it could retry a few times.
18:56:01 <pingou> - adjust the script to be smarter with error
18:56:16 <nirik> if it's processing against bugzilla it could retry and perhaps skip that record
18:56:30 <pingou> (that one is kinda hand-waivy, we'll need to define it better)
18:56:34 <nirik> yeah.
18:56:50 <nirik> just that it was broken for a long while because someone left out a : in the override repo
18:57:00 <pingou> ^^
18:57:03 <nirik> and it often fails fetching from fas
18:57:14 <cverna> requests (library) allow to retry x times a request
18:57:38 <pingou> we'll likely want to look at this yes
18:57:47 <nirik> can we end up with no nag emails at the end here? I guess to do that we would need to create the users?
18:57:50 <cverna> I am using this in fedora-packages because requests to mdapi fails a lot
18:58:16 <nirik> which we couldn't do before, but I wonder if we can now that there is fedora login...
18:58:39 <pingou> we've had a couple of reports of it being broken though
18:59:00 <nirik> yeah, but only in the last few days...
18:59:12 <nirik> I'm sure it will get fixed (and it's working fine for me)
18:59:32 <pingou> (just tested it, worked for me)
18:59:59 <pingou> if we have enough priviledges to create the account on the user's behalf, I'm fine with this
19:00:00 <nirik> I can see if our user can make users.
19:00:08 <pingou> (we may want to document/announce that change though)
19:01:12 <nirik> yeah. I just think it's better than the nag that everyone ignores.
19:01:46 <pingou> should we just send an email once then, when the account is created?
19:01:47 <nirik> I think the reassign on poc change and close retired to new bugs things should be pretty easy to add on
19:02:00 <pingou> then, since the account is created, the email should never be sent again
19:02:08 <nirik> the danger there is that they ignore or don't get it and bugs go to... what? no where?
19:02:14 <nirik> oh, on create...
19:02:27 <nirik> yes, a email about creating it definitely sounds good
19:03:12 <pingou> on poc change and retirement could just be triggered on fedora-messaging
19:03:19 <pingou> and we could port this to loopabull
19:03:20 <nirik> ok, I don't think we have perms to do it... will ask bugzilla folks.
19:03:30 <nirik> sounds good.
19:03:37 <pingou> - adjust the script to allow to sync a single project (or all: default)
19:04:28 <nirik> it could always be message triggered and we run one a day to make sure nothing was missed.
19:04:46 <pingou> +1
19:06:14 <mboddu> I guess these sort of requests come in as tickets, right?
19:06:14 <cverna> if we run it daily not sure the message trigger is needed
19:06:36 <pingou> mboddu: which requests?
19:06:37 <mboddu> I think we should take their consent before processing the ticket, saying that an account will be created
19:07:07 <nirik> mboddu: no, it comes in via them adding to a package... either commit or watch
19:07:55 <mboddu> nirik: Ah okay, so, we cannot actually take their consent at that point
19:07:58 <nirik> so the sync script runs and if they don't have a bugzilla account with the email in their fas account it errors and mails me and pingou and them
19:08:20 <nirik> well, I would argue that by them becoming commit or watch on a package they did consent
19:08:31 <mboddu> Okay
19:08:39 <nirik> although I guess someone could have added them...
19:08:59 <pingou> only via the overrides
19:09:22 <nirik> admins can add other users tho right?
19:09:29 <cverna> how can you be a packager and not have BZ account ?
19:10:05 <nirik> if you use different email in the review, or if a sponsor just adds you
19:10:44 <cverna> so if that could create a second account in some case
19:11:06 <nirik> yeah.
19:11:14 <nirik> BTW, the script is currently failing everytime with:
19:11:21 <nirik> fedora.client.AppError: AppError(FASError, FAS server unable to retrieve group erlang-sig, extras=None)
19:11:39 <cverna> ok not a big fan of the create a second account thing
19:11:54 <nirik> well, another option:
19:12:33 <nirik> if script runs, a poc doesn't have a account, it just orphans the package. if a watch/commiter doesn't it just ignores them.
19:13:40 <cverna> that sounds better to me, considering that packager will have the option to set their BZ email in src.fp.o
19:14:15 <pingou> cverna: nope
19:14:32 <nirik> thats by fas account / group right/
19:14:32 <nirik> ?
19:14:58 <mboddu> nirik: I like the second option, but with an error message of why we are doing that
19:15:02 <pingou> nirik: yes, though we did mention it could also accept an email address so python-maint would work
19:15:13 <nirik> erlang-sig is a user in fas, but a group in src.fedoraproject.org
19:15:50 <nirik> if it could accept a email address, that could also have our overrides? ie, wouldn't need to be in conf file anymore?
19:16:36 <pingou> true there
19:16:55 <nirik> so one less thing for us thats self service. win!
19:17:05 <pingou> I'm a little bit worried about the possibility of abuse
19:17:13 <nirik> well, make it require admin?
19:17:28 <nirik> we trust our maintainers lots of other ways too
19:17:46 <pingou> yeah and we'll need to log somewhere who does what for tracking/auditing
19:17:46 <cverna> what would be the possible abuse ?
19:18:07 <cverna> assign someone else email address ?
19:18:08 <pingou> cverna: trust me: you don't want to become the POC for all the kernel bugs :)
19:18:10 <nirik> and it would need a check for the bz account existing or a big note to make sure it does.
19:18:47 <cverna> yes I think that would be pretty obvious abuse and not very damaging
19:18:59 <cverna> ie it can be dealt  with relatively easily
19:19:31 <nirik> I think it would be ok too... but we can run it by our security officer
19:19:48 <nirik> anyhow, we could do that out of band?
19:19:48 <cverna> +1
19:19:58 <pingou> +1
19:20:11 <pingou> we've 10 minutes left
19:20:27 <pingou> do we consider this is a week worth of work, or more?
19:20:38 <nirik> so, the only other RFE thing I had was subcomponent, but we can say that is out of scope for now
19:20:50 <pingou> what are those?
19:21:25 <nirik> so right now we have 'kernel' we could have 'kernel/graphics' and 'kernel/filesystems' and ... and have a different poc for each
19:21:31 <pingou> oh
19:21:33 <pingou> hm
19:21:56 <nirik> or for any larger components that have different people working in different areas.
19:21:59 <pingou> that should be fairly easy when adding support for the distributions/spins
19:22:13 <pingou> updating this would then require an infra ticket, so less self-service
19:22:25 <pingou> but how often does this change?
19:22:32 <pingou> (how much is it used?)
19:22:51 <nirik> probibly not much... kernel was the case I thought of, and I haven't asked them if they want it yet.
19:23:17 <cverna> so maybe let's put this on the side and wait for someone to ask for it ?
19:23:25 <pingou> (the how much is it used could be a chicken/egg problem)
19:23:38 <pingou> I still think this when adding support for spin we'll get this for free
19:23:45 <nirik> right... but yeah, we get this mostly for free with the non package config ones I guess?
19:23:48 <pingou> it's just a new component, with a new POC
19:24:04 <cverna> then happy days
19:24:05 <nirik> well, I think the bz calls might be slightly different... not sure.
19:24:16 <pingou> whether the component is: LiveCD-XFCE or kernel/graphics shouldn't matter
19:24:33 <pingou> nirik: https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora I'm not seeing a subcomponent field here
19:24:55 <nirik> thatsbecause we haven't enabled any. ;)
19:24:57 <nirik> https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%207
19:25:05 <nirik> type in 'kernel' there
19:25:15 <nirik> then it should show "Sub Component"
19:25:38 <nirik> and there's a ton of em
19:25:38 <pingou> then we should be able to re-use this for the spins/distributions
19:25:51 <pingou> distribution: sub: xfce, lxde, kde...
19:25:57 <pingou> liveCD: sub: cf ^
19:26:22 <nirik> well, I guess. Right now 'distribution' is it's own thing... basically bugs for the distro that don't fit in any component.
19:26:38 <nirik> the "LiveCD - Xfce' or whatever are seperate components.
19:26:52 <nirik> but yeah, I don't care how they are organized much
19:27:03 <nirik> if they can all be under distribution thats probibly fine.
19:27:39 <nirik> distribution is the one that has all the tracker bugs...
19:27:41 <pingou> it's just opens up a new way to hierarchize the tickets, not sure if it's worth going for it, but it's a possibility :)
19:28:00 <nirik> I can see how much the kernel team is interested?
19:28:30 <cverna> ok should we wrap up ?  Do we want to record all these tasks in the ticket or a separate email ?
19:28:30 <pingou> sure
19:28:35 <nirik> +1 for ticket
19:28:53 <pingou> do we want a formal requirement document or is the task small enough to not require one?
19:29:30 <cverna> pingou: how much work do you think it is for 1 person ?
19:29:39 <cverna> more than a week ?
19:29:43 <nirik> eastimating is hard... I think if we can get focus on it might be doable in a week or so... in stages? get the overrides working (2-3days), fix the script for robustness (1-2) fix other rfes (2)?
19:30:03 <nirik> I don't think it needs a formal doc, but that could be just me
19:30:10 <mboddu> If everything is recorded in the ticket, I dont think we will need a formal requirement doc, we can questions questions in the ticket itself if anyone has any
19:30:46 * cverna does not have a strong opinion
19:31:12 <pingou> let's start to gather all the requirements from this meeting in a ticket
19:31:21 <pingou> from this we'll see if we want to make it more formal or not
19:31:27 <cverna> the req doc is nice because it can be use as documentation
19:31:48 <cverna> +1
19:31:49 <nirik> we should use the existing meta ticket for this
19:32:06 <pingou> +1
19:32:24 <nirik> pingou: can you add those notes? or want someone else to?
19:32:41 <pingou> I can, but not today :)
19:33:12 <pingou> if we're fine with waiting for tomorrow, then I can do it
19:33:23 <pingou> I'll just extract them from the logs of this meeting
19:33:26 <nirik> ok. So whats next steps? add notes, run by team for pri?
19:33:32 <nirik> profit
19:33:32 <nirik> ?
19:33:51 <pingou> I'll try to make something up for tomorrow's team meeting
19:34:07 <pingou> I'd propose the week of May 20th as target week to work on this
19:34:16 <cverna> yes it would be good
19:34:22 <pingou> (if the team is ok w/ this as well)
19:35:41 <pingou> ok, if we have nothing else, I think we can close :)
19:35:48 <nirik> sure. In the mean time can we fix the erlang thing?
19:35:48 <cverna> +1
19:35:57 <nirik> I guess I can remove it from that package
19:36:00 <nirik> for now
19:36:06 <pingou> +1 for me
19:36:34 <nirik> https://pagure.io/fedora-infrastructure/issue/7456
19:36:56 <cverna> closing the meeting feel free to continue the conversation here :)
19:36:57 <cverna> #endmeeting