14:31:49 <bstinson> #startmeeting Taskotron Tests to Fedora CI
14:31:49 <zodbot> Meeting started Wed Mar  6 14:31:49 2019 UTC.
14:31:49 <zodbot> This meeting is logged and archived in a public location.
14:31:49 <zodbot> The chair is bstinson. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:31:49 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:31:49 <zodbot> The meeting name has been set to 'taskotron_tests_to_fedora_ci'
14:32:09 <bowlofeggs> .hello2
14:32:10 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
14:32:13 <kparal> .hello2
14:32:14 <zodbot> kparal: kparal 'Kamil Páral' <kparal@redhat.com>
14:32:15 <tflink> .hello2
14:32:18 <zodbot> tflink: tflink 'Tim Flink' <tflink@redhat.com>
14:32:18 <dperpeet> .hello2
14:32:20 <mvadkert> .hello2
14:32:21 <zodbot> dperpeet: dperpeet 'None' <dperpeet@redhat.com>
14:32:24 <zodbot> mvadkert: mvadkert 'None' <mvadkert@redhat.com>
14:32:25 <siddharthvipul> .hello siddharthvipul1
14:32:28 <jskladan> .hello2
14:32:29 <zodbot> siddharthvipul: siddharthvipul1 'Vipul Siddharth' <siddharthvipul1@gmail.com>
14:32:30 <jbieren> .hello2
14:32:32 <zodbot> jskladan: jskladan 'Josef Skladanka' <jskladan@redhat.com>
14:32:36 <zodbot> jbieren: jbieren 'John Bieren' <jbieren@redhat.com>
14:32:39 <pingou> .hello2
14:32:40 <zodbot> pingou: pingou 'Pierre-YvesChibon' <pingou@pingoured.fr>
14:32:48 <bstinson> #chair bowlofeggs bookwar dperpeet kparal jskladan siddharthvipul tflink mvadkert jskladan pingou
14:32:48 <zodbot> Current chairs: bookwar bowlofeggs bstinson dperpeet jskladan kparal mvadkert pingou siddharthvipul tflink
14:32:52 <bstinson> .hello2 bstinson
14:32:54 <zodbot> bstinson: bstinson 'Brian Stinson' <brian@bstinson.com>
14:32:56 <bgoncalv> .hello2
14:33:00 <zodbot> bgoncalv: bgoncalv 'Bruno Goncalves' <bgoncalv@redhat.com>
14:33:06 <bstinson> #chair bgoncalv
14:33:06 <zodbot> Current chairs: bgoncalv bookwar bowlofeggs bstinson dperpeet jskladan kparal mvadkert pingou siddharthvipul tflink
14:33:33 <bstinson> #topic Agenda
14:34:16 <bowlofeggs> i always knew brian was a person with an agenda…
14:34:19 <bstinson> #info Topic: Current work on tests like rpmlint/rpmgrill internally and externally
14:34:34 <bstinson> #info Topic: Open RFEs/blockers
14:35:01 <mvadkert> those 2 we do not run internally
14:35:23 <bstinson> #info Topic: Who wants to be involved going forward
14:35:35 <bstinson> i should have put in a section for background
14:35:41 <bstinson> #topic Background
14:36:32 <bstinson> so, one of the efforts we're looking into is how to best support the tests that are currently being executed by taskotron
14:37:40 <kparal> I have prepared a list of possible requirements for Fedora CI, I can paste them in and we can discuss them, just tell me when
14:37:40 <bstinson> sharing resources with the Fedora CI pipeline makes sense, since that's meant to be one of the major places to look for tests related to packaging in Fedora
14:38:00 <bstinson> kparal: perfect! i'll cue you up in a moment
14:39:39 <bstinson> kparal: we had a discussion about this late last year, and i can't find it in my email. do you happen to remember where we left off?
14:39:45 <bstinson> or is that part of your requirements?
14:40:07 <kparal> I don't remember that conversation :)
14:40:13 <dperpeet> well, we had some usability issues
14:40:16 <dperpeet> with the CI pipeline
14:40:17 <dperpeet> mostly
14:40:41 <dperpeet> but I think bookwar and mvadkert are people working on a lot of this
14:40:58 <dperpeet> it might be good to just start with the requirements
14:41:00 <dperpeet> and go from there
14:41:49 <bstinson> i think we may have been waiting to do some RFEs to the standard test interface or some such
14:42:05 <kparal> I found the email thread in my inbox
14:42:55 <kparal> I'd rather start the discussion fresh
14:42:59 <bstinson> let's start with the current work, so we have an idea of what's happening currently
14:43:02 <bstinson> kparal +1
14:43:06 <kparal> it just involved hacky workaround how to make it work at that point of time
14:43:12 <bstinson> #topic Current work
14:43:22 <kparal> Just a quick background. In Taskotron we currently run: abicheck, rpmlint, rpmgrill, python-versions, and rpmdeplint. The first 4 are similar (executed on each new Koji build), the last one is very special (runs on Koji tags, but checks rpms/builds).
14:43:48 <kparal> I'll start with the most obvious requirement
14:43:51 <kparal> 1. CI needs to support executing a generic test for all new Koji builds (scratch builds, pull requests, etc)
14:43:59 <kparal> This means that for e.g. ssh package not only the ssh test suite from distgit is executed, but also all available generic tests (rpmlint, abicheck, etc).
14:44:19 <mvadkert> ack
14:44:19 <kparal> I don't know whether there's some support already in Fedora CI for this, would be nice to know.
14:44:35 <kparal> the last time I talked to mvadkert, it seemed like there isn't :)
14:44:43 <mvadkert> kparal: we have some formw of that on downstream, some are coming to upstream this year
14:45:06 <bgoncalv> and these tests when they run for different archs, can they be run on x86_64?
14:45:07 <mvadkert> kparal: I would start with metadate definition and we will make a CI system that support that
14:45:10 <mvadkert> https://pagure.io/fedora-ci/metadata
14:45:31 <bstinson> mvadkert: is that work (the bringing things upstream) tracked somewhere?
14:45:53 <kparal> bgoncalv: I have a note here regarding arches, will post it right away
14:45:56 <mvadkert> bstinson: I have first MVP defined
14:46:04 <mvadkert> here
14:46:16 <kparal> mvadkert: what's MVP?
14:46:28 <tflink> minimum viable product
14:46:30 <mvadkert> https://projects.engineering.redhat.com/browse/PLATFORMCI-2415
14:46:47 <tflink> basically, the smallest thing that could possibly work
14:47:00 * kparal translates to PoC in his head
14:47:01 <bstinson> mvadkert: great, that's helpful
14:47:04 <mvadkert> tflink: it is even less that, it is basically open-sourcing 1 small thing
14:47:07 <tflink> mvadkert: does it exist outside of the VPN?
14:47:18 <tflink> the docs, I mean
14:47:22 <mvadkert> bstinson: do we ahve that taiga now?
14:47:27 <mvadkert> tflink: partly
14:47:36 <bstinson> mvadkert: i'm pushing to get us as early adopters
14:47:46 <bstinson> the "2 weeks" after devconf turned into a little bit more
14:47:52 <mvadkert> bstinson: np
14:47:53 <bstinson> the taiga folks have an instance for us
14:48:02 <bstinson> i'm just waiting for the "yes" from management
14:48:19 <kparal> so when talking about fedora-ci/metadata, isn't that something that a particular test has defined in its distgit repo?
14:48:26 <bstinson> for this meeting, i'm gathering stuff like this so we can have it in there for the initial import
14:48:28 <mvadkert> tflink: bstinson: so my current plan is to have an MVP#1 for flock, but I am all in to provide what use case to cover
14:48:31 <tflink> I'm not a huge fan of discussing stuff in Fedora forums that non-RH folk can't read
14:48:43 <mvadkert> tflink: yah right :) they asked for link :)
14:48:45 <mvadkert> tflink: sorry
14:48:54 <bstinson> tflink: fair point
14:48:59 <mvadkert> 09:48 < kparal> so when talking about fedora-ci/metadata, isn't that something that a particular test has defined in its distgit repo?
14:49:05 <mvadkert> it is now also about CI metada
14:49:12 <mvadkert> metadata, not just test metadata
14:49:20 <kparal> so, server-side metadata, yes?
14:49:21 <mvadkert> kparal: and this is one of the use cases for CI mestatadata
14:49:29 <kparal> because generic tests should be defined server-side
14:50:00 <kparal> it doesn't make sense for each distgit package to specify that rpmlint should be executed on it
14:50:06 <mvadkert> kparal: yes
14:50:09 <kparal> ok
14:50:16 <kparal> I'll look at it
14:50:17 <mvadkert> kparal: that should be possible, to define generic tests
14:50:28 <mvadkert> kparal: just I am not sure we have an example there for that now
14:50:33 <mvadkert> psss: ^
14:50:57 <mvadkert> psss: we need an example for defining a generic test
14:51:14 <bstinson> sounds like a good action item to come from this
14:51:20 <psss> So far we have a bunch of examples to demonstrate where we are heading:
14:51:22 <psss> https://pagure.io/fedora-ci/metadata/blob/master/f/examples
14:51:37 <mvadkert> bstinson: woudlbe nice to create a Fedora CI Sig or something with a taiga bord
14:51:45 <bstinson> mvadkert: indeed
14:51:47 <mvadkert> so we do not have stuff on internal systems
14:51:57 <mvadkert> bstinson: that is an idea of bookwar
14:52:05 <mvadkert> bstinson: with the sig
14:52:08 <dperpeet> we've been waiting for that taiga board
14:52:17 <mvadkert> bstinson: do drive this and also CI dashbaord and such things forward
14:52:23 <mvadkert> dperpeet: yeah, ok :)
14:52:30 <bstinson> to bring this back around to features and blockers
14:52:33 <bstinson> i heard
14:52:36 <kparal> so, additional note regarding arches: It's good to think about different architectures when scheduling a generic test. Some tests might be able to test all available test subject architectures in a single execution (like rpmlint), some might require running on a host of the same architecture as the test subject architecture and therefore require several executions (per-arch). In Taskotron, for simplicity-sake, we always run generic tests
14:52:36 <kparal> per-arch, because it increases test reliability, but the test can override it and schedule just a single "noarch" execution (that makes sense for a test that only works with src rpms, for example).
14:52:54 <bstinson> ah kparal read my mind
14:53:07 <mvadkert> psss: ^ use case
14:53:14 <mvadkert> for a generic test
14:53:32 <mvadkert> kparal: would you mind to open an issue for that use case to the fedora-ci/metadata?
14:53:40 <kparal> ok
14:53:41 <bstinson> jbieren: ^ this is something for us as well, since we'll want to orchestrate the pipeline across architectures
14:53:50 <jbieren> kparal, does taskotron report the results of its tests via fedmsg?
14:54:00 <kparal> jbieren: yes, through resultsdb
14:54:04 <jbieren> bstinson, ack
14:54:11 <jbieren> okay good
14:54:20 <kparal> Last but not least, it might be nice to support a blacklist/whitelist of packages for a certain test. For example, we maintain a blacklist of packages for which we simply can't run abicheck, because it requires far too much memory or time to execute (think libreoffice or kernel).
14:54:35 <mvadkert> bstinson: I would start with metadata which ci systems can then implement
14:54:40 <bookwar> kparal: i'd like to ask a generic question: do you want to keep running tests on Taskotron infra, or you want to had them over to the Jenkins-based CI pipelines infra?
14:55:01 <jbieren> are the blacklist/whitelists static?
14:55:04 <mvadkert> bstinson: like standardize the way hwow you tell the CI system what it should do
14:55:09 <tflink> as far as I know, the plan is to move things over to the CI pipelines infra
14:55:15 <dperpeet> bookwar, last time we spoke it was about moving them over for easier maintenance
14:55:17 <kparal> bookwar: I'd rather move them over. It doesn't make much sense for us to maintain the system when all new tests would go to Fedora CI
14:55:19 <pingou> has taskotron on its roadmap to be ported to fedora-messaging?
14:55:23 <mvadkert> that blacklist/whitelist should be another use case for metadata :)
14:55:25 <mvadkert> psss: ^ :)
14:55:26 <tflink> pingou: yep
14:55:33 <psss> One of the ideas around ci/metadata was to have a generic configuration and then ability for a specific package to override/adjust.
14:55:34 <pingou> cool
14:55:40 <kparal> jbieren: static, yes
14:55:40 <bookwar> kparal: thanks, thats what get me confused
14:55:47 <mvadkert> pingou: tflink: is there an EOL date to fedmsg?
14:55:49 <tflink> pingou: there are folks working on that already :)
14:56:11 <kparal> fedora-messaging patch was just merged to resultsdb
14:56:16 <pingou> mvadkert: for systems involved in process which need guaranted delivery, the earlier the better
14:56:26 <pingou> we're making it a requirement for rawhide gating
14:56:34 <kparal> for triggering, we're said there's a compat layer for fedmsg, so we should be able to still intercept all the events
14:56:39 <pingou> for other systems, the transition will be slower
14:56:50 <mvadkert> pingou: do you knwo it it is just as simple as moving to an AMQP client? or are there some additional changes?
14:56:58 <mvadkert> (like message foramt or something)
14:57:01 <bstinson> mvadkert: who is involved in the metadata changes that are needed for something like this?
14:57:13 <bstinson> anyone not in this room that we should keep in mind?
14:57:33 <bstinson> also apologies folks, i seem to be experiencing some lag
14:57:42 <pingou> mvadkert: currently we're just trying to port to AMQP w/o changing the messages (as it would break the clients)
14:58:10 <mvadkert> bstinson: psss me and mprchlik are the ones who contribure to fedora-ci/metadata
14:58:21 <mvadkert> bstinson: we can include mprchlik in the next conversations
14:58:24 <jbieren> should probably check with greg allen if that port will matter to the jenkins jms-messaging plugin
14:58:41 <bstinson> jbieren: i have a separate thread with him and scott about changing all of that up
14:58:46 <mvadkert> jbieren: I do believe it is jsut witching to "UMB" providedr
14:58:48 <jbieren> bstinson, oh okay excellent
14:58:52 <mvadkert> jbieren: but we need to test :)
14:58:54 <jbieren> mvadkert, ack
14:59:10 <mvadkert> jbieren: not enve sure that if that is upstream or not
14:59:11 <mvadkert> jbieren: the plugin
14:59:14 <bstinson> so to step back for a moment
14:59:15 <mvadkert> jbieren: hopefully it is]
14:59:28 <jbieren> jms-messaging plugin? it is upstream. it is used by the jenkins master in apps.centos.ci now
14:59:37 <bstinson> i heard the following requests in flight:
14:59:55 <bstinson> - wait on metadata to describe generic tests and how to execute them
15:00:21 <bstinson> - work on a set of examples showing what the "generic" test would look like in the metadata
15:00:33 <bstinson> - alternate architectures
15:01:08 <bstinson> do we have any other major themes that need to touch this? i'm deliberately factoring away the fedora-messaging component we just discussed, since that's happening in parallel
15:01:13 <mvadkert> - setup a new sig for Fedora CI with taiga board to be able to track ^
15:01:21 <kparal> I have quite a few more requirements
15:01:25 <mvadkert> bstinson: maybe that is the same
15:01:28 <bstinson> kparal: let's hear them
15:01:34 <kparal> ok
15:01:37 <kparal> 2. Test results format must be able to specify *what* got tested.
15:01:44 <kparal> Currently the test results format [1] only allows to specify test case names and their result, and it is assumed that all results apply to the test subject as the CI system knows it. However, certain generic tests like rpmdeplint (dependency checking on an rpm repo) do not have 1:1 mapping of input test subject and output test subject, and the actual (output) test subjects are figured out dynamically during execution. Concretely,
15:01:44 <kparal> rpmdeplint takes a koji tag name as an input (f29-updates-pending) and it checks all rpms inside that tag/repo for dependency issues. The output of the test is a set of "NVR: pass/fail" results. Here's an example results file in Taskotron format:
15:01:44 <kparal> https://taskotron.fedoraproject.org/artifacts/all/499480d4-4015-11e9-be29-525400fc9f92/tests.yml/taskotron/results.yml.gz
15:01:48 <kparal> The "item" attribute specifies what got tested.
15:01:50 <kparal> [1] latest draft: https://pagure.io/fedora-ci/docs/pull-request/5#request_diff
15:02:50 * bowlofeggs has one too when we're ready
15:03:08 <mvadkert> kparal: so rpmdeplint actually tests a koji tag
15:03:14 <kparal> rpmdeplint has always been a special snowflake, so we can also think about how to implement it differently rather than support its current form. but it might be the case that some future tests will need this as well, I don't know
15:03:31 <mvadkert> kparal: we shouls rport on koji tag artifact?
15:03:44 <kparal> mvadkert: yes, but the results, to show up in bodhi, and to be able to search for them in greenwave, are reported against koji builds
15:04:49 <mvadkert> kparal: yeah, I guess it needs rethinking
15:05:15 <bstinson> we're a bit over time, but i want to continue. the output of this meeting will be a document posted to you via email and likely on the wiki if you need to drop off
15:05:35 <bstinson> kparal: what work is needed on this draft to get it completed?
15:06:01 <bookwar> i don't quite get,  why do we try to port generic tests (like rpmlint, rpmdeplint and so on) to standard test interface at all. STI is a framework to describe the test run of a upstream test suite, it is an interface where developer of a component describes to CI system, how the test should be run. For generic tests there is no developer involved, CI system knows the way the test should be run, and there is no need for configuring
15:06:02 <bookwar> ponent specific details
15:06:26 <jskladan> mvadkert: well, there is currently (AFAIK, correct me if I'm wrong) no way to report against "koji tag artifact" reasonably, since: there is no way to specify "koji tag X as it was at time Y", and also, the rpmdeplint takes koji tak as an input, but really tests builds/updates (which is to save time, bandwith, and to work around some koji/bodhi slowness, IIRC)
15:06:39 <bookwar> for example in downstream we have STI-tests plus rpmdeplint and installability check as three separate pipelines
15:06:39 <kparal> bstinson: well we either need to rework rpmlint and probably some part of the gating infra, or we need to add something like "subject: nvr" into the CI test results format
15:06:46 <mvadkert> kparal: it even feels like that test should report on all rpm builds in that koji tag, or koji tag should provide list of rpms it included so we can somehow match these things together
15:07:00 <mvadkert> jskladan: I see
15:07:06 <mvadkert> jskladan: so we should report on all rpms tested there
15:07:14 <mvadkert> jskladan:  if there is no way how to uniquerly indentify that
15:07:16 <jskladan> mvadkert: that's what we do, really
15:07:34 <kparal> bookwar: I'm not tied to STI, we just want to execute the tests in Fedora CI
15:07:47 <kparal> and I had the impression that STI is the universal format, not just for upstream test suites
15:07:52 <mvadkert> jskladan: cool, so trying to re-read where is the problem then :)
15:08:27 <jskladan> mvadkert: https://taskotron.fedoraproject.org/resultsdb/results?groups=c3d009cc-401f-11e9-a73c-525400fc9f92
15:08:37 <jskladan> mvadkert: problem is, the "builds" can not be tested in isolation
15:09:33 <kparal> mvadkert: we never needed to report with "rpms" granularity, "build" granularity was always enough for us (all test results consumers consume build results, not rpm results)
15:10:02 <jbieren> so you trigger on a koji tag and send results for each koji build in that tag?
15:10:10 <kparal> jbieren: correct
15:10:13 <jbieren> okay
15:10:22 <mvadkert> that feels a bit broken
15:10:24 <kparal> because dependency problems need to be tested as a whole repo
15:10:25 <mvadkert> in my world
15:10:42 <jskladan> mvadkert: dependency checking is hard :)
15:10:43 <mvadkert> right, but then you test the repo really, what does that information give the user?
15:10:46 <mvadkert> jskladan: I know :)
15:10:55 <bookwar> but when we run gating we test build with respect to the current state of a main tag, result of this test is tied to one package which we have changed, not the entire state of the tag
15:11:09 <jskladan> the minimal necessary context for depchecking test would be an update anyway, and that's only if all the updates were self-contained (which is the policy, but not reality)
15:11:09 <bookwar> so this test result is about the repo/tag we build from one particular change
15:11:10 <mvadkert> jskladan: kparal: just trying to udnerstand if we should not try to completely rethink this
15:11:10 <kparal> mvadkert: we're able to detect builds with broken deps and show that up in bodhi
15:11:32 <tflink> didn't bowlofeggs or someone run some analysis recently that said that 90% or so of fedora builds are self-contained with respect to dependencies?
15:11:32 <bookwar> and that's why we report it against this change, not all packages in that repo
15:11:34 <kparal> mvadkert: I'd be happy to try to rethink this, if it makes your life implementing support for it easier :)
15:11:53 <bowlofeggs> tflink: it was 95% of *updates*, not builds (slight difference)
15:11:54 <bstinson> let's add this to the list of themes and we can split off to work on this
15:12:01 <bowlofeggs> tflink: also, that was for stable releases, not rawhide
15:12:07 <mvadkert> bstinson: yeah :)
15:12:09 <bowlofeggs> not sure how i could get data for rawhide…
15:12:14 <tflink> bowlofeggs: ah, yeah - that would make a significant difference
15:12:27 <bstinson> let's keep going with a list of requirements
15:12:33 <bstinson> kparal: what's left on your list?
15:12:33 <kparal> ok
15:12:37 <bowlofeggs> tflink: i cold probably get teh data for builds if you want it though
15:12:40 <kparal> 3. CI should be able to avoid installing test subjects into the test environment, because most generic tests don't need it.
15:12:43 <bowlofeggs> (for stable releasess)
15:12:44 <kparal> This is not strictly required, but most generic tests don't operate on an installed package but rather on a set of rpms, and therefore it would be quite inefficient to always install these test subjects (and all its dependencies) before executing the generic test. If the CI system can pre-download the test subjects and provide them in a directory, that could be beneficial, but again not strictly required (our current tests handle
15:12:44 <kparal> downloading themselves, at the moment). So overall the test metadata should probable indicate to the CI system how the test subjects should be handled (pre-installed, just downloaded, nothing done).
15:13:09 <mvadkert> kparal: that is just saying it should be really a diffrent pipeline
15:13:17 <tflink> bowlofeggs: we'd need to support all of the builds eventually, though - even the "corner cases" of builds that require other builds to satisfy deps
15:13:19 <mvadkert> kparal: but yeah, defi nitely not needed to do sych a thing for hti stest
15:13:44 <mvadkert> this test :)
15:14:08 <bowlofeggs> tflink: indeed
15:14:10 <kparal> mvadkert: so, "different pipeline" still means Fedora CI?
15:14:18 <kparal> I'm not completely clear on terminology here
15:14:19 <bowlofeggs> tflink: i don't think of those as corner cases haha
15:14:24 <pingou> bowlofeggs: +1
15:14:27 <jbieren> the Jenkins master that hosts the "Fedora CI Pipeline" can have more pipelines added to it
15:14:31 <tflink> bowlofeggs: hence "corner cases"
15:15:12 <kparal> so is the different pipeline something that fedora-ci team can implement?
15:15:19 <jbieren> if its performance starts degrading, we just ask brian =)
15:15:27 * bstinson adds more machines to openshift
15:15:46 <tflink> ie, not the majority of builds - still important but not the most common case
15:15:50 <bstinson> did we come to an agreement to trial separate pipeline, or does this need extra scoping?
15:16:28 * kparal doesn't see into the implementation details of this
15:16:58 <jskladan> What is the scope/impact of "separate pipeline"?
15:18:11 <bstinson> jbieren can correct me, but the impact would mostly be on the infra side
15:18:23 <jbieren> bgoncalv, do you agree a separate pipeline could make sense here?
15:18:31 <bgoncalv> yes, it makes sense
15:18:45 <jbieren> yeah it is just another Jenkinsfile created that gets pointed at by a separate "Jenkins pipeline"
15:18:53 <kparal> sounds reasonable
15:19:22 <kparal> ok, and since rpmdeplint is a special snowflake, I have one more point
15:19:26 <kparal> 4. CI system should support some kind of rate-limiting for certain tests
15:19:29 <kparal> This is currently specific to rpmdeplint and not strictly required, but very helpful. Because rpmdeplint is executed for koji tags and not koji builds, the event that triggers it is a koji tag change. However, since rpmdeplint checks all rpms inside the tag every time, it doesn't make sense to trigger it for very fast consecutive events, that would only cause several of these tests to run in parallel and end with the same results. In
15:19:30 <kparal> this case, we employ a rate-limiting that only executes rpmdeplint at maximum once per half an hour (or similar).
15:19:47 <mvadkert> kparal: yeah
15:19:48 <bookwar> jskladan: for tests which don't require installation of the packages, we can lower requirements on workers and build a simplified workflow, without the need of nested virtualization step and other features of the "big" pipeline
15:19:53 <mvadkert> kparal: same infra
15:20:28 <kparal> that's all I had
15:20:31 <jbieren> kparal, we call that build throttling. very easy to add. but if I get 5 in one half hour, which of the 5 do i use?
15:20:54 <jskladan> bookwar: makes sense
15:20:57 <tflink> jbieren: not sure I follow
15:21:05 <tflink> you mean which trigger message do you use?
15:21:27 <jbieren> tflink, yes
15:21:43 <kparal> as long as the input subject is the same (e.g. f29-updates-pending), it doesn't matter
15:21:44 <jbieren> hopefully the first of the five? and drop the others?
15:21:45 <bookwar> kparal: the 4th requirement won't apply for gating. We are going to create individual sidetag for every package we gate, this means test results for one sidetag can not be used as a test for other package, it is all isolated
15:21:50 <kparal> it results in the same test being executed
15:21:54 <tflink> we have it set up so that the rpmdeplint job has a "fuse" on it. the first trigger starts the fuse and subsequent triggers are effectively ignored
15:22:07 <jbieren> okay
15:22:16 <jbieren> i don't suspect that would be much of a problem
15:22:42 <bookwar> kparal: in gating "one build = one koji tag"
15:23:22 <kparal> bookwar: in that case there might be areas how to simplify rpmdeplint substantially. but we also need to support the "manual" use case of stable releases where package maintainers can push or unpush updates at any time, not just automatic rawhide gating
15:23:40 * jskladan needs to step out for a bit, will be back in ten or so minutes
15:24:39 <bstinson> i'm hoping we can get through the rest of our list since we're over time, we'll definitely want to do some followup syncs for the various "themes"
15:24:48 <bstinson> kparal: anything else on your list?
15:24:57 <kparal> bstinson: nope, that's all I could think of
15:24:57 <bstinson> bowlofeggs: i saw you had something...did i miss it?
15:25:16 <bookwar> kparal: agreed, we should run periodic or custom triggered "koji tag and/or compose" testing as well, just wanted to highlight the package-gating specifics here
15:25:35 <bowlofeggs> bstinson: yeah one feature i would like to see is a "re-run this test please" API ☺
15:25:46 <kparal> I'll go through the meeting log and file tickets in fedora-ci/metadata and elsewhere for specific problems
15:26:03 <bowlofeggs> for when a packager thinks a test might have been wrong, or (more common) it didn't run at all
15:26:06 <bstinson> kparal: thanks
15:26:21 <kparal> bowlofeggs: that's a very good point
15:26:40 <mvadkert> 10:25 < bowlofeggs> bstinson: yeah one feature i would like to see is a "re-run this test please" API ☺
15:27:01 <mvadkert> yeah, a "retrigger" event in fedora-ci/messages
15:27:13 <mvadkert> that Ci systems shoudl then be able to trigger on
15:27:17 * pingou remembers a past initiative for that
15:27:34 <bstinson> is that scoped somewhere already?
15:27:44 <kparal> pingou: it might be easier when all the tasks are in Fedora CI :)
15:27:46 <bowlofeggs> i think something simple like sending a message over amqp might be nice
15:27:55 * tflink doesn't remember how far we got with it
15:27:59 <mvadkert> kparal: no :) it whould be on the message bus
15:28:27 <pingou> tflink: there was a POC in a service that got shut down in favor of a library that never happened
15:28:35 <bookwar> there are two aspects of it: message to retrigger test from a particular ci system vs message to say "everyone, whoever you are, please retest this artifact"
15:28:56 <tflink> pingou: ah, I didn't realize it got that far
15:29:05 <pingou> message bus +1, it's the most test system agnostic
15:29:49 <bstinson> sounds like this may need to be rescoped/rediscovered
15:29:55 <bstinson> ok i have a list of themes here
15:30:13 <bstinson> how this is going to work (until we get the taiga thing) is that i'm going to put these in a document for all of us to share
15:31:02 <bstinson> if you spoke up under a particular theme, i'll send those groups a message summarizing and we can talk about next steps for each
15:31:10 <kparal> bstinson: here's my draft of requirements in a more accessible form than the meeting log: https://etherpad.gnome.org/p/taskotron-fedora-ci
15:31:22 <bstinson> kparal: that's very helpful, thanks
15:31:38 <bstinson> if you didn't speak up, but would like to participate on one of the themes, let me know
15:32:12 <bstinson> what else should we cover? we got a lot of information out here today
15:32:24 <bstinson> thank you all for staying over
15:32:44 <kparal> thanks everyone for participating
15:32:57 <jskladan> ad retriggering - tasktron trigger supports it for quite a long time, not sure about any other bits outside taskotron
15:33:45 <bstinson> with no new topics, we'll close the meeting in 1 minute
15:34:52 <bstinson> thank you all again!
15:34:55 <bstinson> #endmeeting