16:30:33 <dgilmore> #startmeeting RELENG (2015-03-02)
16:30:33 <zodbot> Meeting started Mon Mar  2 16:30:33 2015 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:42 <dgilmore> #meetingname releng
16:30:42 <zodbot> The meeting name has been set to 'releng'
16:30:42 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou
16:30:42 <zodbot> Current chairs: bochecha dgilmore masta nirik pbrobinson pingou sharkcz tyll
16:30:43 <dgilmore> #topic init process
16:30:46 <nirik> morning
16:32:00 * sharkcz is here
16:33:30 <dgilmore> lets wait a minute for a few more to turn up
16:35:45 * jreznik around
16:36:46 * pwhalen lurks
16:37:02 * pbabinca here
16:37:12 * stickster here, fwiw
16:37:29 <dgilmore> #topic #5931 [Proposal] Move new branch and unretire requests to pkgdb2
16:37:35 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5931
16:37:51 <dgilmore> pingou: are we ready to move this live post alpha?
16:38:02 * nirik thinks we are.
16:38:09 <dgilmore> I think so too
16:38:17 <pingou> I'd like to hear back from more testers, especially admins, but otherwise +1 forme
16:38:30 <dgilmore> pingou: cool, I will try do some testing
16:38:34 <pingou> dgilmore: please :)
16:38:46 * pingou likes to squash bugs early :)
16:38:48 <dgilmore> #info 16:39 < pingou> I'd like to hear back from more testers, especially admins, but otherwise +1 forme
16:39:01 <dgilmore> pingou: squashing bugs early is good
16:39:11 <dgilmore> #topic #6016 Use fedpkg-minimal in Fedora buildroots
16:39:16 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6016
16:39:26 <pbabinca> No update there :(
16:39:31 <dgilmore> pbabinca: so we have made this change for f22 and rawhide
16:39:46 <pbabinca> oh
16:39:49 <dgilmore> pbabinca: we need branches for epel and f21 and f20
16:40:21 <dgilmore> it cut about 20 seconds off making a srpm on x86 and about 60 seconds off making a srpm on arm
16:40:33 <pbrobinson> \o/
16:40:48 <pbabinca> Will do then.
16:41:01 <nirik> every bit helps. ;)
16:41:08 <dgilmore> we need to make the changes elsewhere. just need builds
16:41:16 <nirik> also makes it less likely to break the srpm buildroot
16:41:24 <dgilmore> indeed it does
16:41:52 <dgilmore> #action pbabinca to get builds on all supported releases
16:42:05 <dgilmore> #topic #5959 Enable keep-alive connections for koji (primary and secondaries)
16:42:09 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5959
16:42:24 <dgilmore> I got this done last week prior to infra freeze
16:42:38 <nirik> yep. done -> closed. ;)
16:42:49 <dgilmore> just forgot to close the ticket
16:42:54 <dgilmore> I did update it
16:43:09 <dgilmore> #topic #5963 Orphaned vulnerable packages in EPEL
16:43:14 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5963
16:43:33 <dgilmore> #info waiting on fedpkg-minimal to be in use everywhere
16:43:44 <dgilmore> #topic #6027 secondary arch old mash trees cleanup
16:43:49 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6027
16:44:01 <dgilmore> pbrobinson: any change to work on this?
16:44:13 <nirik> we might clean up primary again too soon...
16:44:16 <nirik> getting low on space.
16:44:21 <dgilmore> I am not sure this needs to stay on the list to be discussed each week
16:44:28 <nirik> yeah, probibly not.
16:44:36 <nirik> really it should just go into buildrawhide / buildbranched
16:44:46 <dgilmore> but we do need to get something in place
16:44:55 <dgilmore> also for updates mashes
16:45:23 <nirik> yep. we can work on it as we have time.
16:45:39 <dgilmore> #action dgilmore to remove meeting keyword
16:46:39 <dgilmore> #topic #5886 need method for distributing urgent fixes... urgently
16:46:45 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5886
16:46:57 <dgilmore> I have been thinking about this a bit
16:47:10 <nirik> I think we just need to gather stakeholders and decide things.
16:47:25 <dgilmore> I think for f22 we should add a security repo and some tags etc. we can do something manually
16:47:42 <dgilmore> but really I think we need a way to deal with it in bodhi
16:48:00 <dgilmore> to manage untagging etc
16:48:20 <dgilmore> perhaps we should start an email thread
16:48:23 <nirik> I think we should do nothing until we decide all the questions around it. ;)
16:48:30 <nirik> but yes, having it in bodhi might be good.
16:49:00 <dgilmore> a big thing missing without bodhi is the updateinfo for those using the security plugin
16:49:10 <nirik> true.
16:49:20 <nirik> we may be able to inject that without bodhi... not sure.
16:49:29 <jreznik> nirik: what kind of questions?
16:49:31 <dgilmore> maybe
16:49:43 <dgilmore> jreznik: I think mostly how it should work
16:49:55 <nirik> jreznik: should we do multillib, should we do delta rpms, should we put in mirrormanager or just have it on master mirrors,
16:49:57 <nirik> I had a big list
16:51:21 <nirik> https://fedoraproject.org/wiki/Urgent_updates_policy
16:51:27 <nirik> there's a list there
16:51:27 <jreznik> ok but maybe the email thread can help to get answers for it
16:51:51 <nirik> perhaps.
16:51:52 <dgilmore> nirik: we really need to use MM unless we sign the metadata
16:52:15 <nirik> dgilmore: well, if the packages are signed, why isn't that enough?
16:52:16 <dgilmore> but if we sign the metadata it will being up doing the metadata signing everywhere
16:52:30 <nirik> metadata signing is it's own can of worms. ;)
16:52:35 <dgilmore> nirik: the whole reason we have metalinks and not mirrorlists anymore
16:52:40 <dgilmore> cache poinsoning
16:52:54 <walters> nirik, http://theupdateframework.com/ has a paper which talks about some attacks on package-but-not-metadata signing
16:53:02 <nirik> yes yes, I have read it. ;)
16:53:09 <dgilmore> metalink provides a way to verify you are getting the intended contents
16:53:14 <nirik> that would also require MITM https right?
16:53:55 <dgilmore> nirik: maybe
16:54:09 <dgilmore> when that was all looked at there was no https in use
16:54:18 <jreznik> one optional thing to add could be private builds, I know how hard it could/will be but can help in such cases to have all in place and "just" push the button... the last time we talked about it was when Java guys proposed it for theirs security updates... just note after reading the list
16:54:20 <nirik> yeah, those papers were long ago
16:54:26 <dgilmore> lots of questions
16:54:43 <nirik> there's really no way to do private builds, I don't think, not without tons of work
16:54:55 <walters> i think so, but note that's accessible to anyone who owns or has compromised a CA in ca-certificates
16:54:58 <dgilmore> jreznik: supporting embargos is going to take a massive amount of work
16:55:26 <jreznik> dgilmore: I understand
16:55:30 <jreznik> that
16:55:43 <nirik> also, they were talking about the mirror network and having a bad mirror in it... but if we use dl.fedoraproject.org for these, thats not a problem
16:55:47 <walters> anyways is this the right meeting to talk about atomic rel-eng integration?  Should I file a ticket for next week?
16:55:51 <dgilmore> walters: or anyone using a package that installs a ca key into the keyring
16:56:04 <dgilmore> add 3rd party repo that imports its own ca cert a
16:56:14 <dgilmore> and makes it trusted systemwide
16:56:15 <nirik> anyhow, IMHO, we could try and answer these questions on list or make a meeting on it.
16:56:33 <walters> we could do what RHN does - offer a custom CA root and configure yum to only look at it (as an optional entrypoint)
16:56:39 <dgilmore> nirik: yeah. we should get this moving along
16:56:39 <walters> ostree also supports this
16:57:07 <walters> if you also e.g. turn off SSLv3 and other bad algorithms, you get quite strong security
16:57:21 <dgilmore> we have sslv3 turned off in fedora
16:57:48 <dgilmore> anyway. lets get a wider discussion happening
16:57:55 <dgilmore> #topic #5870 rawhide signing
16:57:59 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5870
16:58:05 <nirik> walters: can talk about that in open floor. ;) or yeah, ticket and meeting next time.
16:58:34 <tyll> I do not have access to the F23 key, therefore I cannot do test runs currently
16:58:36 <dgilmore> so I have almost all of rawhide signed. but we still have a bit to go to get automatic siging
16:58:54 <dgilmore> tyll: I will be getting people access to the f23 key this week
16:59:08 <tyll> #u
16:59:16 <tyll> #info  dgilmore> so I have almost all of rawhide signed. but we still have a bit to go to get automatic siging
16:59:22 <tyll> #info < dgilmore> tyll: I will be getting people access to the f23 key this week
16:59:23 <nirik> yeah, so what are next steps here?
16:59:28 <dgilmore> I do not think we can really enable auto signing and rely on it until we get teh sigul bugs fixed
16:59:42 <nirik> dgilmore: well, it's not actually locked up for a while now.
16:59:58 <nirik> it didn't even yesterday when you were doing rawhide signing, just really slow
17:00:00 <dgilmore> nirik: the client locks up on me frequently
17:00:16 <tyll> nirik: the automatic signing did not run since December
17:00:21 <dgilmore> to the point i have a process running that cleans it up
17:00:24 <nirik> I've had 0 lockups with 1 as batchsize
17:00:37 <nirik> also, I have noticed sometimes somethings take a long time to sign.
17:00:38 <dgilmore> nirik: 1 as the batchsize has never locked up
17:00:41 <nirik> it's not actually hung
17:00:45 <tyll> batchsize 1 would be too slow to catch it up with all builds
17:00:58 <nirik> ie, texlive or debuginfo from various packages that are really large
17:01:00 <dgilmore> nirik: it uses a different code path that doesnt lock up but is slow
17:01:27 <nirik> ive seen it take like 45min to sign a 700MB debuginfo file
17:01:40 <dgilmore> nirik: right
17:01:59 * masta is here
17:01:59 <nirik> anyhow, I guess I'd say lets fire up autosign later this week and see where it's at?
17:02:00 <tyll> can the sigul vault get faster hardware/better network?
17:02:19 <dgilmore> tyll: it is very decent on both
17:02:22 <nirik> tyll: it's already a pretty beefy box. The slowness is not there I don't think
17:02:34 * pbrobinson is back, sorry connectivity issyes
17:03:08 <dgilmore> part of the slowness is that the netapp's are really slow
17:03:31 <masta> I'll be using batchsize 1 going forward, good to know
17:03:36 <dgilmore> it would need proper analysis to see what is the whole picture
17:04:06 <nirik> it's got 16cpus and 16GB memory...
17:04:07 <tyll> are there any plans to setup a sigul in staging to be able to debug issues?
17:04:16 <dgilmore> I think we need to be able to sign in big batches
17:04:20 <nirik> and 1000MB net
17:04:27 <tyll> nirik: if the NFS is slow, I guess that is the issue
17:04:27 <dgilmore> tyll: there is no plans that I am aware of
17:04:36 <dgilmore> tyll: there is also no one working on sigul
17:04:48 <nirik> I'm not sure where the slowness is really. It's all speculation
17:05:07 <nirik> I don't think it's nfs actually... because then all koji builds would be slow...
17:05:08 <dgilmore> it is purely speculation
17:05:30 <tyll> without being able to login to the vault to inspect things it is hard to figure it out
17:05:50 <masta> well I'm guessing the crypto card is the slow part, where the keys are held and the computation happens. Presumably the 16 cores are not computing the signatures, they are safe on the crypto hardware. Right?
17:06:04 <tyll> masta: afaik there is no crypto card
17:06:06 <nirik> there is no crypto hw. sigul is software. ;)
17:06:17 <masta> oh wow! ok then, my bad.
17:06:28 <nirik> anyhow, I am happy to try and track down slowness or whatever as my time permits.
17:06:39 <nirik> ie, I will add it to the list, but it's not urgent
17:06:54 <dgilmore> tyll: the vault is intentionally locked off. almost no one can get on it
17:07:01 <dgilmore> it does not have ssh access
17:07:02 <Corey84> uses a ybikey style setup no?
17:07:16 <dgilmore> masta: there is no crypto card
17:07:22 <tyll> dgilmore: yes, this is why I asked for a staging system
17:07:38 <dgilmore> tyll: someone would need to step up to do the work
17:07:56 <nirik> I could try and setup a stg one too, but... again, there's only so many hours in the day..
17:08:12 <dgilmore> Corey84: no
17:08:12 <tyll> btw. I also proposed sigul fixes as a GSoC project, maybe we will get some help there
17:08:32 <dgilmore> tyll: maybe. but it will be a lot of work to just get up to speed
17:08:33 <masta> I'd like to profile the slowness, but not sure I have the right access.
17:08:40 <dgilmore> masta: you do not
17:08:49 <Corey84> I'm applying for the GSoC I'll look for that one
17:08:49 <dgilmore> masta: almost no one has the access
17:08:52 <nirik> the lockups are more an issue I guess than slowness.
17:08:58 <dgilmore> right
17:09:03 <masta> dgilmore: ok, I get the implications of that. =(
17:09:12 <dgilmore> the slowness while annoying is not a huge deal
17:09:29 <dgilmore> the thing that really needs fixed is the lockups
17:09:37 <masta> yeah, the lockups
17:09:45 <nirik> so, we should re-enable autosign later this week, confirm the lockups still happen...
17:09:47 <tyll> nirik: if you setup the staging vms I can take a look into installing sigul on them
17:09:48 <nirik> and then go from there?
17:10:06 <dgilmore> nirik: we should
17:10:20 <nirik> tyll: it would also involve fixing up the stg koji... it's not happy right now and I haven't had time to figure out why
17:10:47 <dgilmore> and doing a bunch of builds in stage to have something to sign
17:10:58 <nirik> and setting up new keys in stg
17:11:03 <dgilmore> right
17:11:08 <dgilmore> its a big undertaking
17:11:15 <tyll> nirik: ah, I see
17:11:32 <sharkcz> but should be done :-)
17:11:40 <nirik> but we can get there.
17:11:53 <nirik> it will just not be something I at least drop everything to do right now this second. ;)
17:12:52 <nirik> next topic?
17:12:54 <dgilmore> okay lets move on
17:13:00 <tyll> the lock-ups happen on the bridge, not on the vault?
17:13:40 <dgilmore> tyll: mostly
17:13:43 <dgilmore> but not always
17:13:52 <tyll> ah, ok
17:14:05 <dgilmore> #topic #6064 extend srpm-excluded-arch.py so it can read srpms from multiple dirs
17:14:11 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6064
17:14:20 <dgilmore> #action dgilmore to review the patches
17:14:33 <dgilmore> #topic Secondary Architectures updates
17:14:34 <dgilmore> #topic Secondary Architectures update - ppc
17:14:40 <dgilmore> pbrobinson: how is ppc?
17:14:49 <pbrobinson> getting there
17:14:57 <pbrobinson> I'll be doing a TC this evening or tomorrow
17:15:06 <dgilmore> cool
17:15:08 <pbrobinson> just awaiting the signing to complete
17:15:33 <dgilmore> TC's do not have to have everything signed
17:15:34 <sharkcz> from build perspective - gcc5 is fixed, now systemd (big endian fix is in upstream), then seccomp openssh (fixed in mainline already)
17:15:58 <sharkcz> I've asked for freezeexception for the fixed gcc5
17:16:09 <pbrobinson> sharkcz: it should -18 not -17 btw
17:16:21 <dgilmore> sharkcz: cool
17:16:37 <pbrobinson> as -18 fixes a few more things like firefox/thunderbird builds
17:16:41 <dgilmore> ppc is okay otherwise?
17:16:49 <pbrobinson> yes, we're getting there
17:16:57 <dgilmore> pbrobinson: has imagefactory etc been looked ta yet?
17:17:02 <walters> (i filed a ticket for the atomic bits, have to step out for a bit but will be back later.  Please feel free to follow up in ticket or via irc/email)
17:17:12 <dgilmore> imcleod: where is oz/imagefactory for non x86?
17:17:25 <dgilmore> walters: will do
17:17:31 <pbrobinson> dgilmore: some bits, I need to get to the hosts along with the a64 hosts this week with nirik
17:18:14 <nirik> pbrobinson: just ping me if you get stuck anywhere. ;)
17:18:24 <dgilmore> pbrobinson: okay, hopefully imcleod can help with the oz/imagefactory bits
17:18:33 <pbrobinson> nirik: yes, it's purely been timing related
17:18:44 <pbrobinson> dgilmore: was speaking iwth him earlier
17:18:53 <dgilmore> cool
17:18:56 <dgilmore> #topic Secondary Architectures update - s390
17:19:02 <dgilmore> sharkcz: how is s390?
17:19:38 <sharkcz> no time for composes yet, the rest is similar to ppc, both major issues are big endian related
17:20:02 <dgilmore> okay
17:20:15 <dgilmore> sharkcz: will there be an f22 alpha?
17:20:37 <sharkcz> I hope it will, the builds should be ready
17:20:45 <dgilmore> okay cool
17:20:50 <dgilmore> #topic Secondary Architectures update - arm
17:20:57 <dgilmore> pbrobinson: how is aarch64?
17:21:04 <nirik> pbrobinson: do you know if there's any news on power8 stuff? or those machines are all still pending setup?
17:21:13 * nirik sorry to have missed the right topic for this
17:21:19 <pbrobinson> the main issue is ghc is still broken
17:21:37 <dgilmore> :( okay
17:21:46 <pbrobinson> other than that builds are mostly up to date and similar to PPC plan an initial TC early this week
17:22:01 <dgilmore> sounds good
17:22:16 <sharkcz> pbrobinson: is there a bug for the issues fixed in the -18 gcc5?
17:22:16 <dgilmore> hopefully we get a RC request for primary this week
17:22:16 <pbrobinson> nirik: on my pile, this week with luck :)
17:22:33 <dgilmore> pbrobinson: anything else?
17:22:38 <pbrobinson> sharkcz: not sure, I was following the upstream
17:22:46 <pbrobinson> dgilmore: not for a64
17:23:06 <dgilmore> cool
17:23:08 <dgilmore> #topic Open Floor
17:23:18 <dgilmore> anyone have anything for open floor?
17:23:50 <nirik> I had one thing...
17:23:57 <sharkcz> we would like to move the build status agregator jcajka_ is working on to some fedora devel server from his private server
17:23:57 <dgilmore> nirik: shoot
17:24:15 <nirik> seems f22 source and debuginfo isn't right in mm... https://bugzilla.redhat.com/show_bug.cgi?id=1197779
17:24:19 <dgilmore> sharkcz: after nirik
17:24:20 <sharkcz> nirik: I guess jcajka_ should talk with you
17:24:28 <sharkcz> ok
17:24:32 <nirik> would be nice to fix that up. Requires db twiddling?
17:24:34 <nirik> sharkcz: sure.
17:24:37 <dgilmore> nirik: I will poke at the mm db
17:25:11 <dgilmore> nirik: that url the guy used in teh bug is invalide
17:25:18 <dgilmore> invalid
17:25:37 <dgilmore> arch is supposed to be source
17:26:13 <dgilmore> could be a bug in fedora-repos
17:26:21 <dgilmore> sharkcz: so what is it?
17:26:42 <sharkcz> jcajka_: ^^
17:27:00 <dgilmore> http://fedoraproject.org/wiki/Request_For_Resources
17:27:17 <dgilmore> but I am curious what it is.
17:27:19 <jcajka_> dgilmore,  nirik: thanks, sharkcz was quicker than I :), I would like to ask for testing/devel server, preferably in same network as koji hubs, to develop and test http://185.8.164.45
17:27:38 <jcajka_> it is hosted now on my testing VPS
17:27:39 <dgilmore> jcajka_: that does not explain what it is
17:27:47 <dgilmore> please use soem words and not a ip
17:28:16 <sharkcz> it aggregates the status of builds across all koji instances using fedmsg
17:28:17 <jcajka_> it is koji agreggator to provide overview of build statuses over all kojis
17:28:51 <dgilmore> jcajka_: I am still not really sure what that exactly means
17:29:33 <dgilmore> it kinda sounds like a small part of the redesign I want to get done on teh packager web interface
17:29:50 <sharkcz> it allows you to see on what arches/kojis is a build failing
17:29:52 <jcajka_> it provides quick way to see wherever builds are failing only on one koji instance of on multiple, for example
17:30:45 <sharkcz> and later should share notes written by the devel/maintainers about the failures to make later debugging easier, etc...
17:30:47 <dgilmore> jcajka_: okay. honestly I would rather we not do that.  but instead work on building a new interface, that integrates the koji's bodhi and pkgdb
17:31:15 <pbrobinson> can we do it as an interim stop gap?
17:31:23 <dgilmore> I can not stop you doing it. but I would rather provide a unifed developer portal
17:31:59 <pbrobinson> I've been told that should happen for the 5 years I've been involved in secondary arches ;-)
17:32:02 <nirik> would a cloud instance be usefull shorter term?
17:32:10 * sharkcz don't disagrees, but have this now which could serve until the new UI is developed
17:33:28 <dgilmore> nirik: that does seem like the next step
17:34:03 <nirik> RFR needs things like: multiple people who know the code, packaged in epel, monitoring, multiple nodes if possible, etc, etc
17:34:08 <nirik> so that will be a while to work thu
17:34:20 <dgilmore> nirik: right, this seems like its early dev stage
17:34:51 <jcajka_> dgilmore: I do agree, but current web tools, are rather bulky
17:35:15 <nirik> all that seems like a pain, but mostly it's to try and protect us from having some thing that no one knows how to maintain or the like down the road.
17:35:46 <dgilmore> jcajka_: well the web front ends would go away
17:36:03 <dgilmore> jcajka_: we would just use the backends and have a new frontend
17:36:31 <dgilmore> anyway jcajka_ would a cloud instance in fedora be useful for development?
17:36:47 <nirik> this might also be something that could expand the releng dash?
17:36:48 <sharkcz> dgilmore: is this something we should work on (or discuss) during the planed releng FAD?
17:36:50 <smooge> a cloud instance does not require an RFR
17:36:58 <jcajka_> dgilmore: yes, side note also hrw started similar project, so there is someneed
17:37:21 <dgilmore> https://fedoraproject.org/wiki/Infrastructure_private_cloud
17:37:42 <nirik> might be good to discuss on list too and see if several similar things could consolidate
17:37:51 <dgilmore> likely
17:38:24 <nirik> http://threebean.org/fedora-releng-dash/ (releng dash btw)
17:38:39 <nirik> well, thats not the right one...
17:38:56 <nirik> https://apps.fedoraproject.org/releng-dash/
17:39:51 <dgilmore> that seems broken right now
17:40:02 <dgilmore> and its not something releng supported
17:40:04 <nirik> I think it's just really slow...
17:40:16 <dgilmore> perhaps
17:40:18 <nirik> but there's some db changes after the freeze that will help that hopefully
17:40:34 <dgilmore> but it is trying to list out of date info
17:40:40 <dgilmore> f19 updates
17:40:46 <dgilmore> and is missing branched entirely
17:41:25 <dgilmore> and doesnt have the new vagrant image
17:41:33 <dgilmore> anyway that is a bit off topic
17:41:42 <sharkcz> I'm open to any development in this topic, because we need some tool to help us coordinate the work on fixes for secondary arches, to see who picked what failure, etc.
17:41:55 <dgilmore> sharkcz: we need a lot of tools
17:41:59 <nirik> it likely needs updated too.
17:42:04 <nirik> anyhow.
17:42:18 <dgilmore> I want to move koji-shadow into being run in fedora automatically
17:42:37 <dgilmore> sharkcz: and have the compose process for primary do secondary composes
17:43:03 <sharkcz> I know ...
17:43:15 <dgilmore> There is massive amounts of room for improvement
17:43:54 * nirik thinks a concrete roadmap should be a good deliverable for the upcoming FAD
17:44:07 <dgilmore> sharkcz: I think rather than a bug of people going off and developing tools in issolation that we have a cohesive plan that we work towards
17:44:16 <dgilmore> nirik: indeed
17:44:46 <dgilmore> maybe a new tool is step one to unifying things
17:45:03 <dgilmore> then we integrate the existing tools one by one
17:45:10 <sharkcz> dgilmore: sure, that's why I shared the info about the FAD in our team
17:45:13 <dgilmore> but we do need a roadmap
17:45:40 <nirik> and yeah, it would be nice if it was all one tool/place... not 10 tools/sites
17:46:06 <dgilmore> nirik: right
17:46:35 <sharkcz> this concrete tool came from an idea some time ago, served as a learning example, before we have a roadmap and other plans
17:47:38 <dgilmore> okay
17:48:04 <dgilmore> so i guess jcajka_ should get a fedora infra cloud instance
17:49:15 <dgilmore> anything else?
17:49:17 <jcajka_> dgilmore: thank you
17:49:24 <sharkcz> nope
17:50:06 <dgilmore> I will follow up on walters's ticket with why we can not use what he has written in out setup.  though it is useful for people at home to do things
17:50:31 <dgilmore> ill end the meeting in a minute or two if nothing else comes up
17:52:06 <dgilmore> #endmeeting