17:00:07 #startmeeting fedora_atomic_wg 17:00:07 Meeting started Wed Jul 12 17:00:07 2017 UTC. The chair is dustymabe. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:07 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:00:07 The meeting name has been set to 'fedora_atomic_wg' 17:00:13 #topic roll call 17:00:16 .hello yzhang 17:00:16 .hello dustymabe 17:00:16 yzhang: yzhang 'Yu Qi Zhang' 17:00:19 dustymabe: dustymabe 'Dusty Mabe' 17:00:25 .hello sayanchowdhury 17:00:25 .hello jdoss 17:00:25 sayan: sayanchowdhury 'Sayan Chowdhury' 17:00:28 jdoss: jdoss 'Joe Doss' 17:00:29 Howdy. 17:00:32 .hello sinnykumari 17:00:33 ksinny: sinnykumari 'Sinny Kumari' 17:00:36 .hello stefw 17:00:37 stefw: stefw 'Stef Walter' 17:00:40 Oops, wrong channel. 17:00:41 .hello maxamillion 17:00:42 maxamillion: maxamillion 'Adam Miller' 17:00:59 #chair yzhang dustymabe sayan jdoss walters ksinny stefw maxamillion jbrooks 17:00:59 Current chairs: dustymabe jbrooks jdoss ksinny maxamillion sayan stefw walters yzhang 17:00:59 .hello Spyros Trigazis 17:01:00 strigazi: Sorry, but you don't exist 17:01:12 .hello strigazi 17:01:13 strigazi: strigazi 'Spyros Trigazis' 17:01:15 #chair strigazi 17:01:15 Current chairs: dustymabe jbrooks jdoss ksinny maxamillion sayan stefw strigazi walters yzhang 17:01:19 suuup 17:01:28 bowlofeggs: o/ 17:01:31 Ahoy 17:01:38 .fas jasonbrooks 17:01:38 jbrooks: jasonbrooks 'Jason Brooks' 17:01:40 #chair bowlofeggs 17:01:40 Current chairs: bowlofeggs dustymabe jbrooks jdoss ksinny maxamillion sayan stefw strigazi walters yzhang 17:01:47 #chair rubao 17:01:47 Current chairs: bowlofeggs dustymabe jbrooks jdoss ksinny maxamillion rubao sayan stefw strigazi walters yzhang 17:01:54 #chair scollier 17:01:54 Current chairs: bowlofeggs dustymabe jbrooks jdoss ksinny maxamillion rubao sayan scollier stefw strigazi walters yzhang 17:02:00 .hello jberkus 17:02:00 jberkus: jberkus 'Josh Berkus' 17:02:12 #chair jberkus 17:02:12 Current chairs: bowlofeggs dustymabe jberkus jbrooks jdoss ksinny maxamillion rubao sayan scollier stefw strigazi walters yzhang 17:02:12 .hello rubao 17:02:17 rubao: rubao 'rubao' 17:02:18 good turnout today 17:02:20 .hello scollier 17:02:21 scollier: scollier 'Scott Collier' 17:02:40 dustymabe, that ping helped :) 17:03:01 roshi around today? 17:03:21 he was been quiet lately, but now that f26 is out i think he should perk back up a bit 17:03:29 s/was/has 17:03:50 kushal around? 17:04:02 oh well - let's get started 17:04:08 #topic previous meeting action items 17:04:22 I have to jet to another meeting in 15 but I wanted to ask if it was OK to open an issue to consider getting Fedora 26 added to Google Compute Engine's official images since CentOS and RHEL images are present already. 17:04:24 .hello miabbott 17:04:25 miabbott: miabbott 'Micah Abbott' 17:04:54 jdoss: that's something I wanted to start the discussion in the open floor 17:04:55 jdoss: I think it's something we want to do - there may be a ticket for it already, but if not feel free to open one 17:05:03 If the answer is yes I will open an issue with the steps I used to get it going for my own needs 17:05:14 ok here are the previous meeting action items 17:05:17 * maxamillion roshi to come up with guidelines for meeting quorum for 17:05:18 the atomic working group 17:05:20 * dustymabe to propose a VFAD for wiki and docs discussion 17:05:22 * maxamillion to look at logs from last meeting and add summary of our 17:05:24 discussion to ticket https://pagure.io/atomic-wg/issue/284 17:05:26 * maxamillion to release the fedora-minimal base image with next 17:05:28 container release: https://pagure.io/atomic-wg/issue/290 17:05:30 * dustymabe to update the kubernetes thread with tracking options: 17:05:32 https://pagure.io/atomic-wg/issue/287 17:05:34 * dustymabe to open a ticket for cloud/atomic confusion 17:05:48 #info dustymabe proposed a VFAD https://pagure.io/atomic-wg/issue/294 17:05:58 so for https://pagure.io/atomic-wg/issue/290 ... I commented, but I didn't really sum up what was said in the meeting, which I completely forgot about 17:06:22 #info dustymabe added update to kubernetes thread: https://pagure.io/atomic-wg/issue/287#comment-447645 17:06:27 because I'm just bad at that lately 17:06:38 * maxamillion can't seem to keep his head on straight 17:07:00 #info dusty opened a ticket regarding mailing list/irc channel for atomic/cloud working group: https://pagure.io/atomic-wg/issue/295 17:07:02 dustymabe: I'm triple-timing until :30, will be more responsive at that point 17:08:14 maxamillion: you mean for #284? 17:08:35 dustymabe: I do 17:08:41 ok 17:08:49 for #290 we are done - its in the registry 17:08:51 right? 17:09:03 dustymabe: yes 17:09:14 #info fedora-minimal container image is in the fedora registry now https://pagure.io/atomic-wg/issue/290 17:09:34 ok the only item left is: * maxamillion roshi to come up with guidelines for meeting quorum for the atomic working group 17:09:43 re-action I assume? 17:10:59 dustymabe: yeah, I'll try and track him down soon 17:11:18 #action maxamillion roshi to come up with guidelines for meeting quorum for the atomic working group 17:11:25 ok moving on to meeting items 17:11:44 #topic Start using new mailing list/IRC channel around f26 release time 17:11:52 #link https://pagure.io/atomic-wg/issue/295 17:12:15 * dustymabe will give people some time to read the ticket 17:12:23 TL;DR - let's start using 17:12:31 the atomic mailing list? 17:12:31 'atomic' named things for this working group 17:12:58 atomic mailing list - fedora-atomic irc channel - atomic calendar entries in fedocal 17:13:03 atomic namespace on the wiki 17:13:20 etc.. 17:13:22 makes sense 17:13:29 I plus one-d 17:13:44 yes - please add your thoughts to the ticket 17:13:46 +1 17:14:13 +1 17:14:28 cloud* stuff won't go away - just will now be used more eclusively for cloud base image 17:14:34 and related issues 17:14:35 what happens to the cloud channel? 17:14:44 sayan: ^^ 17:14:45 ah ok 17:14:47 Yeah 17:14:50 Is there too much traffic in the existing places? 17:14:59 I mean, why bother? 17:15:02 gholms: it's mostly just confusing to new users 17:15:06 this proposes adding #fedora-atomic in addition to #atomic ? 17:15:26 walters: yeah - i teetered on that one 17:15:36 could be swayed 17:15:43 indeed, why not just #atomic ? 17:15:53 dustymabe: A redirect can solve that one. 17:16:00 i feel like there are a lot of fedora specific discussions that happen in #fedora-cloud 17:16:10 that #atomic channel won't care much about 17:16:24 Is it okay for people to go to the Atomic upstream channel with Fedora traffic? 17:16:39 That'd be the main argument for or against that. 17:16:43 right 17:16:45 isn't the Fedora version of atomic *the* upstream Atomic Host project? 17:16:53 stefw: +1 17:16:58 or should be anyway :-) 17:17:00 stefw: depends on who you ask 17:17:04 well - i think there is grey area 17:17:11 #atomic is more a collection of upstream projects 17:17:18 including atomic CLI 17:17:19 If it is then the messaging needs an awful lot of work. 17:17:19 well for the host as a whole 17:17:21 "project atomic" adds to the confusion 17:17:26 rpm-ostree ostree blah blah 17:17:29 it's already confusing it enough as it is ... i'd say if you're changing things around ... give a nod toward simplicity and consistency 17:17:52 It was always confusing to me on all the atomic chatter in the #fedora-cloud channel since I haven't drank the atomic koolaid yet 17:17:55 stefw: suggestion? 17:17:56 I wouldn't mind using #atomic for more fedora atomic (and centos atomic) stuff 17:18:12 jdoss represents an outside user 17:18:14 If we're changing channels, it would be better to just use #atomic 17:18:15 #atomic ... or if you don't want host specific stuff there then #atomic-host 17:18:21 jberkus ++ 17:18:24 so he is a good example of someone that would be confused by some of our choices 17:18:34 i think i'm +1 17:18:39 the fact that we'll be mixing with cockpit and Atomic CLI etc. is a feature, not a bug 17:18:45 someone want to lay out a proposal? 17:18:46 miabbott: +1 17:18:50 I get it now but any new user that is using Fedora Cloud images is going to be hella confused if they join 17:18:51 I just want to make sure people will be okay with having every distro using the same channel to coordinate their atomic stuff. 17:19:02 jberkus: +1 17:19:07 jdoss: #fedora-cloud will still be there 17:19:10 for the cloud base image 17:19:28 sweet! 17:19:30 here is a question 17:19:38 walters: - if the endless os guys want to talk about ostree stuff 17:19:42 where would that happen? 17:19:44 #ostree 17:19:46 we chat today in #ostree 17:19:51 ok 17:20:00 * dustymabe adds that to his list of channels 17:20:19 yeah - it's all a balance 17:20:30 I'd also like to discuss the tradeoffs of using atomic-devel@projectatomic.io instead 17:20:53 jberkus: i think our list is way too chatty for that 17:21:10 dustymabe: which list? 17:21:18 cloud@lists.fp.o 17:21:26 which would become atomic@lists.fp.o 17:21:32 well, the only part that's chatty is the automated notices 17:21:40 dustymabe: and is there really a lot of conversation on that list that isn't germane to atomic-devel? 17:21:42 otherwise, the cloud@ list is rarely used 17:21:42 but that's where the discussion is happening 17:21:58 jzb: there really isn't 17:22:18 Fedora Atomic discusses things in two locations: this channel, and Pagure issues 17:22:18 * jzb will get the same amount of traffic either way 17:22:26 i'm just saying that the mails from our discussions in our issue tracker would dominate that mailing list 17:22:27 I like more atomic alignment 17:22:37 Does the issue tracker go to the list? 17:22:42 jzb: i'm not worried about people who are subscribed to both lists 17:22:50 If the projectatomic.io site wasn't so unquestionably not a Fedora-focused page I could see that. 17:22:51 jbrooks: I think so? 17:22:57 * stefw notes for reference that we talk about lots of distro specific and packaging Cockpit stuff in #cockpit. And Cockpit is also branded, customized, and heavily tested, and delivered per distro 17:22:58 jberkus: yes 17:22:59 jbrooks: yes 17:23:10 * gholms ponders 17:23:17 what about bringing Project Atomic under Fedora? 17:23:27 officially 17:23:39 That'd fix that. 17:24:05 centos atomic is still a thing. So is Red Hat Atomic 17:24:25 jberkus, but making one the upstream has real merit 17:24:31 yeah honestly I don't really care about all that stuff (who is under what umbrella etc) 17:24:32 jberkus: both downstreams (more or less) of Fedora Atomic 17:24:35 Yeah, but those aren't relevant to Fedora. 17:24:54 however, realistically, there aren't many members of any of the associated atomic projects who aren't also on this team 17:25:02 * dustymabe notes it's interesting this topic sparked this long of a discussion 17:25:08 and it's *way* easier for new users to have Just One List 17:25:14 jberkus: i would say that's not the case 17:25:15 I don't see the value in folding project atomic into fedora, but I see the value in using the project atomic ml for us 17:25:18 dustymabe: of course, naming discussions always do ;-) Also green. 17:25:18 any new outside users that might help out with stuff might be confused on what project is what 17:25:23 since we are not part of this team 17:25:31 so we often don't have container team members attend our meeting 17:25:46 also this is the first time we've had someone from cockpit AFAIK 17:26:11 which is ok - just saying it has traditionally been more focused on "host" 17:26:14 just as a side note, if Project Atomic claimed to be moved under the Fedora umbrella ... would there need to be a formal proposal to Fedora Council as well as some group of Project Atomic folks? (I don't know the project's governing structure) 17:26:26 etc etc etc yeah 17:26:35 don't see the value in all that 17:26:41 maxamillion: yes, there would 17:26:46 rgr 17:26:50 ok let me try to summarize 17:27:03 the prd stuff was painful enough ;) 17:27:22 - 1 - some concern over creating new lists/places - why not use atomic-devel mailing list and #atomic irc channel 17:27:51 - 2 - mostly because there are things that are very fedora specific happening in #fedora-cloud and on the cloud mailing list and we don't want to flood the channel 17:28:30 - 3 - atomic-devel is meant for all project atomic projects and we could cause too much chatter there 17:28:43 so that's the gist, from what I gather? 17:28:47 There's no need to fwd all issue tracker items to the list, though 17:28:51 4 - chatter is really just automated notices, so we could handle those separately 17:28:53 And who's saying #3? 17:29:13 jbrooks: I guess me for #3 17:29:35 https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2017-June/thread.html 17:29:37 I would like to keep the pagure emails going to a list 17:29:42 not a ton on there 17:30:15 jbrooks: yeah - we would probably double or triple the traffic 17:30:19 I've got one: it would be the only Fedora product that doesn't use Fedora lists/channels/etc. 17:30:37 gholms: +1 17:30:50 Yeah, fine, we can just do new list / irc 17:30:55 anyone want to talk about notifications going to a list? 17:30:57 I'm going to get them both anyway 17:31:06 I think it's excessive 17:31:13 I've been thinking about filtering some of it 17:31:14 which is or isn't a real concern, I don't really know ... Atomic has broken a lot of the mold already, but it's a fair point to make note of 17:31:16 i like them going to a list because you get a global view of what everyone is seeing 17:31:18 dustymabe: I was going to ask that we kick them out of the cloud list anyway. 17:31:34 it doesn't depend on each persons individual settings 17:31:38 The notifications 17:31:41 if they are on the list, they are seeing the emails 17:31:50 gholms: yes, when we move they would move with it 17:31:54 I figure, if I want to be subbed to an issue, I'll sub myself 17:32:02 gholms: note that the cloud-sig repo will still send notificatiosn to the list 17:32:33 dustymabe: Well, I was going to ask that it not anyway. ;) 17:32:47 another reason to send notifs to the list is because that is how we communicate some changes (i know we should send formal emails more often, but it doesn't always happen) 17:32:47 So I'm +1 to a new fedora-atomic irc and ML 17:33:10 There is so much pagure mail on the list that I just ignore it. 17:33:25 yep 17:33:32 gholms: right - but you are more interested in cloud base image aren't you? 17:33:44 same, I filter all pagure email out and just ignore it ... I'm subscribed to like 20 repos, the noise is insane 17:34:00 maxamillion: i just filter the lists to separate folders 17:34:05 I'm -1 on a new IRC channel, but willing to discuss list particulars 17:34:07 and then run through them 17:34:35 jberkus, Ah, yeah, that'd be something, a split 17:34:41 dustymabe: I'm not sure what that has to do with the mailing list. 17:34:52 Really, I'd love to make #atomic my main hangout for fah and cah 17:35:01 gholms: in other words our atomic tracker has been sending emails to the cloud list 17:35:04 so sure, you don't care about them 17:35:11 materially, many devs use the two channels interchangeable 17:35:13 so that is probably why you ignore them 17:35:19 so they might as well be one channel 17:36:06 ok i think we've talked about this enough and no real concensus at this point 17:36:13 Heh 17:36:16 please add concerns/thoughts to the ticket 17:36:18 continue discussion on ticket 17:36:22 and vote if you will 17:36:31 add proposals etc.. 17:37:19 #info lots of good discussion here - no real concensus (spelling?) - continue discussion in ticket https://pagure.io/atomic-wg/issue/295 17:37:28 #topic Atomic Host images omit many common locales that all other flavors include 17:37:34 #link https://pagure.io/atomic-wg/issue/282 17:37:51 so jlebon worked on some items around this ticket 17:38:10 which should address adam's most pressing concerns 17:38:26 He'll continue to update the ticket as things go 17:38:41 #info jlebon worked on some items around this ticket which should address adam's most pressing concerns 17:39:07 #topic Delivering Atomic Host via CI/CD pipeline 17:39:12 #link https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/message/ZS7XY7NHXGGOPOB2YKNQUWSDUGCMYIL5/ 17:39:18 #link https://fedoraproject.org/wiki/Objectives/Continuous_Integration_and_Delivery_of_Fedora_Atomic_Host 17:39:22 stefw: take it away 17:39:24 so the basic elevator pitch: 17:39:47 We compose atomic host artifacts exactly as run by users 17:39:52 test those artifacts during integration testing 17:40:07 provide feedback to packagers, and optionally gate packagers, gate inclusion in the host based on that testing 17:40:19 and then deliver exactly those bits (qcow2, ostree, ami) as tested 17:40:45 This would be an Objective for Fedora 17:41:14 it will benefit the quality of Fedora overall, by gating broken package changes, and providing feedback from testing directly to those making changes to packages. 17:41:21 but it's focus and scope is specifically on Atomic Host 17:41:32 I'm in 17:41:34 Atomic Host is based on technology (OSTree) that enables this 17:41:36 stefw: one question - from a packager perspective - this would integrate with bodhi? 17:41:41 but we never went the full way to actually pull this off 17:41:41 sounds awesome 17:41:49 dustymabe, I believe it would 17:41:53 i believe there would be two levels of gating 17:42:04 one at the package level ... where a packager receives feedback to say the package is broken for Atomic Host 17:42:07 so in this, do i still have to beg for karma in bodhi too? 17:42:11 e.g. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ad3832d881 17:42:14 Nice 17:42:14 walters, i don't think so 17:42:29 as far as process goes bodhi is the most broken IMO 17:42:31 a packager receives feedback ... that a broken change (ie: NVR) is not going to get into Atomic Host 17:42:43 pingou has done work to make bodhi approval automatic based on integration testing 17:42:47 that is not functional yet 17:42:50 but it would go into buildroots/workstation? 17:43:03 walters, i believe the solution there is related to modularity 17:43:11 but we may choose another path 17:43:13 basically the concept that 17:43:24 a specific set of packages (NVRs) known to be good go into an Atomic Host compose 17:43:33 some of those packages are shared by other buildroots, and general delivery of Fedora 17:43:36 or other modules 17:43:37 stefw: we'll need to improve our testing 17:43:43 Interesting idea 17:43:46 jberkus, certainly 17:43:52 so porting tests to support this effort is underway 17:43:59 is roshi around? 17:44:10 Early this year Fedora approved the concept of placing tests in dist-git 17:44:11 jberkus: I pinged him earlier, he may be AFK 17:44:18 to be changed along with packages and/or modules (for higher level tests) 17:44:27 stefw: one thing we'd really need is end-to-end testing for either FAO or Fedora+Kube, ideally both 17:44:30 and there is active movement to create or port such tests 17:44:41 jberkus, the current PAPR tests are a great place to start for that 17:44:55 staging Pagure instance for tests ready to go into dist-git 17:44:56 https://upstreamfirst.fedorainfracloud.org/ 17:45:09 jberkus: yeah - i think system level tests for atomic host + openshift and/or kube are important 17:45:11 pingou and threebean puiterwijk are working on getting Pagure in front of dist-git 17:45:25 dustymabe, indeed, and they fit really well in the concept of a module in dist-git that defines Atomic Host 17:45:25 +1 for pagure on dist-git - can't wait 17:45:37 that module which defines Atomic Host, would contain broader Atomic Host specific tests 17:45:42 stefw: a few more questions 17:45:44 so the story is rather consistent end to end 17:45:49 stefw: We are trying to make Fedora Atomic Host available for aarch64 and ppc64le as well along with x86_64 from F27 onwards. I am hoping that these arches will get benifited too with Fedora Atomic CI 17:46:04 ksinny: +1 17:46:05 is anyone working on this now (i.e. the fedora build process/tooling) and how atomic host is composed 17:46:14 and how do we start working with them 17:46:23 dustymabe, if you look at the Objective there are links to some initial work done 17:46:26 * stefw gets a link 17:46:37 whoops it's on this landing page 17:46:43 https://fedoraproject.org/wiki/CI 17:47:14 in addition on that landing page there are links to more information about test layout and test invocation 17:47:20 aswell as links to the pipeline 17:47:39 lastly Fedora has put together an outline for how a CI/CD system and pipeline would interact with Fedora: https://fedoraproject.org/wiki/Fedora_requirements_for_CI_and_CD 17:47:45 #info some people have been looking at fedora build process/tooling for fedora atomic and have some initial work links from https://fedoraproject.org/wiki/CI 17:48:02 ksinny, the multi arch question is a good one 17:48:03 it's two fold 17:48:08 first of all when you run CI at scale 17:48:24 it's pretty clear that you get 95% of the way (or more) by running continuous integration testing on one or two architectures 17:48:34 usually those are very different architectures such as 64-bit x86_64 and 32-bit i386 17:48:47 it is often the case that you make up for the other architectures during the delivery phase 17:49:03 stefw: I assume that CI work will use CentOS infra. If yes, we are working on adding ppc64le hardware at least for the start 17:49:07 but yes, these are aspects that that can change over time 17:49:28 ksinny, yes, that's good news and will be a big help 17:49:43 stefw: good to know :) 17:49:49 but my point is that it's possible not to have literally everything duplicated, and still have an effective CI system 17:50:01 a CI system will run thousands of composes, many will fail in testing, and be thrown away 17:50:14 stefw: I know you mentioned pingou, is there anyone else from the fedora community we can collaborate with 17:50:15 it's possible to do that in an intelligent way, by maxing out x86_64 hardware, before composing elsewhere 17:50:29 there's a mailing list here: ci@lists.fedoraproject.org 17:50:36 in addition, there are people like: 17:50:51 merlin mathesius shepherding the porting of tests for core packages 17:50:55 #info there is a mailing list for the ci efforts in Fedora, that will initially focus on the set of packages in atomic host: ci@lists.fedoraproject.org 17:51:15 tim flink running the staging area for tests being ported from Red Hat 17:51:35 does the group associated with that list have a pagure issue tracker or irc channel? 17:51:38 stefw: is there a IRC channel where you do the discussions? 17:51:45 there is an IRC channel #fedora-ci 17:51:56 there is no pagure issue tracker yet 17:52:14 i'm wondering if using the Fedora Atomic/Cloud issue tracker would be a good thing for now? 17:52:26 given that this effort is very much focused on Atomic Host for the time being 17:52:33 stefw: i thik it depends on how chatty it is 17:52:45 indeed. 17:52:54 would be cool if we could configure notifications individually based on labels 17:53:03 i imagine it will start off steadily and as CI in Fedora goes beyond Atomic Host, we could probably outgrow the Atomic Host specific tracker 17:53:19 we can start there and then grow 17:53:50 starting there might be a good way to raise awareness too 17:54:04 cool ok, anything you need from the working group for now? 17:54:06 So i'd like to raise this an a Fedora Objective 17:54:08 action items, etc? 17:54:28 And I can't just do that on my own 17:54:30 also, anyone have any concerns about an effort like this? 17:54:37 so I'd like some general feeling on the concerns around this effort 17:54:44 and who would back it, who needs more time to think 17:54:45 etc. 17:54:45 stefw: jinx 17:54:50 :D 17:55:21 I'm in, I love the idea ... I'm concerned about some of the implementation details, it seems like a lot of work and it touches a lot of moving parts but ultimately I'm in 17:55:31 stefw: honestly I think the idea as a whole sounds good - but we don't have much to go on as far as specifics go 17:55:47 i think most of the people here agree more testing would be lovely :) 17:55:50 maxamillion, indeed, you're absolutely right, touches a lot of things, requires some change of mindset too, tough stuff 17:55:59 if i can stop manually editing specfiles and manually initializing kerberos to tell koji i did a git push and then manually telling bodhi i built an rpm...i am really excited 17:55:59 there is a lot to digest in the wiki pages that stefw linked 17:56:05 stefw: indeed 17:56:06 if I have to do that *and* watch something else, that kind of sucks 17:56:06 and this will take a lot of collaboration with releng/infra 17:56:17 dustymabe, indeed, i left out many of the imagined or prototyped details 17:56:25 miabbott: +1 - I had already read it, I saw mention of it on th emailing list 17:56:39 because I didn't want it to seem like this was all already decided ... therhe are lots of ideas about how things would work 17:56:56 should I add an ideas section to the Objective? things that are not set in stone, but help paint a fuller picture? 17:57:20 stefw: sure, sounds great 17:57:35 stefw: thanks for bringing this to the group 17:57:36 dustymabe: note that members of both fedora and centos infra are involved in the CI Objective 17:57:51 maxamillion: cool 17:58:16