17:00:26 #startmeeting Fedora Atomic Working Group 17:00:26 Meeting started Wed Nov 30 17:00:26 2016 UTC. The chair is kushal. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:26 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:00:26 The meeting name has been set to 'fedora_atomic_working_group' 17:00:32 #topic Roll Call 17:00:39 .hellomynameis kushal 17:00:41 kushal: kushal 'Kushal Das' 17:01:18 * coremodule is here. 17:01:24 .hello trishnag 17:01:25 trishnag: trishnag 'Trishna Guha' 17:01:26 .hello maxamillion 17:01:28 maxamillion: maxamillion 'Adam Miller' 17:01:33 .hello dustymabe 17:01:35 dustymabe: dustymabe 'Dusty Mabe' 17:01:39 .hello coremodule 17:01:40 coremodule: coremodule 'Geoffrey Marr' 17:02:26 #chair coremodule trishnag maxamillion dustymabe jberkus 17:02:26 Current chairs: coremodule dustymabe jberkus kushal maxamillion trishnag 17:02:43 .fas jasonbrooks 17:02:44 jbrooks: jasonbrooks 'Jason Brooks' 17:02:47 .fas rtnpro 17:02:48 rtnpro: rtnpro 'Ratnadeep Debnath' 17:03:04 #chair jbrooks rtnpro 17:03:04 Current chairs: coremodule dustymabe jberkus jbrooks kushal maxamillion rtnpro trishnag 17:03:20 .fas jberkus 17:03:21 jberkus: jberkus 'Josh Berkus' 17:04:52 Okay 17:05:18 Should we discuss about the F26 change proposal ideas today, or go to standard tickets? 17:05:31 .hello vvaldez 17:05:32 vvaldez: vvaldez 'Vinny Valdez' 17:05:39 #chair vvaldez 17:05:39 Current chairs: coremodule dustymabe jberkus jbrooks kushal maxamillion rtnpro trishnag vvaldez 17:06:50 Is this just me or people are actually silent? 17:07:02 i'm not sure if it's a change but I plan to keep working on openshift-related items, mostly the installer, system containers 17:07:07 .hello bowlofeggs 17:07:07 well, people are still talking on fedora-cloud 17:07:09 bowlofeggs: bowlofeggs 'Randy Barlow' 17:07:31 and rpm-ostree work, things like https://bugzilla.redhat.com/show_bug.cgi?id=1393545 17:07:57 walters: is there any reason why an Openshift install just using layering wouldn't work? 17:08:21 #topic Fedora 26 change proposal ideas discussion 17:08:27 walters, thanks, I will start a new wiki page, and share it in the list. Can you please add these ideas there? Just to keep track. 17:08:45 jberkus, none, but i think long term, we want to decouple: host, platform, and app updates 17:09:02 walters: no question 17:09:13 walters, I'd love a how to make a system container guide 17:09:15 or sometihng 17:09:15 people running openshift-via-yum already hit the problem that you really don't want `yum update` to potentially update your paas 17:09:19 on the other hand, there's getting something running *right now* too 17:09:28 how the config.json.template and stuff works 17:09:46 jbrooks, yeah, i think work on improving system containers is all part of this 17:09:52 jbrooks: that's a runc thing. I'll admit that docs on it are kinda poor 17:09:54 anyways i'm not sure how this maps into a Change 17:10:01 Knowing how to make one is blocking me on that 17:10:09 With sayan I want to work on a change to rewrite Fedimg, and add power to run the same tests though autocloud in AWS. 17:10:13 No, it's just open mic 17:10:20 back to topic 17:10:28 This will allow us to block any non-working AMI(s) in the open. 17:10:57 do we still have tests we're ignoring? 17:11:01 kushal, +1 to making a f26 changes wiki page or issue for collection/discussion 17:11:12 jberkus, means? 17:11:15 yes, kushal, can you action that? 17:11:40 we had some tests we disabled for releases a couple months ago 17:11:40 #action kushal to create the change wiki page for 26 for collection of ideas 17:11:55 jberkus, not disabled, but marked them non-gating 17:12:31 Which are passing now. 17:12:37 ok 17:12:43 We can mark them as gating test again. 17:12:53 I will have to open the ticket for doing so. 17:13:20 #action kushal to reopen the ticket for marking the non-gating tests to gating (change was made during F25 cycle). 17:13:52 jbrooks, btw, as dustymabe suggested in the last meeting, I have started the discussion in the mailing list. 17:13:59 Just that not much replies to that. 17:14:07 Even I will have to reply with my ideas. 17:14:38 ack, I'll reply there 17:15:01 The second idea is about providing a fedora base image for rkt (like we release the docker base image) from official Fedora build systems. Any comments on that? 17:15:06 one thing i want to mention too is https://pagure.io/releng/issue/6545 17:15:24 walters, checkign 17:15:27 * checking 17:16:07 kushal: I would nix that idea 17:16:21 * walters unfortunately has to head out, back later, but will read scrollback 17:16:32 I think we need to walk before we run. i.e. let's do what we already set out to do well before we add new things 17:16:41 walters: thanks for joining 17:16:41 does that mean we'd also offer rkt support on AH? 17:17:25 jberkus, nope if I understand correctly, but our users can run rkt on proper fedora base images. 17:17:31 which is something I would want. 17:17:40 walters, thanks :) 17:17:50 kushal: yeah, here is where things start to get confusing 17:17:57 because we are the Atomic WG now 17:18:17 so if not for Atomic Host, what is the point? 17:18:51 kushal: if we're going to support rkt, we should support it 17:18:53 dustymabe, To keep it ready when we say we will support rocket on Atomic Host (if ever). 17:19:07 if we don't have rkt on fedora, then who woudl use the base images? 17:19:17 What's required to run rkt? 17:19:23 We could install via pkg layering 17:19:26 as an option 17:19:31 jbrooks: afaik, some binaires 17:19:33 jberkus: we have rkt in Fedora, just not in Atomic Host 17:19:44 jberkus, we have rkt 17:19:51 package layering lets us leave that to the user 17:20:04 or if we can pkg it into a system container 17:20:32 I would like to note that adding rkt building to OSBS would be a major development effort and is not currently on the roadmap anywhere 17:20:38 jbrooks, still for anything, if we want to run any fedora based on top of it, we will require a fedora base image for the same. 17:20:44 so, aside from the work involved, what's the reason to NOT have rkt available for AH? 17:20:46 Before we can talk about supporting this, do we even have a poc? 17:20:57 maxamillion: congrats, btw 17:21:00 Has someone done a writeup or something 17:21:11 maxamillion, yes, that is why I am talking about only generating the base image as the artifact. 17:21:20 jbrooks: not afaik. I was under the impression that asking about rkt was political 17:21:31 I don't think so 17:21:35 I'm not against enabling rkt, but it's not something we'll be able to build as official Fedora deliverables for some time 17:21:38 jberkus: thanks :D 17:22:32 yeah I'm against efforts to enable rkt until we are comfortable with our current workload 17:22:41 Well, whoever is caring about rkt can start by taking an action to write up something about it? 17:22:53 unless we have an influx of community members who want to donate the time/effort 17:22:57 jbrooks, Okay, I will take that on my head. 17:23:01 Cool 17:23:08 dustymabe, I am volunteering :) 17:24:17 Any other topic/ideas for F26 timeframe? 17:24:23 kushal: so you're comfortable with doing rkt in addition to all of the other work that needs to be done in this WG? 17:24:36 kushal: we have a card with some info about what would be required in tagia for the releng tools team 17:24:39 kushal: it's *old* 17:24:56 dustymabe, Looking at my personal workload, yes. 17:24:59 I hate to say it but this WG has a ton of technical debt 17:25:14 dustymabe: really? no! 17:25:32 dustymabe, For that may we can start writing them down, and then eliminate them one by one. 17:25:44 kushal: we literally almost missed F25 release because no one was looking at the ISO images 17:25:53 dustymabe: there's a lot of people with a lot of great ideas but little time to actually solve problems or write code (myself included) 17:25:58 do you not think there is a serious problem there that needs to be resolved 17:26:08 dustymabe, and yet we released much more promptly than last time 17:26:13 We're making progress 17:26:40 yes we are making progress. but literally again it was up until the last few days before we had something 17:26:40 dustymabe, We keep missing builds also same way, and generally it is you who point it everytime :) 17:26:46 speaking of which ... is the f25 base image out yet? 17:27:13 dustymabe: it was the night before when the release script go rewritten (well, half rewritten) because I didn't know that things changed out from under it 17:27:17 Ah, that's something to discuss -- are we pawning the base image off to server wg for 26? 17:27:18 :) 17:27:28 jberkus: "f25 base image" ... you mean docker? 17:27:35 jbrooks, that is a million dollars question. 17:27:46 the docker image is out 17:27:53 I'm just saying. lets grease our machine and get better at what we are currently doing 17:27:54 yeah 17:28:05 jberkus: yes, I released it on f25 GA day 17:28:16 if anyone thinks they have ample free time, let's chat 17:28:17 a little late in the day because things got crazy, but it's there 17:28:28 maxamillion: hmm. docker hub was still not finding it the day after. something wrong with hub? 17:28:39 dustymabe, I will have a chat with you. 17:28:41 :) 17:29:22 kushal: also if we are doing so well as a WG then why did we have no answer for kubernetes on release day for F25 17:29:36 we have a lot of work to do :) 17:30:08 dustymabe, I never said we are doing so well, in my blog post I mentioned about few things went really good. 17:30:17 we do have an answer, just no docs 17:30:40 jberkus, That is a common problem without any answer still now. 17:30:46 We have some answers, and some questions 17:30:52 We need people who can write. 17:31:04 I don't think that is the problem. we might have an answer now. 17:31:04 For instance, we need containerized kube, but we have no official fedora containres 17:31:05 well, I'm rewriting the multi-node getting started guide 17:31:16 but we should have actually had an answer when the change went in 17:31:18 dustymabe, not having enough docs is a big problem for us. 17:31:21 to remove k8s 17:31:31 jbrooks: we will have official fedora containers in about a week and a half 17:31:33 way before f25 release 17:31:34 Even then our kube is ancient 17:31:47 jbrooks: that is another problem, yes 17:31:54 maxamillion, awesome -- where can I submit dockerfiles for those? 17:31:57 oh, yeah, the container images 17:32:04 jbrooks, yeah, suggesting to run ancient kube on a modern os does not make that happy users :) 17:32:16 ok, i'll stop ranting now, in short. we have a lot of work to do for f26 17:32:25 jberkus: DistGit, it will be managed just like packages via pkgdb ... it's a namespace in distgit already 17:32:28 and I look forward to making things better 17:32:33 my plan is to write docs which use the official 1.4.5 images 17:32:35 jbrooks: errr r... tab fail ^^^ 17:32:38 dustymabe, don't stop, you are really bringing out the pain points. 17:32:48 maxamillion, so that's via email? I've never grokked how one collabs w/ dist-git 17:32:50 This is useful, as we can discuss these now. 17:32:57 until we have *current* fedora-based images 17:33:01 kushal: :) - we just have to be much more enganged 17:33:05 jberkus: are you a fedora package maintainer? 17:33:05 and we will be 17:33:11 maxamillion: nope 17:33:13 god damn irssi 17:33:17 me neither 17:33:19 oh 17:33:21 maxamillion, :D 17:33:21 But I'd like to become one 17:33:27 * misc can sponsor 17:33:34 well it's the exact same workflow as being a fedora package maintainer (which we did on purpose) 17:33:45 I'll ping you, misc 17:33:46 misc: :) 17:33:48 thanks 17:34:05 jbrooks++ misc++ 17:34:05 dustymabe: Karma for jasonbrooks changed to 1 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:34:08 dustymabe: Karma for misc changed to 1 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:34:18 maxamillion: well, there's multiple steps involved. first we need RPMs for current kubernetes, *then* we can build contianers 17:34:20 dustymabe, okay, after this discussion settles down, I will put topic as suggestion to improve the wg. Please suggest then. 17:34:22 always happy to increase my army of mini^W^W^W^W^W help others to be happy in Fedora land 17:34:22 jbrooks: basically 'fedpkg clone docker/NAME_OF_MY_CONTAINER' and then 'vim Dockerfile' -> 'fedpkg commit -m "made awesome changes"' -> 'fedpkg push' -> 'fedpkg container-build' 17:34:39 ack 17:34:49 we could use karma on the kube pkg: https://bodhi.fedoraproject.org/updates/FEDORA-2016-fffea4b1c3 17:35:04 jbrooks: I've been meaning to get there 17:35:06 :( 17:35:08 simple testing directions: https://gist.github.com/jasonbrooks/f1aa092e63edce5272451c5845f72750 17:36:41 ok - so didn't mean to derail the discussion 17:36:42 jbrooks, thanks, will do that tomorrow morning my time. 17:36:44 just one more vote on that and it passes 17:37:14 Can we now go to the topic of improving the working group? 17:37:25 so do we have any f26 changes that will improve our current workflow and/or make our entire process better? 17:37:50 I think I have one 17:38:03 which is something colin has wanted for a while 17:38:04 dustymabe, tell us more :) 17:38:32 basically ostree is the actual release artifact of the Atomic WG 17:38:45 and that actually gets built and released every night when bodhi runs 17:39:04 we'd like for the release of the ostree to coincide with the release of the images 17:39:24 dustymabe, release of ostree means? 17:39:39 taking it out of bodhi's hands seems simplest 17:39:53 kushal: i.e. a new commit gets written to an ostree repo and the ref gets updated to point to that commit 17:40:09 we have a draft of a proposal for doing this 17:40:18 i'll update the list with this information 17:40:27 dustymabe, okay, but I thought that has to be done before we can generate the atomic images. 17:40:31 I must be wrong. 17:40:32 i'm also planning to reach out to releng to talk with them about this change 17:41:09 dustymabe, oooh, sort of related -- we don't have any updates-testing ref yet for 25 17:41:12 kushal: right it does need to be done, but there is a subtlety 17:41:17 .fas linuxmodder 17:41:18 linuxmodder: linuxmodder 'Corey W Sheldon' 17:41:20 (late) 17:41:22 jbrooks: yeah, that is something else that I want 17:41:27 We need that 17:41:36 I think I'm going to try to add that myself (soon after I learn more about the process) 17:41:47 I'm going to use it as a learning experience if that is ok 17:42:02 linuxmodder: welcome! 17:42:03 sounds good 17:42:58 kushal: I'll write to the list with this information 17:43:14 dustymabe, Thanks :) 17:44:04 np 17:45:01 Now any ideas about how to improve the working group? 17:45:21 dustymabe, btw, feel free to add an action item :) 17:46:07 #action dustymabe to try to meet with releng to learn more about ostree composes and ways to change things 17:47:19 kushal: ideas on how to improve the WG? 17:47:21 Did we miss any other actionable item? 17:47:41 kushal: if others could it would be good to test/karma that k8s update 17:47:54 kushal, you or linuxmodder or anyone else 17:48:04 if anyone agrees then we can action that 17:48:06 dustymabe, improve == like you mentioned about a lot of technical debt. 17:48:36 #action kushal to test https://bodhi.fedoraproject.org/updates/FEDORA-2016-fffea4b1c3 after he wakes up 17:48:39 dustymabe, done :) 17:49:22 dustymabe, sorry got pulled away what now 17:49:46 kushal: I guess what I mean is that for the most part this group is disjoint and not very well organized. half the time we don't know when things fail to build or even where to look 17:50:09 dustymabe, ah re: k8s update karma ? 17:50:22 dustymabe, yes, that is mostly not from issues related to working group. 17:50:23 even if we do identify a problem. we open a ticket and don't follow up 17:50:29 We are dependent on many other teams. 17:50:44 kushal: right, that is true 17:51:03 I just think we need to grasp those dependencies better and manage "what is coming down the pipe": 17:51:13 so if you have suggestions on how we do that better 17:51:24 then that would definitely be something we can improve 17:51:27 i see dustymabe 's point tho it has become as if we are going the way of hop scotch with the WG on communication and tech debt 17:51:56 dustymabe, For example rel-eng is started to have some documentation done, that will help us to understand many points. 17:52:09 linuxmodder, ? 17:52:31 linuxmodder, I have no clue of what you just now said. 17:52:35 kushal: yeah. I think we need to be more proactive though 17:52:38 Sorry, not a native English speaker. 17:52:42 dustymabe, True. 17:52:48 i.e. if we really want documentation, then we need to help them make it 17:53:12 we'll either annoy them enough (in a good way) that they'll be sure to get it written, or we'll learn enough to write it ourselves 17:53:22 dustymabe, we can make our own docs (and we are not doing that properly), I don't know how to help other teams. 17:53:24 even tho to the avg user atomic can seem abstract the lack of docs and a visible core of users to go to makes things worse for the WG imo 17:53:24 and contribute back 17:53:48 linuxmodder: agreed. I think this will all get better over time 17:54:02 we have had people write docs in the past, but then no one aggregating them and curating them 17:54:25 I don't even want to write docs because they would just poof away because we don't have someone to do that 17:54:27 as a member of -dcos myself I'm down for helpign with edits and getting any docs we do get written out there 17:54:28 dustymabe: also a poitn of note, right now there's a major knowledge transfer happening from dgilmore to mboddu and mboddu is being kind enough to write up docs on everything (he's the newest member of the Fedora RelEng group and will be taking over for dgilmore on a lot of things so that dgilmore can work on other items) 17:55:03 maxamillion: agree.. TBH they should make that a series of meetings and recommend everyone in fedora attend 17:55:04 dustymabe, not offering to do the initial but more than willing to maintain them with possibly one other contrib once started up 17:55:26 linuxmodder: thanks! 17:55:44 linuxmodder, problem is about new content, we can maintain if we have actual content. 17:55:58 i honestly think we have plenty of casual contributors that would make very good contributions if we have a group of people to help them 17:56:17 kushal: I'm going to have some new content very soon 17:56:20 once we get a baseline draft would anyone object to doing a monthly end user poll of sorts of what they want / need in that to steer the docs 17:56:31 dustymabe, please send in PR to https://github.com/fedora-cloud/fedoracloud for the same :) 17:56:57 linuxmodder, jsmith volunteered to help to fix any docs get submitted (from the docs team side). 17:57:10 linuxmodder, But our problem is that we could not generate enough content. 17:57:20 dustymabe: also for reference, docs are all now here https://docs.pagure.org/releng/ and they are sphinx-doc reStructuredText in the releng pagure repo -> https://pagure.io/releng/blob/master/f/docs 17:57:25 jbrooks, this reminds me, we need our k8s related docs there. 17:57:26 kushal, in any sub area or overall 17:57:35 linuxmodder, overall 17:57:47 can someone send an email to our list about "where to send docs?" 17:57:54 dustymabe, I will 17:57:58 or even better yet, make a wiki page that we can link to? 17:58:07 is there any present draft linkable anywhere atm even if its a git doc 17:58:09 dustymabe: there's a new policy that if you change code, it needs docs updates too or else the PR won't be merged (when applicable) 17:58:10 #action kushal to send reminder to list about where to send the docs. 17:58:15 kushal: also link to those releng resources that maxamillion just referenced 17:58:20 linuxmodder, I just now shared a link 17:58:21 kushal, OK 17:58:22 dustymabe, Yes. 17:58:34 maxamillion, awesome, finally 17:58:38 lagging some sorry 17:58:45 see it now 17:58:47 #action kushal to create wiki page on documentation for atomic WG and send mail to the list 17:59:03 maxamillion, All good projects, like Qt, Mariadb, Django has the same rule iirc 17:59:08 linuxmodder: what's your github ID? I'll have some docs for you to review soon 17:59:25 jberkus: and where are your docs going? 17:59:30 jberkus, linux-modder 17:59:30 projectatomic.io? 17:59:39 dustymabe, btw, what to put in that wiki page for docs? 17:59:40 projectatomic.io; we dont have another place for them AFAIK 17:59:45 we should consider joining our resources 17:59:51 kushal: ^^ 17:59:59 dustymabe, I was thinking to add the docs links to the the page. Like one place to read all the docs. 18:00:00 kushal: +1 18:00:21 jberkus, linux-modder ? 18:00:26 dustymabe, what's wrong with keeping on pa.io and linking to pagure for upstream work on docs etc? 18:00:27 dustymabe, Yup, or at least add a copy in our repo too 18:00:56 kushal, +1 on the repo copy 18:01:07 linuxmodder, because right now there are way too many places to look for, also people do not know if things are updated or not. 18:01:12 linuxmodder: pa.io is fine. I'm thinking we should actually just use pa.io 18:01:21 kushal, I think our wg pagure is a good place, actually 18:01:26 for pa.io, I'd love to move them out to a separate repo for RTD eventually, but don't currently have the technical setup for it 18:01:37 RTD? 18:01:38 Ok 18:01:42 readthedocs 18:01:47 oh got ya 18:01:48 Oh, sorry, I'm confused 18:01:51 or more specifically, docs.projectatomic.io 18:01:53 jberkus, rtfd :p 18:02:11 Heh, let's just write them wherever and figure out where to put them from there 18:02:12 kushal: do you have experience doing that? 18:02:17 could you help jberkus ? 18:02:18 jbrooks, it is just a git repo of sphinx docs, we can put it inside our pagure. 18:02:45 kushal, right, I thought this was about that github repo vs our pagure repo 18:02:46 kushal: that's not the issue. we have a stalled project to composite-build docs for all the PA subprojects 18:02:52 dustymabe, yes, and I know the rtfd devels too, so if required we can ask for help. 18:02:58 who is maintainign pa.io atm in the WG ? would it be hard to do a webhook or similiar for the repos to update on pa.io 18:03:05 based on pulling from a bunch of different repos 18:03:26 jberkus, I can help you to find volunteers if you want. 18:03:50 in my last ? more like docs.pagure.io/ regularly updating to docs.pa.io via commit or cron 18:03:54 Okay, people we are 3 minute overtime, but I am so happy that we are talking about documentation for such a long time. 18:03:54 that would make integration with, for example, paguire docs much easier 18:04:19 jberkus, pagure has power to do rtfd.io 18:04:28 jberkus, so we are covered from that side :) 18:04:43 kushal, and easily link back to pa.io ? 18:04:55 linuxmodder, what do you mean? 18:05:22 linuxmodder, links are standard rst link, one has to write them manually. 18:05:23 where docs.pa.io or similar auto updates to the most current `stable` copy on rtfd.io 18:05:48 like a gh.io page does from a gh repo 18:06:09 yes, that is why people use readthedocs in the first place. 18:06:40 We push to our git repos say github or pagure. All our docs automatically gets rebuild 18:06:48 Anyway, I am going to end the meeting now. 18:07:07 I am planning to write a blog post from this discussion, is that okay for everyone? 18:07:19 +1 18:07:22 +1 18:07:27 +1 18:07:36 Like with the details of different pain points as mentioned by dustymabe. 18:07:43 * vvaldez is trying to dual-meeting, apologies 18:07:56 vvaldez: thanks for being here 18:07:57 kushal, consider throwing on a planet feed too ( or with perm I will repost via mine) 18:08:05 doesn't someone else have this channel now? maybe we should move to fedora-cloud? 18:08:07 +1 18:08:17 only dual ? whimp :) 18:08:22 ;) 18:08:22 linuxmodder, I am on the Fedora planet from the time it came up I think. 18:08:46 #endmeeting