15:02:08 #startmeeting Server Working Group Weekly Meeting (2014-03-11) 15:02:08 Meeting started Tue Mar 11 15:02:08 2014 UTC. The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:02:08 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:02:16 #chair sgallagh mizmo nirik davidstrauss Evolution adamw simo tuanta mitr 15:02:16 Current chairs: Evolution adamw davidstrauss mitr mizmo nirik sgallagh simo tuanta 15:02:18 #topic roll call 15:02:34 .hellomynameis mizmo 15:02:35 mizmo: Sorry, but you don't exist 15:02:37 .hellomynameis duffy 15:02:38 haha 15:02:39 mizmo: duffy 'Máirín Duffy' 15:02:41 .hellomynameis sgallagh 15:02:46 sgallagh: sgallagh 'Stephen Gallagher' 15:02:57 Hello 15:03:00 .fas tuanta 15:03:01 tuanta: tuanta 'Truong Anh Tuan' 15:03:23 .hellomynameis adamwill 15:03:24 adamw: adamwill 'Adam Williamson' 15:03:34 mizmo: man, that's a sucky way to start the morning 15:03:48 being told you don't exist by a bot :) 15:04:05 i know! 15:04:09 :( 15:04:12 :) 15:04:41 If you rely on IRC bots to confirm your existence, I don't know if I can help you 15:05:51 validatemyexistencebot 15:06:10 ok, we have a quorum, so shall we start? 15:07:05 #topic Firewall 15:07:28 So, I think the conversation here has largely died down. Anyone want to try to phrase the decision for approval? 15:08:47 mitr: You had the most concerns, originally. Have those been addressed? 15:09:21 My understanding is: 15:09:23 Add "If the user hasn't specified firewall status explicitly, role deployment will inform the user whether the service's ports have been opened by default" to either the 2nd or the 3rd paragraph of the Firewall section of the tech spec. 15:09:34 simo: is that correct? 15:12:00 how would they be informed? 15:12:21 * nirik arrives back after comcastic connection issues. 15:13:02 mizmo: If it's being deployed by the cmdline tool or Cockpit, they should provide user feedback at the end 15:13:20 The TUI can simply write that ("... the service is now {not,} available for connection; to override, use $command"); the GUI might just return back to the service view that also includes a firewall indication, no idea 15:13:20 If it's being deployed in an unattended mechanism, it needs to be possible to query what it changed after the fact. 15:13:34 is this too far into implementation detail? 15:13:43 Good point, non-interactive are probably out-of-scope. 15:13:57 s/role deployment/interactive role deployment/ above 15:14:01 adamw: I think we can describe this at a high-level. 15:14:10 fair enough 15:14:14 At least say "it must be visible if it's attended-mode" 15:14:48 non-interactive, log it? 15:15:11 mizmo: That's one part of it, but I think we want the Role API to also allow querying of that 15:15:29 So we can ask it "What ports do you *want* open" and then we can ask firewalld if they are. 15:16:03 Logs are only useful to tell us what they were *at that moment*, but such an API can provide live information at any point. 15:16:25 ... and that point it doesn't really need to be in the logs. I think it's reasonable for someone designing an automated system to test the final status, we want the interactive / newbie case to be easy to do and difficult to do wrong. 15:16:42 okay sounds reasonable 15:16:58 sure. 15:17:36 Ok, so... 15:18:08 * sgallagh tries to phrase all this 15:20:11 Add "If the user hasn't specified firewall status explicitly, interactive role deployment will inform the user whether the service's ports have been opened by default" to either the 2nd or the 3rd paragraph of the Firewall section of the tech spec. ? 15:20:28 Proposal: If the user hasn't specified firewall status explicitly, interactive role deployment will inform the user whether the service's ports have been opened by default" to either the 2nd or the 3rd paragraph of the Firewall section of the tech spec. It must be possible to query the API for the required state of the firewall to support the role, which can then be compared to the active firewalld state. 15:21:09 sure, that seems reasonable I guess. 15:21:17 +1 15:21:28 Either is fine with me. (We already have the API requirement in "Role Definition Requirements") 15:21:59 +1 15:22:02 +1 15:22:21 mitr: Right, but that entire section was written by me without external input. So this is me asking :) 15:22:38 +1 (for the record) 15:23:59 #agreed If the user hasn't specified firewall status explicitly, interactive role deployment will inform the user whether the service's ports have been opened by default" to either the 2nd or the 3rd paragraph of the Firewall section of the tech spec. It must be possible to query the API for the required state of the firewall to support the role, which can then be compared to the active firewalld state. 15:24:12 #action sgallagh to update the Technical Specification with this phrasing 15:25:00 #topic Implementation of Roles 15:25:48 sgallagh: for the record +! to your wording on firewall 15:26:02 sorry I am caught in 4-5 convos and other things so I am slow :( 15:35:23 simo: well, we have 1. the gold repo. 15:35:27 if you can downgrade, often you cant 15:35:28 however fedora makes it difficult 15:35:36 nirik: that's not sufficient 15:35:54 simo: where do we get the storage to hold all these pinned packages? how do we keep consistency between them? 15:36:04 mizmo: often you can't because we wipe out the old package as soon as the new gets in, so you can downgarde only if your machine has the old package in the cache 15:36:19 mizmo: and then when you have issues you run yum clrean all ... and byebye downgrades 15:36:30 what? clean doesn't do that. 15:36:33 do you mean distro-sync ? 15:36:37 oh, your cached copy, i see. 15:36:44 adamw: dunno where we get the storage, why debian can and fedora can't ? 15:36:56 simo, ive done downgrades before that wouldn't work because some package s needed a newer version of y but if i downgraded it would change the version of package y and blah blah blah 15:37:08 this still feels like people wanting to do gross hacks as an incorrect way to fix a problem to me. 15:37:16 mizmo: sure it is not always a silver bullet 15:37:21 adamw: yes 15:37:22 however the point here is strawman 15:37:29 additionally: due to the way mash/koji works, it's hard to do more than most recent package tagged in. What do you do with 1.0 and 1.1 in packages, but 1.0 has serious security bugs? allow people to downgrade to it anyhow? deltarpms do what with all the new updates? 15:37:30 simo, it's like that rope bridge in indiana jones 15:37:41 adamw: me too. 15:37:47 Ok, so crazy alternative approach: Instead of depending on other packages, we do a monstrous, bundled SCL for a Role. Discuss :) 15:37:53 we do not need downgrades but multiple pacages so we can pin a specific version and when an upgrade comes out it doesn't why the older "poinned" one that is still a hard dependency for something else 15:38:06 sgallagh, SCL? 15:38:10 why can't we just disallow the broken ones into stable? if we know what we need to test more now, that should be easier no? 15:38:20 mizmo: Software Collection 15:38:27 i think it would be fine to say something like updates to packages critical to a 'supported' / 'premier' / whatever Server role should be tested in that role before going stable, whatever the details of that are: basically, we just don't push them out until they're properly tested. 15:38:28 I mentioned downgrades because that is another case where we need multiple versions 15:38:30 Essentially a special RPM that can install in somewhere other than /usr 15:38:35 nirik, sometimes people want an older version not because newer is broken, but because newer in incompatible with some 3rd party shit they need 15:38:38 sgallagh: alternative-alternative approach: In general, assume that packages that keep an API are maintained as to not generate intentional breakage/functionality regressions; For those packages that don't, put them into the much-smaller per-role SCL 15:38:41 sgallagh: no 15:39:09 mizmo: they could then pin it locally or grab it from koji. I wouldn't expect that to be something we need to bend over backwards for. ;) 15:39:13 sgallagh: it's like saying we do multiple distributions or fork the distribution in different projects per product, only worse, different project per role 15:39:14 mizmo: That's explicitly what SCLs are for 15:39:14 simo: It wasn't serious, it was just meant to change perspective 15:39:24 sgallagh: do you have any idea of the dependency chain for freeipa ? 15:39:48 simo: I used to do the Bodhi updates. Yes, I have a pretty good idea :) 15:39:52 is an scl a chroot on the system where rpms get installed outside of the system wide db?? 15:39:58 we just need multipl epackages so we can qualify stuff doesn;t break spectacularly before a new freeipa version allows to use a new component 15:40:02 mizmo: No 15:40:07 mitr: our example case was characterized as 'accidental', not intentional. 15:40:09 They are installed in the system-wide DB 15:40:11 simo: Is your concern that there is too much "intentional" breakage, or that the breakage is in principle unintentional, but we just don't have the testing capacity/capability to detect it? 15:40:18 They just put their files somewhere non-standard. 15:40:25 mitr: no packages get different names and are installed in different location 15:40:33 sgallagh, i dont udnerstand how that solves the prob 15:40:33 I'm not sure how it works for apps with config files... I've never seen it used for that 15:40:48 i ask again as i didn't see a good answer last time, btw: why is it important to push out 389-ds point updates before they're tested with freeipa? 15:40:57 (especially if we *know* they're dangerous to freeipa?) 15:40:57 mizmo: They have a unique name and don't conflict with a different version of hte same package onthe system 15:41:00 mitr: it is usually not intentional, but the interaction are so complex, sometimes it happen an update that seem uncontroversial breaks stuff 15:41:16 sgallagh, oh so it allows two vesrions to live on the system at the same time? 15:41:20 mitr: keep in mind we have beelding edge versions of everything in fedora so it is not always super stable stuff 15:41:24 mizmo: Precisely 15:41:27 simo: would gating any update on a freeipa test run solve the problem in principle? would it be doable in practice? 15:41:28 okay thanks! 15:41:37 there is experimentation, new features in all the packages and sometimes they break stuff 15:41:44 That's what it's used for in Red Hat Software Collections right now: it gives people access to newer compilers on RHEL 6 (for example) 15:41:50 mitr: and if it doesn't, how exactly does this 'have freeipa depend on a specific version' thing work? when is that dependency bumped? 15:41:59 mitr: in principle yes, if you had a very thorugh test suite 15:42:08 simo: that sounds a lot like you need 389-ds to have stable branches. 15:42:20 mizmo: we do not have a full public test suite yet and lots of the RHEL QA is also manual 15:42:23 you are not supposed to be experimenting in bugfix releases. 15:42:30 so we are far from being able to automate it fully yet 15:42:30 the more i hear about this, the more it sounds like upstream's problem. 15:42:47 adamw: it is partly upstream problem 15:42:55 adamw: same difference for 8us* 15:42:59 *us* 15:43:07 stuff breaks 15:43:07 not really, no. 15:43:24 we need to have some time to test before allowing things through 15:43:31 that's what updates-testing is for. 15:43:34 w/o tieing *everything* to freeipa 15:43:36 *precisely* what it is for. 15:43:39 389ds is used standalone 15:43:53 adamw: but you are gating stuff to freeipa then 15:43:57 sure. that doesn't mean standalone deployments need all the latest updates immediately. 15:44:03 sure. what's the problem with that? 15:44:13 Perhaps we should consider asserting that if a package is part of a Role, then its individual needs are considered secondary to the Role? 15:44:15 it's a valid use for it. it should not break it. 15:44:23 do standalone 389-ds deployments magically stop working if they don't get the latest partly-bugfix, partly-experimental updates? 15:44:35 i.e. No Role package exits updates-testing without being tested holistically with the Role. 15:44:44 sgallagh: i don't even see that anything in the current discussion requires *that* much yet. 15:45:04 sgallagh: ok *if* we can force that then yes, but would need changes to bodhi I fear 15:45:23 adamw: if you could downgrade we could relax this requirement 15:45:26 i keep asking, but no-one has explained to me the urgency of the specific or general example cases yet: why it's actually a huge problem to wait for testing of a significant dependent package before pushing an update stable. 15:45:27 simo: Well, in the short-term, it could be policy rather than enforced 15:45:30 but people whine about storage space 15:45:35 so downgrades are not feasible 15:45:37 Slap people around if they violate it 15:45:38 and we are stuck 15:45:44 *WHEN* it breaks it stays broken 15:45:50 what is a user supposed to do ??? 15:45:55 simo: you can force it very easily at present: submit your updates without autokarma. 15:46:02 sgallagh: slapping people does not help users 15:46:03 you are then in complete control of when it goes stable. 15:46:09 so, don't push it stable until it's tested with freeipa. 15:46:16 adamw: I am not in control of the packages I depend on 15:46:27 simo: where 'you' is the 389-ds package maintainer. 15:46:31 and there is a slew of them 15:46:39 We also already have the Bodh link to test procedures that could apply to 389-ds 15:46:55 (and significant dependencies) 15:46:55 adamw: so I need to go to each package maintainer and ask them kindly to always test with freeipa ? 15:47:07 simo: What I mean is that in the short term, we could rely on policy (and potentially the removal of packager privileges) to solve this 15:47:07 down to the mainatainer of the java headless packages ? and tomcat ? 15:47:11 if we need more strict critera on a set of packages updates, we could do something like the critical path setup 15:47:14 And get Bodhi enforcement in later 15:47:14 simo: maintainer of a package you depend on which has major development issues, sure. 15:47:14 and cmlrpoc libs ? 15:47:17 and python ? 15:47:21 and all python modules ? 15:47:21 simo: Solving communications problems by doing extra technical design seems not optimal. If we have to do this, let's solve this by doing extra process, but not complicating the technical design. 15:47:25 oh and glibc ? 15:47:26 and sssd ? 15:47:27 simo: do changes to those packages in stable updates frequently break freeipa? 15:47:34 oh wait and *everything* ? 15:47:46 everything in the distro depends on glibc, more or less. 15:47:54 yet we ship updates to it, and in the main, things seem not to explode. 15:47:57 yay development practices! 15:48:06 adamw: right, I was exaggerating obviosuly 15:48:26 there are more important and less important hings 15:48:41 and as I said I would be content if we got downgrades possible 15:48:44 i remain unconvinced that anyone has demonstrated anything particularly specific to the .next effort or to Products or to any of our roles that make a convincing case for doing variant stable package sets, basically. 15:48:51 it would handle 99% of the problem cases 15:49:11 this still sounds a lot like the wrong fix to a problem trying to get attached to a process it really has little to do with. 15:49:30 it is a last resource fix 15:49:33 when all else fails 15:49:37 therefore i'm a moderate -1 to it in general and a strong -1 to it being tied to Server product in any way. 15:49:46 and it happens not so rarely that people need to downgrade in fedora 15:49:52 some more downgrade problems: people will start using them and get mad when they don't work on things with scriptlets or database migrations, etc. 15:50:17 nirik: people use them today, I haven;t seen an uptick in screaming 15:50:37 they are just not very reliable as sometimes you do not have a package to downgrade to 15:50:51 simo: the people using them usually know they exist and the issues... if we documented the official server product to use them I bet we would see more. ;) 15:51:03 So should we throw Fedora Atomic into the mix here? 15:51:09 I am not for making it necessarily an official way to fix poblem 15:51:19 sgallagh: what's Fedora Atomic ? 15:51:27 first time I hear this name 15:51:39 simo: http://rpm-ostree.cloud.fedoraproject.org/#/ 15:51:41 * nirik doesn't think we should. unless we are planning on using it for f21. 15:51:45 simo: we can always downgrade with epoch 15:52:00 mitr: "Here, have a bigger hammer"? 15:52:16 mitr: NO 15:52:37 if you have to release a new package you may as well fix the problem 15:52:48 whatever 15:52:57 I think my pov is clear 15:53:16 do with it what you think is best, it can;t suck more than it sucks now, which is quite a lot :) 15:54:37 here's a way to look at it: if the problem is that packages freeipa depends on ship updates that claim not to change their public API but in fact do (in effect), it is craziness to suggest that the solution to this is 'legitimize carrying multiple, identically named variants of the same API in our stable updates repository', which is basically your initial proposal 15:55:10 Proposal: For Fedora 21, we address upgrade compatibility through policy rather than enforcement. Any core piece of a Role is expected to coordinate with that Role before upgrading (the responsibility is on the package maintainer, not the Role owner). 15:55:21 i mean, now what do we whenever anyone updates anything else that depends on that package? test it against both stable versions? 15:55:24 adamw: I think legitimizing this _for those specific packages_ is ... acceptable; what I hate is the side-effect of legitimizing this for every other package. 15:55:41 sgallagh: +1 15:56:11 mitr: if we have specific problematic packages that refuse to fix their development practices, sure, use one of the established approaches like multiple *differently-named* packages for the same project. 15:56:13 carrying old updates around somewhere to make downgrades more viable is less hairily insane, but i'm still not a huge fan 15:56:39 adamw: again not legitimizing anything 15:56:47 just give an escape route to poor users 15:56:53 sgallagh: sure, but I think we could put in policy if needed (at the bodhi level) but we can cross that later if it turns out to be a big deal 15:56:58 i'm now never sure which of your proposals you're talking about 15:57:05 with abig sticker "this is a last resort, not supported option, but give it a try if you have no other recourse" 15:57:23 adamw: multiple package versions in the repo 15:57:27 we should be trying to make sure our poor users never hit that. ;) 15:57:29 nirik: Yeah, that was the subtetx 15:57:35 allowing explicitly for multiple versions of a package in the default-enabled repos and allowing packages to depend on specific versions sounds a hell of a lot like 'legitimizing' to me 15:57:42 nirik: yeah and we should solve world huger too :) 15:57:44 +1 to sgallagh's proposal 15:57:57 simo: we should! :) 15:58:06 nirik: within fedora ? 15:58:12 15:58:22 and to nirik's suggestion that we can consider a bigger stick in future if necessary 15:58:27 and, of course, insert my usual bodhi 2.0 spiel 15:58:33 sgallagh: +1 reluctantly (ie we changing nothing we just acknowledge the status quo, but ok) 15:58:40 If we could, but I think it's more reachable to just work harder to never push out packages that break our roles. 15:58:41 No, there's one change 15:58:42 (i assume it's screamingly obvious to everyone, but this is clearly a bodhi 2.0 problem) 15:58:49 adamw: yeah. 15:58:56 We're explicitly putting the responsibility on the dependent package owner not to screw up the Roles 15:59:08 sgallagh: which is the current situation 15:59:13 With the implication that FESCo could take action if they don't 15:59:24 which fesco can already do :) 15:59:29 simo: Right now, it's FreeIPA's job to check if 389 breaks things 15:59:35 not really 15:59:43 Effectively... 16:00:46 Ok, we're over the hour. Do we want to continue here or open a discussion on the mailing list about Role packaging? 16:01:09 * nirik is fine either way, but needs coffee. 16:01:31 * mizmo could stay 16:01:48 * mizmo has a much harder time following the list :( 16:01:50 i have time 16:02:14 I can continue as well 16:02:23 Me as well 16:02:28 OK 16:02:38 we should finalize this topic 16:02:50 Well, this specific discussion at least. 16:03:02 Anyone want to contradict my proposal above? 16:03:12 sorry I have another call and really need to eat 16:03:26 you have my +1 16:03:26 simo: No problem. Thanks for your time 16:03:30 im always a much bigger fan of the design enforcing rules than people's adherence to policy... but ill +1 anyway 16:03:54 +1 too 16:03:55 mizmo: Me too, but I think solving that is a bigger effort than we can swallow in F21 16:04:00 exactly 16:04:21 mizmo: That would happen by actually writing a comprehensive test suite, which is too much 16:04:21 #info For Fedora 21, we address upgrade compatibility through policy rather than enforcement. Any core piece of a Role is expected to coordinate with that Role before upgrading (the responsibility is on the package maintainer, not the Role owner). 16:05:02 Ok, so implementation proposal for the Role packaging: 16:06:16 Proposal: A) All roles will have a special package named fedora-server-role-$ROLENAME that will include strict requirements on its dependencies. Installation of a Role will be performed by simple 'yum install' 16:07:00 Proposal: B) We will defer handling Role disable/remove to after Fedora 21 16:07:41 Proposal: C) The Role package will also provide the necessary plugins to Cockpit to enable deployment and management of the role. 16:07:53 EOF 16:08:15 sgallagh: will fedora-server-role-$rolename also require fedora-server-release? (with the base stuff for all roles, etc) 16:08:22 A) I do want to reiterate that upgrading a role package and upgrading a deployed role shouldn't be the same thing 16:08:24 C) It's kind of strange to decide that Cockpit will be doing this without deciding what will be done at both ends, but whatever :) 16:08:38 nirik: Yes. I will amend the proposal to state that 16:08:58 am i weird to be disappointed that we aren't using containers for roles? 16:09:15 or would you deploy these to containers? 16:09:15 mitr: Well, the Cockpit Project has volunteered to implement the D-BUS API and graphical UI for us, so it seems pretty much the "right way" 16:09:30 mizmo: That's being left ambiguous right now 16:09:31 mizmo: Again, we create containers by telling the software "install these RPMs" 16:09:39 i like the basic idea of doing everything we can with packages, but is it the Right Way for our role deployment...things? cockpit etc 16:09:54 also, this is basically metapackages, and historically fedora has always said 'use comps groups' 16:09:54 adamw: Sorry, could you elaborate? 16:10:17 mitr, if yum installing fedora-server-role-blah.rpm is effectively deploying a container and is the implementation detail why not call it deploying a container 16:10:20 sgallagh: that's literally the question - it is not a loaded question. i am requesting information, basically 16:10:21 adamw: Well, the package will have its own content 16:10:46 mitr, it's like saying you make dinner with noodles, but the dinner is actually chow mein so why not call it chow mein 16:10:48 sgallagh: i had the impression there was already a plan for what we'll be using to deploy/manage roles; are those things expected to work well with RPM packages? 16:10:53 mizmo: Not everything necessarily can live in a container 16:11:06 FreeIPA would probably have a tough time with that (though I think they're working on it) 16:11:25 is that the only reason why we're not talking at that level? 16:11:25 mizmo: Two problems: 1) how do you get software into the container (answer = RPM, and that's what we need to design in any case), 2) given a possible container content, whether/why/how do we deploy it into containers vs. on the root system (answer = it might be useful but we can't do it until 1) is solved) 16:11:52 adamw: RPM packages are just "how do the bits get onto the system". Making those bits do something useful is the responsibility of the configuration API 16:11:54 so the outline here is that we define a set of packages you need installed to implement a role, and a separate mechanism for containerizing the role which assumes all the packages are available to it, and then integrate those at a higher level? 16:11:57 rgr 16:11:58 i dont even see how 1 is a problem. thats how we deploy software, im not sure why that would be a question 16:12:01 mitr ^ 16:12:02 s/configuration/deployment and configuration/ 16:12:16 containerizing/configuring 16:12:20 re: 2, for portability no? it solves a million problems for server admins 16:12:49 eg 16:13:02 at least for me, being as stupid as i am but needing to deploy shit on servers because of various circumstances 16:13:21 if this is just bit-deployment, yeah ,the basic shape of the proposal seems fine 16:13:31 mizmo: 1) see the conversation we are having now :) metapackages vs. comps groups, version requirements vs. multi-version etc. 2) portability is useful _only for admins who don't homogenously use a single distribution_ or don't use "latest RPMs"; we would far more benefit from isolation 16:13:59 the alternative is to use comps: have the fedora-role-foo package but don't give it the bunch of dependencies, group them in a comps group. but i'm not saying that's actually better, just running it up the flagpole. 16:14:04 the idea of grabbing a container (however that process is specifically implemented) off-the-shelf with a running wikimedia server i can set up for my project, sounds a lot nicer than installing a super rpm that pulls in a bunch of crap and i dont know where it all is and what stuff acme from upstream vs what stuff is special 16:14:17 * nirik is fine with the plan/idea. 16:14:34 adamw: Well, either way we'll need the fedora-role-foo package, and it will need to provide some data (like the plugins) 16:14:39 mizmo: this doesn't preclude that, afaics 16:14:44 adamw: problem there is that you can't require a comps group from a package (that I know of) 16:14:47 I'd argue we'd still want the strict deps there to keep everything together 16:14:50 portability is useful for me when i set shit up on my laptop and need to deploy it to amazon or whatever. i dont know what amazon uses, i dont really care 16:14:58 mizmo: someone somewhere still has to *build* that container image, and that person needs to be able to deploy the right bits. 16:15:00 But we could then add a comps group in addition to make it easy to install, I guess 16:15:11 adamw, i know it doesn't preclude it i just feel like we are skipping a whole layer 16:15:13 Formally, counterproposal A): 16:15:15 fedora-server-$rolename will not have strict version dependencies; it will support major versions of roles as far into the past as necessary. Installation of a role happens primarily via a comps group; alternatively, deployment of a role (with fedora-server-$rolename installed) will pull the appropriate versions of dependent packages 16:15:23 nirik: no, you can't, in that design you'd theoretically be able to have the role package installed without its 'dependencies'. 16:15:28 (which obviously would be wrong.) 16:15:42 mizmo: i think we're building up from the lower layers to the higher ones 16:15:51 mitr: Could you rephrase the last sentence? It reads ambiguously to me. 16:16:02 adamw, that seems like the wrong order to be (the ux designer, admittedly) 16:16:05 sgallagh: if the package has deps, your comps group would just have the role package, i guess. 16:16:25 Are you suggesting that the deployment step might do new package installation? 16:16:37 mizmo: i think that's a perspective thing indeed =) i think engineers tend to default to building from the bottom up, UI folks from the top down... 16:16:47 sgallagh: Either (yum install fedora-server-group-postgresql; fedora-server-deploy postgresql # which only does the deployment), or (yum install fedora-role-postgresql; fedora-server-deploy postgresql # asks to install postgres-server) 16:16:48 the top down is the right way! 16:16:48 I'd somewhat prefer to conflate the two (with the exception for add-on capabilities) 16:16:56 postgres$N-server, rather 16:17:02 comps groups sounds nicer to me than 'special' rpm, but still not satisfactory 16:17:11 although there is a lot of cool shit built into anaconda that would make the comps group way sexier 16:17:25 mitr: where does the concern with versioning come from? i didn't see that the initial proposal involved any version-specific dependencies 16:17:29 sgallagh: ISTM that the major version upgrade scenario requires the ability to have support files for a new version of a role installed while the old version of the underlying service is still running. 16:17:41 Perhaps we should listen to mizmo here. 16:17:52 mizmo: How do you see this working from a top-down approach? 16:17:57 adamw: "Fedora server implementation straw man" thread recently 16:18:03 yes, instead of playing with a bucket of legos, let's talk about what to build first 16:18:07 (Apologies if you're repeating yourself, there's a lot of chatter going by fast) 16:18:16 mitr: thanks 16:18:16 * adamw goes to read in the meantime 16:18:34 sgallagh, i want to talk about it at the level that i'm an admin (let's say... one of the personas, miss macguyver how about) 16:18:58 Ok, go on (type EOF when you finish, so we know you're done) 16:19:06 and i need something, i guess let's say i need an identity server for my startup company cuz we're getting big and winging it on gmail shit isn't working out anymore 16:19:16 so i hear about this fedora server thing, and the identity server role 16:19:31 so i grab a vm to play with, to try to set up a test identity server 16:19:54 for me, miss macguyver.... what experience would i like to see, what would make sense to me, to start from fedora server => identity server running on fedora server 16:20:04 what steps would I expect to take? what steps make most sense to me? 16:20:43 this is where i was going a bit... for me, installing some special rpm that blows its nose all over the system in a very fedora/redhat specific way seems intimidating (and im from that world) 16:21:17 whereas - even if it is implemented using the RPM / noseblow method under the covers!! it doesn't matter it's apperaances - if i conceptually make a mental model of it being a container sitting neatly on top with seperation from the base os 16:21:23 it seems a bit easier to grok 16:21:32 okay EOM 16:22:00 Ok, so I have several points to reply to 16:22:20 1) Calling something a container doesn't necessarily have meaning. I agree that we want it to appear as a *unit*, however. 16:22:26 mizmo: you already have a VM to have a safe way to experiment; do you really need another layer? 16:22:56 1) implementation wise it might not have meaning but to end users it creates a clearer mental model not muddled in our specific tech choicse 16:22:59 2) The special RPM would be entirely under the covers. I don't think anyone really suggested otherwise. 16:23:16 mitr, yes, definitely 16:23:37 im trying to think of a clearer way to explain the problem is 16:23:45 mizmo: We weren't talking about the public view in this discussion. The public view should be "This Rule is [not] currently deployed on this machine" 16:23:51 s/Rule/Role/ 16:24:03 when i think about 'deploy this fantastic and complex mechanism with a lot of moving parts' i don't think 'install an rpm' because 'install an rpm' is already used at a much smaller scale 16:24:09 rpm is the wrong level 16:24:20 Right, I don't think we're going to talk about it that way at the user level 16:24:22 container makes more sense conceptually because it's more abstract 16:24:55 okay 16:25:00 At a user level, we'd be talking about a two-phase process: Configure and Deploy 16:25:29 We would present a UI (TUI, GUI, WUI...) that asks the necessary questions in a nice way. 16:25:53 Then when we deploy it, *under the hood*, this means "put the bits in place, apply config files, start services" 16:26:12 and we're not talking about multi-tenancy yet? 16:26:56 mizmo: No, we're talking about on the bare [virtual] machine at this point 16:27:27 so when we eventually move to multi-tenancy (is that planned?) then this implementation would necessarily have to change right 16:27:52 Multi-tenancy _within a single VM_ is IMHO a really specialized problem, for OpenShfit to solve. (And yes, we should keep an eye on OpenShift to make autos-scalable roles usable within that model - but "auto-scalable" is a significantly smaller subset) 16:27:55 by going this way... is this all going to have to be refigured out for multi-tenancy further down the road? 16:28:24 eg if we say magic rpm deploys this stuff, does that make it impossible to do multitenancy in a year or two without rearchitecting everything 16:28:36 mizmo: No. We will keep all our RPMs (with the non-trivial business logic of deployment, APIs and the like) and stuff them into a container (with the comparatively trivial logic of separate namespaces and putting files in place) 16:28:37 mizmo: multi-tenancy is more interesting for applications than infrastructure services 16:29:13 One of our future roles would likely be "Container Host Role" or "OpenShift Node Role", which would set up a framework for something like that 16:29:24 But I don't think the Role mechanism is the right place to be talking about app deployment 16:29:41 sgallagh, (i hope its okay to ask i dont mean to grind everything bc of my not being as tuned into the implementation stuff) - where do you draw the line between app and infra service 16:29:46 sgallagh: Well a JBoss app server could reasonably be a Role to also deploy within an autoscaling container 16:29:52 eg mailserver - infraservice. mailman - ? app? infraservice? 16:30:15 sgallagh: (talking about something I don't know that much about...) 16:30:16 mizmo: mailman is probably an app, yes 16:30:43 is infraservice = role? at the same level? and app is a level above? 16:30:56 mizmo: Basically, yes 16:30:58 eg mailman as an app, could be part of a role (eg collaboration server.) is collaboration server an infraservice? 16:31:18 That's why "Database Server Role" is sensible, but "LAMP Stack Role" is probably not. 16:31:33 I don't view servers and apps as fundamentally different FWIW... what's the difference? The multiplexer on port 80? 16:31:44 mizmo: Collaboration Server is one of those nasty edge-cases :) 16:31:49 would you deploy mailman on the same system as you have the mailserver? would having a role that includes both require that both be on the same machine? is that a problem bc it's better to deploy on different systems? 16:32:09 sgallagh, what's a better case to use, i dont mean to pick edges, i work on mailman upstream which is why i brought it up im familiar with it :) 16:32:26 mizmo: No, it's a valid question. I just hate the hard questions :) 16:32:46 im glad its a hard question in that im apparently not spectacularly stupid / ignorant then 16:33:06 mizmo: A collaboration server is probably something I'd (personally) categorize as a candidate for a Role, yes 16:33:19 Because it has clear boundaries on its configuration 16:33:59 could an easier role to use as an example for purposes of discussion be a version control server? 16:34:02 Whereas "HTTP Server" is a lot harder (and requires so much configuration that it probably doesn't make sense to attempt) 16:34:14 it's got git, and it's got cgit or some other webui (gitorious! <= very complex) 16:34:36 'http server' is too generic i think 16:34:42 what's it serving and how 16:35:00 Sure, I was pointing out that without clear boundaries, we shouldn't consider it 16:35:17 I also think that Drupal and Wordpress are probably outside our Role interest 16:35:31 what about version control server? <= is that a thing? 16:35:39 Because they're frameworks that can be configured in vastly different ways (and would be much better suited to living inside a PaaS environment) 16:35:41 i know we have them set up as infra inhouse here 16:35:48 I'm thinking about that one in parallel. 16:36:34 I'd suggest that 'gitorious' is possibly more interesting as an infrastructure component than git-server would be 16:36:50 im basically thinking, the role is purpose-oriented (version control) but there maybe be a core and apps that together give you a full working infrastructure setup for the purpose (eg core is git and http, the extra shit would be like gitorious or cgit or whatever) 16:36:56 I'd say that both a version control server and any_random_webapp (perhaps not wordpress or an "app platform") could easily be a role but for the unforutnate fact that "our OS" doesn't say how to integrate all of them to the port-80 multiplexer (aka "any of the web servers that conflict") 16:37:20 Yeah, there's that technical issue too :( 16:37:38 so you go to the hostname on port 80 and it doesn't know whether to show you cgit or gitorious front page or whatever 16:37:56 * nirik in general thinks we should look at moving to containers for roles, but I don't think they are mature enough for a first cut. 16:37:57 Right 16:38:11 Even if you do know that you want one to own /mailman/listinfo and another to own /cgit, we don't have an universal way to configure it 16:38:14 nirik: I pretty much agree with that entire sentence 16:38:18 nuts 16:38:53 my keyboard battery is dying bwahhaha 16:38:58 * mizmo looks for battery 16:39:50 So... can we agree that containers are probably desirable, but are not necessarliy end-user-visible (... in the GUI) anyway, and we don't need to block the RPM design on having containers ready now? 16:39:56 And we've already got one role that's going to claim 80: FreeIPA's web UI 16:40:06 mitr: +1 16:40:28 mitr: +1. I think once we have a rpm role (even an early one) we could look at containerizing it (is that a word?) 16:40:59 +1 as long as the RPM design doesn't preclude or negatively impact adopting containers later 16:41:31 sounds reasonable 16:41:45 nirik: "containing", I think ;-) 16:42:09 that doesn't sound as cool. ;) 16:42:13 side-note: I'm trying to drum up support for replacing "containers" which is overloaded with "isolation pods". :-D 16:42:37 * nirik thinks we could ask davidstrauss to help with that as he was wanting to work on docker stuff. ;) 16:43:01 yeah, i was working on the basis that this is underlying plumbing 16:43:10 * sgallagh nods 16:43:17 me too 16:43:32 That's what we get for inviting mizmo. We have to *justify* our crazy decisions ;-) 16:44:13 BTW. are there known items that would "preclude or negatively impact adopting containers"? 16:44:40 there were a couple of issues that the discussion brought to the surface too, the port 80 problem between cockpit and any other apps a role might require, and there was something else 16:44:42 migration from existing bare-metal roles might be "fun" 16:44:55 mizmo: Cockpit doesn't use 80 16:45:09 oh okay 16:45:20 At least at present 16:45:34 well, still - the essential problem is there isn't a way to manage multiple web apps wanting 80 16:45:39 I don't think it ever will, since most servers would want 80 for other things 16:45:46 from the role deployment/mgmt pov 16:45:49 we could add that to the role api... 16:45:53 mizmo: Yeah, that's actually a topic of discussion on fedora-devel at the moment 16:45:58 sgallagh: if RPM tells us what files to use, and the role tells us where the data is, it actually might be fairly doable 16:46:17 nirik: Probably not reliably. Apache config is nightmarish 16:46:30 Let's not fixate on port 80 now - we do need to solve it but we can do it as the major feature of F22 :) 16:46:40 yeah, it is... but might be doable... dunno. 16:46:43 mitr: Sure, but I think we'd want to improve tools for that first 16:47:07 I think Debian has some tools for managing vhosts and dropdir configs for example 16:48:25 Ok, so do we want to cover anything else today, or take other topics to the list? 16:48:29 so, um, we're talking around stuff a lot, but can we really say with confidence we know precisely which approach to bit deployment is the right one yet? 16:48:29 We're coming up on the second hour 16:48:59 it still may just be my biases talking but i'm feeling castle-in-the-air-y again. has anyone tried actually building some toy version of this stuff yet to see what works best? 16:49:16 i'm not sure it's a good idea to write metapackage-vs-comps-group or whatever into the spec yet 16:49:34 I've been planning to for a month, but I couldn't find the time :( 16:50:06 I think we really need to agree on the required feature set, and the implementation will then either be dictated by it, or it won't matter 16:50:26 mitr: true enough 16:50:32 if its required to be installed in anaconda, metapackage isn't good enoguh, needs a comps group 16:50:54 mizmo: Sure, but a comps group could simply contain the metapackage. That's not hard. 16:50:55 mizmo: We can always create a single-metapackage comps group if that's the only concern 16:50:58 right. 16:51:00 yeh 16:51:01 * nirik hasn't tried to make one yet either. 16:51:24 i guess metapackage vs compsgroup is the control 16:51:39 i think who has perms to edit comps vs who has perms to edit any particular packages differ 16:52:03 Getting a comps edit is likely the least of our concerns here 16:52:23 nirik, mitr: Exactly what have you tried to make? 16:52:29 (or been planning to) 16:52:31 well i mean less about the getting the edit and more about, the changes etc would be managed in comps vs in a spec file or whatever 16:52:34 nothing. I haven't. 16:52:38 which i think is a good thing 16:53:29 mitr: Is the set of requirements we've put on the PRD and Tech Spec not sufficient to make those decisions? 16:53:34 If not, what are we lacking? 16:53:38 sgallagh: I was thinking about what I posted on that thread - per-role Python module keeping knowledge of various major versions, with "fully automated major version upgrade" as the first code to write. (That would involve the basic interaction framework, deployment descriptor -> config, config upgrades) 16:53:41 (Don't answer right now, send it to the list) 16:54:28 sgallagh: Not sure. 16:54:54 mitr: fully automated major version upgrade seems like the hardest problem to solve. I suppose starting with it is one approach. 16:55:15 Do you envision something like fedup's upgrade scripts? 16:55:34 ... This is going to be a long conversation and I have another meeting in four minutes 16:55:46 sgallagh: Well pg already gives us the tools to handle the data, so it would be mainly the config upgrade (... was thinking about git 3-way-merge) 16:55:48 mitr: Can you please expound on this on the mailing list? 16:56:15 mitr: git 3-way-merge isn't going to help if a config format changed completely 16:56:16 sgallagh: I think the "implementation straw man" thread is where I have posted something. 16:56:25 Sure 16:56:28 mitr: Right, but maybe with some more examples? 16:56:51 sgallagh: OK, will add a few more pages of vague pseudocode :) 16:57:06 mitr: That would be a big help. Right now it's too theoretical for me 16:57:21 do we have enough info now to make a test fedora-server-release and fedora-server-role-freeipa? 16:57:36 (at least skeletons) 16:57:58 nirik: Probably, yes. 16:58:10 At least enough to come up with a straw-man 16:58:40 nirik: You mind scheduling something with simo and I later in the week and we can try to hash out a first pass? 16:59:12 I can try... not sure when exactly... 16:59:25 Ok, anything else before I close the meeting? 17:00:22 Thanks everyone. Plenty to think about. 17:00:25 #endmeeting