14:59:04 #startmeeting Server Working Group Weekly Meeting (2014-03-18) 14:59:04 Meeting started Tue Mar 18 14:59:04 2014 UTC. The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:59:04 Useful Commands: #action #agreed #halp #info #idea #link #topic. 14:59:06 #chair sgallagh mizmo nirik davidstrauss Evolution adamw simo tuanta mitr 14:59:06 Current chairs: Evolution adamw davidstrauss mitr mizmo nirik sgallagh simo tuanta 14:59:11 #topic roll call 14:59:14 * nirik waves. morning. 14:59:16 .hellomynameis sgallagh 14:59:17 sgallagh: sgallagh 'Stephen Gallagher' 14:59:39 .hellomynameis adamw 14:59:40 grr 14:59:42 adamw: adamw 'Adam Warski' 14:59:43 .hellomynameis adamwill 14:59:45 adamw: adamwill 'Adam Williamson' 14:59:52 CURSE YOU, WARSKI 14:59:59 CURSE YOOOOOOOOUUUUUUUUUUU 15:00:33 #info adamw curses Warski 15:00:38 #undo 15:00:38 Removing item from minutes: INFO by sgallagh at 15:00:33 : adamw curses Warski 15:00:41 (I'm not that mean) 15:00:44 Hello 15:01:48 * sgallagh looks for one more for quorum 15:01:56 .fas tuanta 15:01:56 tuanta: tuanta 'Truong Anh Tuan' 15:02:02 hello 15:02:20 * tuanta is here 15:02:36 Ok, let's get started and folks can filter in as they do 15:02:44 #topic Server Role PoC 15:03:03 * jzb lurking 15:03:20 Last week, we left off with a tentative plan to have simo, nirik and I meet up to look into hacking up a proof-of-concept straw-man for the Domain Controller ROle 15:03:30 That meeting did not happen, but I've been hacking on it anyway 15:03:40 cool. yeah, I have been busy... sorry. 15:03:50 cool 15:04:06 anywhere we can see? 15:04:52 * mizmo-out here sorry 15:04:59 sigh true 15:05:14 adamw: It's still pretty rough, but I'm going to put up my WIP either tonight or tomorrow, depending on time 15:05:44 I'm just hacking up a D-BUS service with dbus-python that implements a rough set of the interfaces we want 15:05:54 excellent! 15:07:08 cool. 15:07:23 Might not be the worst idea for us to discuss the preferred interface today, then 15:07:51 * adamw prepares to nod intelligently 15:07:53 sgallagh: do you have an idea what the API should look like yet ? 15:08:08 I'm looking at base classes that will abstract a set of common methods that the individual Roles need to implement 15:08:37 The two primary ones I'm currently calling PreloadRole and DeployRole, which is basically the package-installation and config-and-start APIs we've discussed before 15:09:47 I've also got GetFirewallPorts which is a thin wrapper around getting the firewall properties from the D-BUS object 15:10:17 My thought is that the Role should only tell us what it wants, and that we should have a separate interface to firewalld to check whether it's available and on what interfaces 15:10:39 (I think that agrees with mitr's statements in the past as well) 15:11:03 sgallagh: sounds ok to me 15:11:03 sgallagh: I wouldn't mind roles building a facade over firewalld 15:11:03 You know what, give me two minutes, I'll push this to github now (warts and all) 15:11:37 sgallagh: I'd prefer for it to refer to firewalld services though, instead of having its own port list (... which would inevitably be extended to support non-UDP/TCP IP protocols, and ICMP, and ...) 15:12:20 mitr: I thought the firewalld 'services' were hard-coded. Are they definable/configurable through the public interface? 15:12:21 I think we want to try and avoid re-implementing firewalld. ;) 15:13:03 sgallagh: /usr/lib/firewalld/services/*.xml, firewalld.service(5) 15:13:49 #url https://github.com/sgallagher/server_roles_poc 15:13:57 sgallagh: so you'd just return a set of protocol/port pairs ? 15:13:57 sgallagh: it uses a /usr, /etc, /home config override system, like is currently in vogue, iirc 15:14:17 #info mitr notes that firewalld 'services' are configurable using local files 15:14:30 simo: That's pretty much what I'm doing right now, yes. 15:14:32 sgallagh: so i think you could override the definition of a service for firewalld purposes, but it wouldn't be entirely obvious what was going on. you can certainly create new ones. 15:15:05 hm, having httpd and fedora-role-httpd would be rather confusing 15:15:19 (Be aware, the above is one day's frantic hacking with no prior knowledge of dbus-python) 15:15:19 sgallagh: PreloadRole, it's unclear to me why the caller should pass in a list of packages ? 15:15:23 Can we bet on being lucky and having the non-role and role configs be the same/ 15:15:30 simo: They shouldn't 15:15:47 simo: The child classes should call the super-class with the appropriate set 15:16:21 ok 15:17:17 mitr: Well, a role may be a superset of other services 15:17:43 i.e. the Domain Controller role is going to need to enable httpd, ldap, kerberos, dns... 15:17:59 sgallagh: right, that shouldn't be a problem ("domain-controller" != "dhcpd" or "freeipa"). I was worried about wordpress-the-package vs. wordpress-the-role 15:18:22 Wordpress-the-role is something I don't want to deal with immediately :) 15:18:31 sure :) 15:19:17 are we sure we're only going to have precisely one possible firewall configuration per role? (just thinking about the name of GetFirewallPorts) 15:19:38 trying to think of cases where the names might be too generic and we wind up having to rename them or have misleading ones later 15:20:04 adamw: Can you come up with an example? I don't follow. 15:20:19 adamw: I do think we should have a single "object that can be enabled" per role, yes. The list of the ports might change depending on role configuration. 15:20:24 This is really just the list of ports the Role is listening on that it doesn't consider purely internal 15:20:30 adamw: no we are not 15:20:42 sgallagh: freeipa server without DNS or CA 15:20:43 i'm just thinking it's the kind of thing where maybe we work out later that some deployments of a role might want port X open but others not, or something like that 15:20:47 == different set of port 15:20:49 s 15:20:56 a case where we have, I dunno, MandatoryFirewallPorts and OptionalFirewallPorts, or something... 15:21:02 right, good example. 15:21:04 simo: Yes, but the set of ports wouldn't be different on the same deployment 15:21:16 sgallagh: ? 15:21:20 mm, true 15:21:21 simo: Would the user want to enable/disable the _firewall ports_, or the functionality behind them? 15:21:22 Sorry, I see the disconnect here 15:21:38 mitr: both go hand in hand 15:21:41 The idea is that GetFirewallPorts is the *current* state of needs for the Role. Not a fixed value. 15:21:57 If the Role has optional components that open additional ports, this property gets updated to include it. 15:22:05 sgallagh: ok and that is determined how ? Do we store some data or the function is expected to do some probing ? 15:22:10 (if and only if that option is in place) 15:22:16 I guess probing is the only sane way 15:22:25 No 15:22:28 no ? 15:22:41 how do you know if you need to open DNS ports ? 15:22:43 Part of DeployRole needs to update the set of ports at the end of operation 15:22:44 simo: We are supposed to have a remote configuration API, so we need to have some local configuration storage (native to the underlying package or supplied by the role) 15:22:57 sgallagh: it would go out of sync *very* quick 15:23:00 and by extension whatever function we have that deploys optional components will need to update that 15:23:02 simo: Why? 15:23:15 sgallagh: because in freeipa services are controlled via LDAP 15:23:30 I can change the list of services for a server from a completely different server just writing to LDAP 15:23:42 simo: can't the role query the same LDAP? 15:23:46 ah, right. I forgot about that... 15:24:09 Yeah, so in the Domain Controller case, we'd probably want to override the superclass and implement a proper lookup. 15:24:39 Of course, that means we can't push a PropertiesChanged signal on port changes 15:25:11 sgallagh: we could have a persistent search but it seem kind of a waste of resources 15:25:25 sgallagh: also the change is not immediated 15:25:41 we should probably just check if named is running or not 15:26:00 or even better if netstat says someone is listening on 53 15:26:04 simo: Could we hook up the role to the freeipa reconfiguration mechanism (so that it pings the role at the same time that it detects the change and starts/stops named)? 15:26:17 simo: That doesn't help us if some other process (or attacker) took that over... 15:26:27 mitr: do you want to write plugins to freeipa's LDAP server ? don;t think oit makes sense 15:26:48 sgallagh: if an attacker was able to open 53 it means he got root and it is already game over 15:26:57 (cockpit is enthusiastic about the GUI immediately reflecting changes that are done by non-cockpit tools) 15:27:10 simo: Sure, but that won't be guaranteed for the general case 15:27:14 mitr: then we should listen on netlink to see if nmaed opens port 53 15:27:17 simo: If we want the firewall managed by the role, and the role needs to know if the configuration changes... 15:27:17 and react to that 15:27:26 We might have a > 1024 port for some other Role 15:27:30 mitr: the role knows 15:27:37 sgallagh: we could write data out of ipactl 15:27:48 ipactl decides what services to start or not 15:27:48 simo: OK, listening on netlink works - we need the role to notice, _how_ it notices is role-specific implementation 15:27:59 sgallagh: so I guess we can have a local list 15:28:09 simo: Question about changes to the service list: does it only take effect on ipactl restart? 15:28:13 we just need to make ipactl update it when it strats/restarts services 15:28:19 sgallagh: yup, so far 15:28:32 ok, then we could probably hook in there. 15:28:35 Yes 15:28:59 (We don't actually need a local _copy of the list_ - we can live with the only copy with LDAP; we just need to know when a change either happens or takes effect) 15:31:12 Ok, so let's back up and talk about the set of interfaces we need first 15:31:23 And try to avoid digging into the implementation for a little while :) 15:32:00 sgallagh: 1) is DeployRole intended to be idempotent? 2) Is this API an implementation detail or the API remote callers would be interacting with? 15:32:42 If the latter, I'd expect DeployRole to e.g. deal with the firewall already; the API shouldn't be asking for a 10-step checklist 15:32:51 1) Good question. I haven't thought about that yet. 15:33:23 2) I'm expecting this to be the API that fedora-role-manage CLI and Cockpit would talk to 15:35:02 mitr: Yes, we may want to have DeployRole include at least an option to have it go and tell firewalld to give it the ports. 15:35:30 But as we'll still want to be able to manipulate the firewall configuration later, I figured being able to retrieve the desired ports was valuable 15:35:39 yes 15:35:56 I'm not sure if we can guarantee that DeployRole is idempotent, because that's going to depend heavily on the underlying tech 15:36:16 sgallagh: so I am trying to understand what DeployRole would do in the freeipa case 15:36:28 sgallagh: our installation script is allowed to ask questions and requires inputs 15:36:33 simo: Essentially it would call ipa-server-install 15:36:34 how would that work ? 15:36:44 ... actually, the _API_ is perfectly fine. For implementation, we might want the superclass to handle the firewall automatically (depending on a setting that is shared by all roles) - DomainControllerRoleObject would not touch the firewall, but set a property indicating the default state, and the API implementation of DeployRole would modify the state depending on both settings and the desired default state. 15:36:55 We would feed it the necessary information for an unattended install of freeipa 15:37:24 sgallagh: so you have to do all the detection ipa-server-install normally does and then ask questions if it fails ? 15:37:29 ... if the sufficient information is available in the settings. If it isn't? In a TUI we can just ask on the console, for the GUI - we fail? 15:37:35 or you just ask questions upfront ? 15:37:54 simo: Upfront was the plan 15:38:04 We'll be able to get certain information ahead of time 15:38:04 Does that mean that we need an API describing the expected settings for each role, or would it be only documented? 15:38:13 like the hostname 15:38:20 I was thinking some questions can be popped back up as 'property changes' and the listener will turn them into requests to the user and feed back the property with the data 15:38:23 but sounds hackish 15:38:30 ... That's probably strongly related to the configuration reading/writing API 15:38:56 mitr: RE: expected settings. That's one of the things we need to discuss, clearly. 15:39:01 sgallagh: so I guess what we need is to provide an API that tells what data is needed ? 15:39:02 Asking for a property is not really a property change - the question would be a property change from what to what? 15:39:34 mitr: data_missing turns from False to True :-) 15:40:07 I'd rather see there be a "conversation" API, if that's going to be common. 15:40:09 simo: I'm not 100% sure, we might only need documentation. Do we expect a "generic role deployer" UI that would need the metadata? OTOH having metadata would allow us to validate the deployment settings in the generic layer 15:40:45 sgallagh: Doing conversation APIs for GUIs is hard. PAM kind of did it but one-question-at-a-time, which is already fairly awkward. 15:40:55 Please, let's not reimplement PAM 15:41:15 well 15:41:27 for freeipa we know what questions we are going to make 15:41:29 The plan of record from previous meetings was that since we expect to have a relatively small number of official Roles, it's acceptable to have unique, statically-typed deployments for them 15:41:34 I'm inclined to punt on GUI, and only allow role-specific GUIs that completely fill the deployment data; for TUI, we can have a conversation API, or just require interactive stdin/stdout and let the role do whatever it wants. 15:41:44 the problem is developing a probe that only tells back what's missing and what we discovered 15:42:02 sgallagh: I think this will require some upstream work, but shouldn't be a lot of work 15:42:15 simo: What parts of "discovery" are mandatory? 15:42:31 Because I'm not sure we should ever be relying on discovery in a Role deployment 15:42:35 simo: You mean that the set of attributes that need to be supplied for the deployment varies between computers/situations? 15:42:42 In that environment, we should be able to get that information ourselves 15:42:48 for the server we need to find out hostname, and then we need to give back the defaults we derive from it 15:42:58 although you could simply recalculate them as well 15:43:02 simo: That doesn't have to be the same channel 15:43:07 we haven't discussed what to do for a replica 15:43:50 mitr: well if the hostname is 'foo' it is not good (not fully qualified) so you'll have to ask for stuff and change it on the machine 15:44:01 or just refuse to deploy and return an error that the hostname is no good 15:44:09 simo: The latter would be my preference 15:44:37 Whatever console we're using to do deployments will have to be capable of detecting and making hostname changes. 15:44:38 Well... It Would Be Cleaner(tm) to just make it impossible to install with such a host name :) 15:44:41 We can safely assume that 15:44:50 but once we have a hostname we derive a number of properties -> domain name, REALM name 15:45:01 then we need to ask for passwords for admi and DM user 15:45:12 and we need to ask whether they want to install DNS or CA 15:45:27 for DNS that means also need to install additional packages in some cases 15:45:40 I think this is probably all doable in a static interface though 15:45:48 as long as the required inputs are available 15:46:01 FYI, just to bring this back up: http://sgallagh.wordpress.com/2013/12/09/proposal-freeipa-role-for-fedora-servers/ 15:46:03 the main issue being to stay in lockstep with upstream should we change something 15:46:35 We can have optional requirements (you need to supply $foo if $bar is enabled); that does call for a ValidateSettings method though 15:46:52 simo: The CA isn't optional anymore, I thought 15:46:58 Or do you mean chaining it to a higher CA? 15:47:27 DNS at least I would say we shouldn't have as part of standard deployment and do that as an optional add-on after 15:48:14 mitr: Yes, I think we do want a ValidateSettings method for pre-testing the responses before we actually call DeployRole 15:48:59 sgallagh: we have the option of using the IPA CA or no CA at all 15:49:09 in the second case you need to provide certs yourself 15:49:36 sgallagh: DNS as part of inistial install is actually quite critical 15:49:37 simo: I'd suggest that we opt not to support that case in the Role 15:49:57 sgallagh: well we can add features later 15:50:00 And always use the built-in CA with the ability to chain it to a superior CA 15:50:08 (Right, I meant initially) 15:50:08 imo DNS must be a default option that you turn off if you do not want it 15:50:23 simo: I can work with that 15:50:57 sgallagh: what about replicas ? 15:51:10 so in general terms we're kinda saying a role should have only one possible deploy-time configuration? 15:51:20 simo: I somewhat wonder if a replica should be treated as a separate role 15:51:36 the installation method is still a 2 step process, but if we can use ssh we can make it look like something you run only on the replica 15:51:42 adamw: Yes, we've been saying that from the beginning 15:51:53 With possibly optional components being available as an add-on later 15:52:13 * nirik is following along, but also dealing with email piles. ;) 15:52:47 simo: Let's ask a more general question first: 15:52:59 sgallagh: huh? Optional components are a different configuration. Do you mean that we would ask the user to do a two-step deployment, "deploy role" and then "add this component", or something else? 15:53:05 "Do we want to address the case of replicas in F21 or just focus on the primary DC?" 15:53:31 mitr: Yes, that's what I meant. A UI might hide that as an implementation detail of course. 15:53:40 sgallagh: well replicas are quite easy, if we can I would do that 15:54:04 but won't cry if we find out we have no time to get replica role management in 15:54:14 sgallagh: In that case I'd rather have the settings with conditional/optional parts, and have the role abstract the multiple steps required away (the same way it abstracts setting up 3 daemons already) 15:54:25 Well, as you noted, they require action on both the existing DC and the pending replica 15:54:40 sgallagh: the thing is, one poart of the role imo is detecting if there are conflicting servers on the network already 15:55:01 sgallagh: Having a domain controller as a SPOF is kind of awkward, so having replica support would be desirable - but probably not blocking 15:55:06 mitr: But there will be Roles where we'll want to install pieces later on. We need to address that too 15:55:08 so from there to telling a user: we found X existing, do you want to install a replica for X domain ? 15:55:16 is not a big deal 15:55:37 sgallagh: To me that's a redeployment with changed configuration (.. .in the "DeployRole is idempotent" model) 15:55:45 simo: Yes, but can we do that entirely from the new replica machine? 15:55:49 sgallagh: I would never want to install packages without changing the "Settings" 15:55:50 sgallagh: indeed adding DNS or CAs to an existing replica is one of the cases we 'should' support 15:56:15 sgallagh: well assuming we can ssh into one of the existing servers on the network, then yes we can drive it 15:56:23 simo: I'm not sure we can assume that 15:56:29 sgallagh: the main annoyance being the need to ask for 3 passwords I guess :/ 15:56:40 sgallagh: cockpit needs ssh access 15:56:50 sgallagh: I thought we assume ssh is available 15:57:08 hmm 15:57:14 sgallagh: well I guess cokpit could be used instead of ssh too I guss 15:57:18 *guess* 15:57:19 ... and if cockpit needs ssh, we might make it a role requirement? It wouldn't make me _completely_ happy security-wise because there's no practical way to constrain what ssh can do 15:57:38 mitr: cockpit can use its http port too I think 15:57:50 ssh would be available I would think, but it might also be key only or the like, so you couldn't actually use it from another machine. 15:57:58 mitr: what ssh can do depends on what the user sshing in can do 15:58:17 nirik: true 15:58:21 nirik: Well, we're presuming a FreeIPA deployment here, which by default means GSSAPI 15:58:25 nirik: but then we just fail to install 15:58:35 * nirik nods. all true. 15:58:45 sgallagh: except we need access as root for replica installation in this case and root is not kerberized 15:58:56 crud 15:59:16 I think in this case, we may just want to deal with it via documentation 15:59:33 well we could use admin's creds to set up sudo rules :7 15:59:33 "You want to add this machine as a replica? You need to perform the following steps on an existing master first..." 15:59:38 yeah, inter-host communication could get tricky 16:00:03 simo: Why does the replica-prep need root? 16:00:11 simo: Pie-in-the-sky: master providing an appropriately authenticated API to add a replica? Or is that just moving the problem to a different port? 16:00:32 mitr: That's where I was going too 16:00:47 sgallagh: needs to copy certs and other secrets 16:01:21 mitr: I've been thinking of how to do that for quite a while, but it will take long to do upstream, won't be done for F21 16:01:35 BTW, we're over the hour. Do we want to continue? 16:01:36 sgallagh: I think the prep thing for now is fine 16:01:58 sgallagh: only if we have things to discuss ? 16:02:05 I have another mtg in 1 hour 16:02:07 sgallagh: I'm not sure whether it was intentional, but the invite is for 2 hours now 16:02:20 so if I can take a lunch break now I'd like it 16:02:35 I did that because we've been going over for a while. I figured I'd block out the realistic time 16:03:13 I think we have a lot to discuss RE: the API, but maybe we need to open an email thread first 16:03:25 yeah, i have trouble following this kinda thing on IRC. 16:03:44 sgallagh: I think we sorted out the basic here, mail thread seem more productive now 16:03:44 adamw: You're not alone :) 16:03:49 and more inclusive 16:03:59 we shouldn;t really do development in these meeting only steer it 16:04:15 Ok, I'll try to summarize what was discussed here and start a discussion on the list 16:04:23 * nirik nods. 16:04:25 ack, thanks a lot 16:04:28 * simo runs 16:04:49 #action sgallagh to summarize API design discussion and start discussion on the server list 16:04:59 #topic Open Floor 16:05:09 Anyone have other items to discuss before we adjourn? 16:08:12 Ok, thanks everyone 16:08:18 #endmeeting