15:00:23 #startmeeting Fedora Base Design Working Group (2014-09-05) 15:00:23 Meeting started Fri Sep 5 15:00:23 2014 UTC. The chair is pknirsch. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:23 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:34 #meetingname Fedora Base Design Working Group 15:00:34 The meeting name has been set to 'fedora_base_design_working_group' 15:00:47 Good morning/afternoon everyone! 15:02:04 So who we got today? 15:02:29 #chair masta dgilmore haraldh vpavlin|afk msekleta 15:02:29 Current chairs: dgilmore haraldh masta msekleta pknirsch vpavlin|afk 15:02:44 Jreznik already mentioned he won't be able to make it today 15:03:12 hey everyone! 15:03:20 * masta is here 15:03:33 <- 15:03:51 * dgilmore is here but needs to relocate 15:06:00 Ah, and hy poetteri1g :) 15:06:05 s/hy/hi/ 15:06:26 pknirsch, your completion for nicks seems to be broken :) 15:06:50 :/ 15:06:53 ah well 15:07:31 alright, lets get started though, such a nice Friday evening here :) 15:07:43 #topic Lennart's proposal discussion, especially in relation to /etc 15:07:52 yeah, 17:00 friday is not a nice time for a meeting 15:07:55 #info http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html 15:07:59 esp. with that weather :) 15:08:10 #info http://0pointer.net/blog/projects/stateless.html 15:08:12 aye 15:08:14 agreed :/ 15:08:34 but i was hard enough to find a time where most people could actually make it 15:08:50 missing festi 15:09:14 i just pinged him 15:11:25 hrm 15:12:43 Well, maybe lets quickly start with msekleta's update about systemd-216 and docker first maybe? 15:12:53 so, yeah, on a side note: you can download experimental btrfs images from my server: http://particles.surfsite.org/ 15:13:25 ah 15:13:27 oh, ffesti_ uses IPv6 :) nice 15:13:28 there he is :) 15:13:55 sri 15:13:57 pknirsch, will do topic first or should I go ahead? 15:13:57 Alright, then lets stick with the current topic 15:14:04 pknirsch, ok 15:14:05 already one half into the week end 15:14:09 :) 15:14:30 in those btrfs images, I copied /etc to /usr/share/factory and did also the sbin/bin merge 15:14:34 Thanks for joining us today ffesti_ and poetteri1g 15:15:13 Hi all, I am sorry to be late:) 15:15:14 and I removed the rpm database 15:15:16 So what exactly would we want to discuss today and whats the goal. 15:15:21 because /usr is read-only anyway 15:15:45 I suppose it's mainly about the rpm related changes, right haraldh ? 15:15:51 (thats why we have ffesti_ ) 15:15:53 I would want to persuade ffesti_ for the %config 15:15:57 yep 15:16:15 ffesti_, did you have time to read poetteri1g 's document ? 15:16:18 so, instead of manually modifying all spec files 15:16:31 link, please 15:16:37 http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html 15:16:48 I would like to use the %config attribute 15:16:55 ffesti_, I see you are not prepared 15:17:10 ffesti_, and you always want documents 15:17:13 ok, I already read this one 15:17:18 ah, ok 15:17:36 no, idea how many other documents you guys have 15:18:10 What should be stressed is that poetteri1g would like to have configs installed into both - /etc and /usr.../factory/etc. And only on demand (by some rpm option) keep /etc empty 15:18:12 I still didn't understand how this stuff is related to the stuff the Base WG is after 15:18:28 i think http://0pointer.net/blog/projects/stateless.html is more relevant to the rpm and /etc stuff, right haraldh ? 15:18:35 right 15:18:59 ffesti_, well vpavlin wanted %config to be in extra rpms 15:19:12 ffesti_, so, I thought, that it is the same problem 15:19:27 try to think harder 15:19:47 ffesti_: Ok, there are 2 unrelated problems - one is that users/customers want to be able to install their configuration in packages; second is that we would like to have vendor configuration also in /usr 15:19:56 * pknirsch nods 15:20:39 haraldh: I think it is not the same problem - I thought we could solve the first problem by solving the second one, but as we would need to also track files in /etc it wouldn't help 15:20:48 i mean, to make this clear, the btrfs stuff i posted, should have no effect on fedora for now 15:21:11 poetteri1g: But (hopefully) will have in the future 15:21:13 the stuff we are proposing there is in a really early phase, with exceptions of harald's images we have no code at all for this 15:21:34 so this posting was simply a heads-up, that we want to do this, and i would love to fedora to make use of this 15:21:44 poetteri1g, well, we have code for sysusers and tmpfiles 15:22:09 but at this point we are not propsing this. when the time comes, we will file feature page, yaddayadda, so that fedora can decide whether they want this stuff 15:22:10 as you might have seen I posted a few question with regard to http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html 15:22:16 haraldh: true, but that's slightly different 15:22:17 and preparing the base base packages for stateless would be a nice step 15:22:19 on the discussion we had 15:22:41 let me see who actually was on that 15:22:50 haraldh: i mean the stateless stuff is much further ahead, and while a building block for the btrfs stuff, evry useful outside of it already, including for atomic and stuff 15:23:11 ffesti_: where did you post the questions? 15:23:32 basically Harald and Vaclav... 15:23:34 so, yeah, this should be more about http://0pointer.net/blog/projects/stateless.html than the other blog 15:23:39 i replied to a number of them on lwn and on g+, but eventually stopped, i ended up repeating myself ;-) 15:24:20 yeah, so regarding that stuff, what i'd really like fedora to do, is adopt the sysuers stuff, and the atomic guys want that to 15:24:25 there's an fpc bug for that 15:24:34 that i haven't updated recently, i probably should though 15:24:43 and the other thing is the /usr/share/factory thing 15:24:54 and i'd really love if we could nicely integrate this into rpm 15:24:56 vpavlin wants %config packaged by the user, so rpm somehow has to exclude %config at installation for him 15:25:13 I want %config also in /usr/share/factory/etc 15:25:14 so that %config for example would have the effect that it places all config files into /etc as well as /usr/share/etc 15:25:17 meh 15:25:19 and I don't care about /etc 15:25:20 /usr/share/factory/etc i mean 15:26:02 well, there are two different concepts in rpm that might be of interest here 15:26:21 %config which is basically a way of treating files on updates (and verify) 15:27:16 and %{_sysconfdir} with is basically a pointer to /etc which could be moved to something else 15:27:43 You guys first have to make up yopu mind what kind of files you are talking about 15:27:55 well, i'd be particular interested in if we wouldn't have to update all the spec files to place the conf files at two places... 15:28:01 my guess is that it is the files in /etc 15:28:11 ffesti_: yes 15:28:24 well, I can live with /usr/share/factory/var also 15:28:39 If some packages need that, too 15:28:54 well, %config files may be very unlike "config files" 15:28:58 like logs 15:29:17 ffesti_: i can see that there is a distinction between the sysconfdir path and %config, but i do wonder. how many specfiles use %config for non-etc files? and how many use %_sysconfdir without %config? 15:29:20 then, that's the wrong attribute :) 15:29:48 and how many do not use %_sysconfdir but /etc 15:29:59 grep is your friend? ;) 15:30:03 as I said %config is just a way of treating files that works for config files 15:30:06 but also others 15:30:21 it does not carry the same semantics as you are using 15:30:52 my guess would be that the directories in /etc are not %confg 15:30:54 ffesti_: but at least rpmlint complains if %config is used on non-sysconfdir files right? and vice versa it complains too, if you have files in sysconfdir without marking it %config? 15:32:24 as I already said, logs and databases (rpmdb) are marked with %config 15:32:48 i mean, i am happy either way, if we can extend %config to also mean that we copy the file in question to /usr/share/factory/$path, or if we can somehow add a logic that also copies all files placed in /etc into /usr/share/factory/etc/... 15:32:53 %config should probably be renamed then :) 15:33:29 "without altering every spec file" 15:33:42 yeah, i'd really prefer if we didn#t have to alter anything... 15:33:43 ffesti_, that would be the best case :) 15:33:58 then keep you fingers from the basic stuff 15:34:11 i mean, another option is to actually just change the rpm install routine, so that it does this even for old rpms... 15:34:25 like the semantics of the rpm attributes 15:35:23 There has been a lot of restistance against adding too much magic to rpm 15:35:25 yeah, rpm already creates new file with *.rpmold *.rpmorig *.rpmnew 15:35:57 the question you have to ask is how to fix stuff if your magic doesn't do the right thing 15:36:23 so if rpm just copies the files in /etc you better make sure that there is no case where this isn't acceptable 15:36:53 like for 3rd party SW in /opt ? :) 15:37:08 *??* 15:37:41 say you have a vmware rpm, which installs to /opt and /etc 15:37:56 btw..let's say we let rpm to install all %config also to factory/ - how are we going to solve filelists? Should we track/list both locations? Just one? Which one? 15:37:57 you do not want to copy this part to /usr/share/factory maybe 15:38:35 vpavlin, in my world, rpm would know about the factory files 15:39:17 and in my world, we would have a switch to turn this feature on on rpmbuild 15:39:43 which should only be used by Fedora (the vendor, which provides /usr) 15:39:53 Maybe an option would be to have a rpm plugin that does this? 15:40:02 pknirsch, exactly 15:40:08 well if you do the copying to /etc in systemd you can just alter %sysconfigdir 15:40:15 and anyone can write that *hint* 15:40:22 :) 15:40:30 pknirsch, not really.. how do you do that? 15:40:32 vpavlin: i think rpm should track both copies, but make clear that the version in /usr/share/factory/ is not actually a %config instance, but the one in /etc is 15:40:42 pknirsch, can something manipulate the filelist? 15:40:52 hmm, same as debuginfo? 15:41:11 for example, yes 15:41:22 so, same as debuginfo, we would then have -config 15:41:48 I don't know how to realize that 15:42:09 esp. that the %config attribute should not exist for the /usr/share/factory files 15:42:12 ffesti_: how does the debuginfo stuff precisely work? how does it extend the file list? 15:42:32 it is a hand coded hack 15:42:49 it does not extend the file list but creates a new one 15:43:02 "%config %sysconfigdir/.... " is wrong for sysconfigdir==/usr/share/factory/etc 15:43:21 we don't want/need *.rpmsave *.rpmnew in /usr/share/factory 15:43:36 ffesti_: is that "hack" available for other things than debuginfo, or is this specifically hooked into rpm for debuginfo? 15:43:50 so, just setting %sysconfigdir to /usr/share/factory/etc does not work 15:44:00 we added a plugin interface for doing such stuff 15:44:12 but I have not used it myself 15:44:14 ffesti_: if there'd be way how from some rpm spec file construct we could manipulate the file list, this would be all we need for this 15:44:19 Panu came up with it 15:44:20 is there documentation available for the plugin interface, ffesti_ ? 15:44:52 Docs? Really? 15:44:56 yep :) 15:46:01 so, there are none? 15:46:27 the thing is that the plugin interface is not available to the spec files 15:47:09 but for implementing extensions within or close by rpm 15:48:12 one could probably implement a plugin that just copies all files in /etc to /...factory and adds them to the file list 15:48:24 ffesti_, correct 15:48:25 while stripping them of their attributes 15:48:28 ffesti_, correct 15:48:38 you might have it be controlled by some macro 15:48:44 and I haven't seen anything like that for rpmbuild 15:48:51 That sounds like something that might actually work:) 15:49:00 I mean, the plugin mechanism 15:49:14 but maybe I should look closer again 15:49:16 ffesti_: actually, if we can make it happen without a macro, that's even better 15:49:37 poetteri1g, system wide switch 15:49:39 well, we'd want to have a macro to disable it 15:50:04 ffesti_: can you tell me what i need to look for for this? googling "rpm plugin" is not enlightening... 15:50:08 haraldh, right, AFAIK the plugin interface concentraces on the rpm side of things 15:50:09 I would make it a global switch (default off in rpm, but on for fedora builds) 15:50:30 haraldh, right 15:51:08 ~/git/rpm/build (master)$ pwd 15:51:08 /home/harald/git/rpm/build 15:51:08 ~/git/rpm/build (master)$ git grep plugin . 15:51:08 ~/git/rpm/build (master)$ 15:51:36 ~/git/rpm/doc (master)$ git grep plugin . 15:51:36 ~/git/rpm/doc (master)$ 15:51:51 I'll ask Panu next week and try to come up with some pointers 15:52:45 He generally was not very fond of allowing such magic to enter into the build process 15:52:58 rpmts.h and rmtypes.h does mention "plugins" actually, but it's not very self-explanatory ;-) 15:53:13 as this messes reproducibilities of builds up fasr beyond the current state 15:54:38 there is lib/rpmplugins.h 15:55:01 but t looks like it is concerned with rpm only 15:55:45 ffesti_, when I looked into rpmbuild, I couldn't any plugin mechanism, thus I wrote a patch :) 15:55:49 http://paste.fedoraproject.org/131308/93245914 15:55:59 which was a proof of concept 15:56:05 and seems to work for me :) 15:56:08 yup, looks like there is no plugin in rpm right now 15:56:47 I know there are some patches out there that go into this direction but as I already said they were not too well received upstream 15:57:38 since June you guys haven't been very helpful, though :-( 15:59:42 well, I am not talking about your patch 16:01:41 ok, so lets assume we'd be willing to include something like you patch 16:01:59 is there backing for this from the Fedora side of things? 16:02:09 So I am not sure...do we want to do this "add to filelist" "magic" to rpmbuild or rpm itself? 16:02:46 order of my preference: 16:03:01 well, there is still another option: Add it to the f****** spec files 16:03:05 1. magic plugin shuffling around the filelist and make a -config subpackage 16:03:14 2. rpmbuild hardcoded 16:03:19 3. rpm at installation time 16:03:36 4. manually modify _every_ f****** spec file 16:04:03 3.5 beat ffesti until 1. is happening :) 16:04:36 5. make systemd copy the files from /etc 16:04:49 the modified ones? 16:04:51 haraldh: Where the -config subpackage would install to both /etc and factory? 16:04:54 systemd != rpm 16:05:10 systemd does not inside a chroot'ed installation root 16:05:18 vpavlin, no.. only /etc 16:05:43 haraldh: How would they end up in factory then? 16:05:46 haraldh, that does not work 16:05:51 ? 16:06:04 the magic plugin would add the factory ones to the main package 16:06:13 and the /etc ones to a -config subpackage 16:06:36 haraldh: Ok, that makes sense 16:06:41 and maybe add a "Wants: *-config" to the main package 16:06:42 ffesti_: systemd wants the pristine vendor versions of the config files in /usr/share/factory. this means we need to get the data from the original rpm, and nothing we end up copying from /etc 16:07:17 I like haraldh's 1. :) 16:07:38 if we ship multiple versions of the config file there is no "pristine vendor version" 16:07:53 ? multiple versions? 16:08:06 ffesti_: You mean Fedora Products? 16:08:08 in different -config packages 16:08:26 I have no clue - you came up with that idea 16:08:27 do we ship them? well 16:08:49 I thought that those were meant to be installed by the user/admin 16:08:49 I am just trying to make sense of this 16:09:15 yeah, but wouldn't they then be part of the /usr tree, too 16:09:21 the different ones, if the admin removes the vendor -config subpackage 16:09:34 what? 16:09:42 /usr is vendor only 16:09:46 and RO 16:09:50 normally 16:10:05 ffesti_: *-config subpackages I was initially talking about would be created by users - let's say you have 1000 servers and want to do the same change in config to all of them - you just create/update *-config package and update from you private rpo 16:10:26 so if an admin did choose to not install the -config subpackges, he can install his own version as an rpm 16:10:28 why would you need conflicting sub packages then? 16:10:56 but is /usr is readonly you cannot remove the vendor config 16:11:02 ? 16:11:11 how would you install your own config 16:11:16 ? 16:11:19 to /etc 16:11:20 ffesti_, to /etc 16:11:28 ffesti_, Schlauch? 16:12:03 ok, ok, I start getting the picture 16:12:26 ffesti_: dir /usr will always contain vendor conf and users are not supposed to mess with it 16:13:27 the -config subpackages supplied by the vendor are ... well.. suggestions :) 16:13:29 ffesti_: But they would be able to create -config package which would conflict vendor shipped -config package (which contains the same content as the main package but installs to /etc instead of factory/etc) 16:13:46 the user config in rpms model stioll doesn't fit together with this read only /usr thing 16:14:01 ffesti_: why? 16:14:15 ffesti_, yeah, adduser and rpm filelist are ugly 16:14:20 adduser in %pre 16:14:34 or what do you mean? 16:14:39 because the whole system becomes basically read only 16:14:44 why? 16:15:07 mymysql-config.rpm might contain only files in /etc and /vare 16:15:09 mymysql-config.rpm might contain only files in /etc and /var 16:15:15 where is the problem? 16:15:33 ok, you want to do this with a writable rpmdb 16:15:39 sure 16:15:58 in our btrfs model, we don't even have rpm in the end system 16:16:12 it's just for assembling and developing systems 16:16:18 but here 16:16:22 ffesti_, and why does one care, having /usr readonly is the point and you are not installing rpms once you have pristine image 16:16:31 for vpavlin's usecase, we have a writeable /var 16:16:45 and the rpmdb in /var 16:18:20 ok, but you need having the files in /etc in separate packages for this to work 16:18:33 ffesti_, correct 16:18:42 so just copying the files won't cut it 16:18:46 yep 16:19:13 and because the unix world is so consistent with config files 16:19:21 we probably want all %config 16:19:28 * vpavlin thinks we are finally all on the same page:) 16:19:29 some services have config in /var 16:20:01 \o/ 16:20:31 so we need some complex heuristic and then a way to tweak it 16:21:34 well, I would copy all /etc and /var we ship to factory 16:21:40 doesn't hurt probably 16:22:20 and always good to know, what the files have been originally 16:22:29 # rpm --resetconfig httpd 16:22:43 bam! 16:22:48 back to factory 16:23:20 # rpm --resetvar rpm 16:23:21 :) 16:23:41 ok, now back to my original question: Assuming we'd be willing to implement this, is there the necessary backing in Fedora? 16:24:03 ffesti_, you mean fesco? 16:24:32 probably 16:24:57 ffesti_, would you be willing to add an rpm plugin, which does the --resetconfig? 16:25:14 and a --diffconfig ? 16:25:18 ffesti_: not officially to my knoweldge, we need to figure out the details yet, but there's interest from various sides, as you can see in this meeting, after all this is the feora base weg. but atomic wants this stuff too (the factory thing is an idea colin came up with actually) 16:25:38 and i want to push this too from the systemd side 16:25:53 but before we can have fesco agree on something we first have to propose something 16:26:13 I think we can get support with a "rpm --resetconfig" "rpm --diffconfig" feature 16:26:16 and for that we need to know how to get rpm hooked up to this nicely.... 16:26:18 every admin would love that 16:26:37 which can easily be implemented 16:26:40 yup, I still have some doubts that we really fully understand if this is the right thing to do 16:26:41 * msekleta nods like crazy 16:26:57 in every detail 16:27:23 ffesti_, you will never know, if you don't try :) 16:27:39 * vpavlin nods now :) 16:27:53 well, if we only change rpmbuild, then this is really very easy to revert 16:27:54 I'd still say we should formulate this as a first REAL proposal and propose that as a feature. otherwise i agree with ffesti_, if we do this now and Fedora in general doesn't want it all the work would be wasted 16:27:58 just rebuild some packages 16:28:18 But imho it would be something very useful 16:28:54 and it seems most of us here agree, and not that that's a significant representation, but it's still a reasonable indicator that this isn't stupid 16:29:07 well, adding support for creating sub packages from some kind of template in rpmbuild is more than just rebuild some packages 16:29:10 and as always, perfect is the enemy of good 16:29:42 just like SCLs weren't perfect by any means or measure, but it was a nice and reasonably good stop gap 16:29:53 ffesti_, well, I mean, it affects only a very limited amount of people 16:30:04 not everybody has to change his specfile 16:30:09 suddenly 16:30:34 well, adding some way to do magic during rpm build is a genie that is difficult to get back into the bottle 16:30:59 and yes, I know you guys do not care 16:31:03 ffesti_, and what is debuginfo? 16:31:23 ffesti_, yes, we only want our shiny new feature :) 16:31:24 and ugly hack no one really want to look at 16:31:33 -d 16:32:38 "rpm --resetconfig" "rpm --diffconfig" ... no more .rpmorig :) 16:32:44 come on! 16:32:57 this should be enough justification/motivation :) 16:33:04 I agree 16:33:25 my doubts are more around the user supplied -config packages 16:33:36 vpavlin, your turn 16:33:50 be cause users are going to expect that this will also works for their own config packages 16:34:01 ah 16:34:03 true 16:34:34 the way you have set things up rpm --resetconfig will do something completely useless for them 16:34:46 true 16:34:58 he will have to do: rpm -ivh --force myconfig.rpm 16:35:09 Ah.. 16:35:24 right, so why are we doing this whole factory stuff again? 16:35:42 /usr is vendor 16:35:49 if I ahve to go back to the rpm package anyway 16:35:59 now you have find a second place for the admin %config 16:36:15 /usr/local/share/factory :) 16:36:26 hmm..I think it's actually ok - you do rpm --resetconfig and then install/update to your config packages.. 16:36:52 which is actually a logical place for it 16:36:55 /usr/local/share/factory :) 16:36:57 I like it 16:37:06 makes sense 16:37:19 you could make it configurable 16:37:23 maybe 16:37:41 but only for disto vendors 16:39:04 so, in the fedora build system factory points to /usr/share/factory and in the admins build system factory points to /usr/local/share/factory 16:39:23 like with %configure 16:39:37 err s/%// 16:40:42 and rpm --resetconfig first looks in /usr/local and then /usr 16:40:42 same for diff 16:41:09 And what is in /etc? 16:41:17 if you instal admin config pkg 16:41:17 ? 16:41:32 first in /opt/$vendor/factory then in /usr/local then... 16:41:50 ffesti_, hehe FACTORY_PATH=... 16:42:34 actually packages should keep their fingers out of /usr/local IIRC 16:43:01 ffesti_, actually _vendor_ packages should only contain files in /usr 16:43:09 admin packages can be anywhere 16:43:18 that's their problem 16:43:43 _vendor_ packages should not have /usr/local or /opt 16:43:54 _osvendor_ packages should not have /usr/local or /opt 16:44:03 _vendor_ packages should not have /usr/local 16:44:18 admin packages can 16:44:20 * vpavlin will have to go in 16 minutes 16:44:33 wait a second 16:44:58 putting files in /usr/local doesn't work with your read only /usr/ 16:45:11 mountpoint? 16:45:32 /usr/local is often a mount point 16:45:38 you are just making this up right now, right? 16:45:46 what? 16:45:56 you do not really have a final layout of your system, right? 16:46:26 remember our talk about the FHS? 16:47:13 anyway, ok, you got another partition which is writable and persistant on /usr/local 16:47:58 just make sure you write this into your proposal 16:48:38 well, that's what /usr/local is for .. local modifications by the admin 16:48:56 because /usr is OS vendor supplied 16:49:14 remember? unix :) 16:49:38 I wonder if I am the only one that ready "read only /usr" as "whole /usr/ being read only" 16:50:02 I know, that's why I am so confused by you suggestions 16:50:13 +r 16:51:34 I must have missed the local: images on http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html 16:52:08 local images don't have to be produced, shipped by anyone 16:52:17 but I will read it once more later 16:52:57 apps should go to /opt ... admin can have his /usr/local or his own /opt/ 16:53:18 or we might want to get rid of /usr/local 16:53:33 and make /usr/local -> /opt/local 16:54:01 so we have a clear distinction from the very beginning / top-level 16:54:29 So we are getting 3rd proposal here..hmm.. 16:54:46 well, we can go on from here :) 16:54:55 sbin <-> bin merge :) 16:55:33 clean multilib /lib subhierarchy like debian... but multilib only.. not multiarch 16:56:09 no multiarch for bin 16:57:00 this is why we have the /usr per $ARCH in the btrfs scheme :) 16:57:09 pknirsch, still awake? 16:57:27 pknirsch, close the meeting? 16:58:18