10:05:33 <FranciscoD> #startmeeting ROS-RPM IRC session 2: RPM packaging basics 10:05:33 <zodbot> Meeting started Thu Apr 24 10:05:33 2014 UTC. The chair is FranciscoD. Information about MeetBot at http://wiki.debian.org/MeetBot. 10:05:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 10:05:43 <FranciscoD> #meetingname ROS-RPM IRC session 2: RPM packaging basics 10:05:43 <zodbot> The meeting name has been set to 'ros-rpm_irc_session_2:_rpm_packaging_basics' 10:05:50 <FranciscoD> #topic Who's here? 10:05:57 <FranciscoD> .hellomynameis ankursinha 10:05:58 <zodbot> FranciscoD: ankursinha 'Ankur Sinha' <sanjay.ankur@gmail.com> 10:06:17 <FranciscoD> If you have an FAS, you can use the hellomynameis <fas account name> to introduce yourselves 10:06:31 <FranciscoD> (It's a good idea to have an FAS) 10:06:40 * FranciscoD hopes he isn't the only one here 10:06:45 <FranciscoD> banas: el ping 10:06:47 <banas> .fasinfo sarupbanskota 10:06:50 <zodbot> banas: User: sarupbanskota, Name: Sarup Banskota, email: sbanskota08@gmail.com, Creation: 2013-03-28, IRC Nick: banas, Timezone: Asia/Kolkata, Locale: en, GPG key ID: , Status: active 10:06:53 <zodbot> banas: Approved Groups: designteam cla_done cla_fpca 10:07:04 <FranciscoD> Anyone else? 10:07:16 <banas> lol, I have to be back in 3 minutes. 10:07:18 <banas> brb 10:07:23 <FranciscoD> XD 10:07:27 * FranciscoD will wait for banas 10:08:23 <banas> back! 10:08:30 <FranciscoD> mpduty: hey, just in time 10:08:32 * banas wonders if he took less than 3 10:08:35 <FranciscoD> .fas mpduty 10:08:35 <zodbot> FranciscoD: mohanprakash 'mohan prakash' <mpduty@gmail.com> - mpduty 'Mohan Prakash' <hamariteam@gmail.com> 10:08:42 <FranciscoD> banas: you took 1 ;) 10:08:46 <FranciscoD> Fine, lets start 10:08:47 <banas> hi mpduty! :D 10:08:49 <banas> FranciscoD: yay :D 10:08:55 <FranciscoD> #topic Important links 10:09:07 <FranciscoD> There are a couple of links you should keep around: bookmark them 10:09:10 <FranciscoD> http://fedoraproject.org/wiki/Join_the_package_collection_maintainers 10:09:15 <FranciscoD> http://fedoraproject.org/wiki/How_to_create_a_GNU_Hello_RPM_package 10:09:28 <FranciscoD> (We'll walk through the hello world one today) 10:09:32 <FranciscoD> http://fedoraproject.org/wiki/How_to_create_an_RPM_package 10:09:32 <banas> cool 10:09:49 <FranciscoD> http://fedoraproject.org/wiki/Category:Package_Maintainers 10:09:56 <FranciscoD> http://fedoraproject.org/wiki/Packaging:Guidelines 10:10:14 <FranciscoD> This is almost all the info you'll need 10:10:49 <FranciscoD> In special cases, you can look up http://www.rpm.org/wiki/Docs#PackagerDocumentation 10:11:04 <FranciscoD> And of course, when in doubt, send a mail to the fedora packaging committee and clear it up 10:11:09 <banas> cool, and this is not ROS specific, correct? 10:11:11 <mpduty> hi banas 10:11:17 <FranciscoD> banas: no, just general today 10:11:23 <banas> great 10:11:25 <FranciscoD> https://fedoraproject.org/wiki/Packaging_Committee?rd=Packaging:Committee 10:11:42 <FranciscoD> Ok, if everyone will open up the "how to create a hello world rpm" link 10:11:56 <FranciscoD> #topic Preparing your system 10:12:10 <FranciscoD> Of course, to build rpms you need to have the tools installed 10:12:22 <FranciscoD> luckily, we have a "fedora-packager" meta package that pulls in most things 10:12:32 <FranciscoD> so 10:12:38 <FranciscoD> #info sudo yum install fedora-packager 10:12:55 <FranciscoD> let me know once done 10:13:30 <banas> my internet is pretty bad, but I'll keep track! 10:13:35 <FranciscoD> Ok 10:14:01 <FranciscoD> This will install the rpmdevtools package 10:14:11 <FranciscoD> this contains a handy tool: rpmdev-setuptree 10:14:15 <FranciscoD> please run the command 10:14:33 <FranciscoD> #info rpmdev-setuptree sets up the required directory structure under ~/rpmbuild 10:14:43 <FranciscoD> once you've run it, cd into rpmbuild 10:14:47 <FranciscoD> and run: tree 10:14:49 <FranciscoD> see what you get 10:15:23 * banas is still waiting for fedora-packager to install 10:16:07 <FranciscoD> It should look like this: http://paste.fedoraproject.org/96510/33450513 10:16:43 <FranciscoD> Ask me to wait if you need to. We have quite a lot of time 10:17:14 <FranciscoD> Each directory serves a specific purpose 10:17:21 <FranciscoD> That should be quite obvious, right from their names 10:17:39 <FranciscoD> We generally don't need to tinker with the two BUILD* directories 10:18:05 <FranciscoD> #topic Lets package GNU hello world! 10:18:25 <FranciscoD> The sources directory is where we keep all our source code tar balls 10:18:33 <FranciscoD> tar balls, zips, patches, extra files 10:18:44 <FranciscoD> banas: still installing 10:18:47 <FranciscoD> mpduty: following? 10:18:51 <mpduty> yes 10:18:59 <mpduty> I've done this before 10:19:02 <FranciscoD> ah, great 10:19:08 <banas> FranciscoD: yeah! I have a question, I got something weird thrown on my screen. pasting 10:19:13 <FranciscoD> heh, go ahead 10:19:53 <FranciscoD> banas: how weird? :P 10:20:06 <banas> http://fpaste.org/96511/33477713/ 10:20:36 <FranciscoD> uh oh 10:20:44 <FranciscoD> you have some messed up packages :/ 10:21:00 <FranciscoD> you'll have to manually remove the older dups and then reinstall the new ones 10:21:09 <banas> argh, okay, you carry on, i;ll try to follow! 10:21:11 <FranciscoD> This is probably as a result of an update during the selinux bug 10:21:28 <FranciscoD> banas: http://fedoraproject.org/wiki/Common_F20_bugs#RPM_scriptlets_fail_during_updates 10:21:37 <FranciscoD> *sigh* 10:21:45 <FranciscoD> Ok, doesn't matter. I'll show you the outputs 10:23:20 <banas> FranciscoD: yup, I'm running some of them :) 10:23:25 <banas> yeah please do! 10:23:50 <FranciscoD> banas: I'd let you ssh into my system on a different user, but I don't think you can. I'm behind uni wifi too :/ 10:23:58 <FranciscoD> Moving on. 10:24:03 <banas> yeah sure! 10:24:03 <mpduty> ok 10:24:05 <FranciscoD> cd into the source directory 10:24:16 <FranciscoD> wget http://ftp.gnu.org/gnu/hello/hello-2.8.tar.gz 10:24:25 <FranciscoD> we're just downloading the source 10:24:31 <FranciscoD> you can use your browser or whatever 10:24:52 <FranciscoD> Open up the tar: tar -xvf hello-2.8.tar.gz 10:24:54 <FranciscoD> take a look inside 10:25:05 <FranciscoD> More on tar at `man tar` 10:25:32 <FranciscoD> This is the source that will generate a binary for us 10:25:49 <FranciscoD> It has the usual structure that an autotools package has: configure script, make files etc 10:26:10 <FranciscoD> #topic build systems 10:26:27 <FranciscoD> What build systems do you folks know? Use info so we have a list 10:26:32 <FranciscoD> #info Autotools 10:26:35 <FranciscoD> what else? 10:26:43 <FranciscoD> #info cmake 10:26:48 <FranciscoD> #info python-setuptools 10:26:56 <FranciscoD> #info waf 10:27:07 <FranciscoD> #info catkin (ROS specific) 10:27:23 <FranciscoD> #info qmake 10:27:31 <FranciscoD> I'm sure there are a couple more 10:27:47 <FranciscoD> To build from the source, you need to know at least how to use these build systems 10:28:03 <FranciscoD> autotools looks simple, but when it comes down to patching build system files, it's a nightmare 10:28:04 <mpduty> ok 10:28:19 <FranciscoD> #topic the spec file 10:28:37 <FranciscoD> change directory to ~/rpmbuild/SPECS 10:28:50 <FranciscoD> rpmdev-newspec hello 10:29:07 <FranciscoD> rpmdev-newspec creates a blank spec for you from templates 10:29:23 <FranciscoD> rpmdev-newspec --help will show you more options 10:29:25 <mpduty> I guess the hello directory should be in the SOURCES 10:30:07 <FranciscoD> For example, you can use -t python xxx to generate a python spec template 10:30:07 <FranciscoD> mpduty: it should be 10:30:07 <FranciscoD> we don't need the uncompressed directory, rpm will uncompress it during the build process for us 10:30:17 <mpduty> ok 10:30:37 <FranciscoD> mpduty: check the output of tree ~/rpmbuild -L 2 -d 10:31:10 <FranciscoD> Open up the specfile using your favourite editor 10:31:17 <FranciscoD> vim/emacs/nano/peco/gedit/whathaveyou 10:32:00 <FranciscoD> It should look like this: http://paste.fedoraproject.org/96517/83355041 10:32:24 <FranciscoD> Now, until the %description part, everything is self explanatory 10:32:47 <FranciscoD> The buildrequires and requires is actually a little interesting 10:33:04 <FranciscoD> When you're building software from source, you might need certain libraries etc. that the source uses 10:33:18 <FranciscoD> So, since these are required while *building*, they are called "BuildRequires" 10:33:35 <FranciscoD> Similarly, some software is required to run the application once its been built 10:33:40 <FranciscoD> These are called "Requires" 10:33:46 <FranciscoD> Mostly, rpm can figure out Requires itself 10:34:01 <FranciscoD> Using logic to find the requires shared objects (library files) etc 10:34:19 <FranciscoD> However, for scripted languages sometimes, like python, we manually specify the "Requires" 10:34:23 <FranciscoD> Clear? 10:34:27 <mpduty> yes 10:34:49 <FranciscoD> The name of the specfile should be the same as the value in the Name: field 10:35:00 <FranciscoD> this is a must 10:35:00 <FranciscoD> fill up bits till %description 10:35:23 <FranciscoD> You can open up another terminal or nautilus to get info on the hello source code 10:35:26 <FranciscoD> The version is 2.8 10:35:31 <FranciscoD> See if you can find the license 10:35:58 <FranciscoD> The COPYING file or LICENSE file generally tells you the license 10:36:12 <FranciscoD> Note that the license in the headers of the source code also needs to be checked 10:36:24 <FranciscoD> There's another utility that does this for us: licensecheck 10:37:00 <FranciscoD> run `licensecheck -r .` inside the hello source directory 10:37:16 <FranciscoD> This will give you http://paste.fedoraproject.org/96521/33580613 10:37:33 <FranciscoD> So, the license field should have GPLv3+ 10:37:39 <FranciscoD> since its GPLv3 and later 10:37:55 <FranciscoD> You can have multiple licenses too 10:38:09 <FranciscoD> "GPLv3+ and MIT" if required sometime 10:38:13 <FranciscoD> Moving on 10:38:18 <FranciscoD> The spec file uses some macros 10:38:36 <FranciscoD> For example, the %configure macro 10:38:55 <FranciscoD> rpm -E %configure will give you this: http://paste.fedoraproject.org/96523/35923139 10:39:05 <FranciscoD> A macro is just a text replacement 10:39:17 <FranciscoD> So, when rpm does it's building, it replaces %configure with *all* of that 10:39:46 <FranciscoD> More on macros here: http://www.rpm.org/max-rpm/s1-rpm-inside-macros.html 10:40:07 <FranciscoD> Now, try `rpm -E %setup` 10:40:27 <FranciscoD> you wont get an answer, because these are internal rpmbuild macros 10:40:59 <FranciscoD> The actual value of %setup is explained on the link I just gave you 10:41:15 <FranciscoD> It unpacks the SOURCE0, and enters the directory 10:41:36 <FranciscoD> (simply put) 10:42:05 <FranciscoD> #info Comments in specfiles are any lines that start with # 10:42:20 <FranciscoD> Comment out *all* the lines under %setup, till the end of the file 10:42:50 <FranciscoD> Then, save your file and in the terminal, run: cd ~/rpmbuild/SPECS; rpmbuild -ba hello.spec 10:42:54 <FranciscoD> See what you get 10:43:43 <mpduty> the hello.tar.gz should be in the rpmbuild directory right? 10:44:22 <mpduty> and the commands should be run from the rpmbuild directory? 10:44:35 <FranciscoD> mpduty: rpmbuild/SOURCES 10:44:41 <FranciscoD> mpduty: rpmbuild -ba hello.spec from the rpmbuild/SPECS directory 10:44:52 <mpduty> ok 10:45:06 <FranciscoD> mpduty: you can run the rpmbuild -ba command from any place, you'll just have to specify the complete path to it 10:45:21 <FranciscoD> cd ~; rpmbuild -ba rpmbuild/SPECS/hello.spec would work too 10:45:22 <mpduty> right 10:47:03 <FranciscoD> You should get something like this if you remove all the part after %setup 10:47:06 <FranciscoD> http://paste.fedoraproject.org/96527/83363871 10:47:28 <FranciscoD> Now, uncomment the stuff 10:47:33 * banas tried to play with some of the steps on that wikipage, but still gets the same errors. will have to examine after the meeting maybe! 10:47:52 <FranciscoD> run it again, see what you get 10:48:00 <FranciscoD> rpmbuild -ba hello.spec 10:48:15 <FranciscoD> This time, you'll see %configure being called 10:48:24 <FranciscoD> You'll also see %make and %make install being called 10:48:39 <FranciscoD> It should fail with errors, telling you that there are files that havent being packaged 10:49:06 <FranciscoD> The hello package luckily has a very simple build process 10:49:20 <FranciscoD> all one needs to do is `configure; make; make install` 10:49:27 <FranciscoD> Our template spec already has these steps 10:49:42 <FranciscoD> The %build section is where you provide commands required to build your software 10:49:56 <FranciscoD> In hello, this is %configure; and the make command 10:50:25 <FranciscoD> Similarly, the %install section is where you provide commands that INSTALL the built files to their correct locations 10:50:37 <FranciscoD> in the buildroot of course, you need root priviliges to use the actual locations 10:51:32 <FranciscoD> One of the most important sections is the %files section 10:51:43 <FranciscoD> In this section, you *list* out all the installed files that the final rpm should have 10:51:56 <FranciscoD> You have to be careful to only own directories that you need to 10:52:12 <FranciscoD> When the rpm is uninstalled, rpm will use this list to remove files and folders 10:52:23 <FranciscoD> so if your list is wrong, you might end up removing things from other packages 10:52:44 <FranciscoD> You can mention each file and directory individually, but that's a lot of work 10:52:47 <FranciscoD> So we use some shortcuts 10:53:00 <FranciscoD> https://fedoraproject.org/wiki/Packaging:RPMMacros?rd=Packaging/RPMMacros 10:53:24 <FranciscoD> From the error that you got, you'll see that something like this should include all the files 10:55:43 <FranciscoD> http://paste.fedoraproject.org/96527/83363871 10:55:58 <FranciscoD> Add it to your spec, rerun rpmbuild 10:56:05 <FranciscoD> it shouldnt' give any errors this time 10:56:32 <FranciscoD> banas: we can fix your system after the workshop 10:57:02 <banas> FranciscoD: yeah np :) I'll refer to the logs after that! 10:58:34 <FranciscoD> Ok, that's it then 10:58:39 <FranciscoD> you've built an rpm ;) 10:58:50 <FranciscoD> check rpmbuild/RPMS/x86_64 and you'll find your rpm there 10:59:40 <mpduty> Its all about writing the spec files properly it seems 10:59:47 <FranciscoD> mpduty: the spec is *everything* 11:00:02 <FranciscoD> if your spec is correct, your software will usually build like a charm 11:00:10 <mpduty> I' 11:00:13 <FranciscoD> unless there are build system issues, which do crop up from time to time 11:01:12 <mpduty> I'll have to go through different spec files for different kinds of rpm builds 11:01:19 <FranciscoD> mpduty: yes, thats what most of us do 11:01:26 <FranciscoD> After a while you learn how to handle them 11:01:36 <FranciscoD> Our spec, while complete, isn't the best spec 11:01:41 <mpduty> yes perhaps 11:01:56 <FranciscoD> For example, instead of using 'hello' everywhere, you can replace it with the %{name} macro 11:02:05 <FranciscoD> The Name: xxxx field defines it for us 11:02:41 <mpduty> I had tried with a c source file, the rpms was successful, not the rpm 11:02:55 <FranciscoD> Similarly, instead of using 2.8 in the SOURCE0 URL, you can use %{version} 11:03:04 <FranciscoD> mpduty: you mean the SRPM was successful? 11:03:13 <FranciscoD> you used rpm -bs 11:03:14 <FranciscoD> ? 11:03:18 <FranciscoD> instead of -ba? 11:03:22 <mpduty> yes 11:03:36 <FranciscoD> The src.rpm file only packs up all your build material 11:03:42 <FranciscoD> itll contain the spec, the source, the patches 11:03:53 <FranciscoD> everything that you need to build the rpm 11:03:57 <FranciscoD> therefore, its called a source rpm 11:04:08 <FranciscoD> you can use rpmls <path to a source rpm> to check its contents 11:04:26 <FranciscoD> to actually *build* the software, either binary or noarch, you have to use rpm -ba 11:04:39 <FranciscoD> that builds the srpm, and then tries to build the rpm from this srpm 11:04:41 <FranciscoD> clear? 11:04:49 <mpduty> yes 11:05:07 <mpduty> there must be dependency issues too? 11:05:19 <FranciscoD> http://paste.fedoraproject.org/96536/33750313 11:05:25 <FranciscoD> mpduty: depends on the software 11:05:33 <mpduty> ok 11:05:35 <FranciscoD> hello just needs C libs as far as I know 11:05:44 <FranciscoD> most of us already have it installed 11:05:54 <FranciscoD> more complicated software will have other deps 11:05:58 <FranciscoD> and you need to figure these out 11:06:02 <mpduty> ok 11:06:07 <FranciscoD> Mostly, the README or the INSTALL file has this information 11:06:21 <FranciscoD> or the configure script will fail, and tell you whats missing 11:06:27 <FranciscoD> and you add it, then re-run 11:06:30 <mpduty> ok 11:06:33 <FranciscoD> if it fails again, you see what else is required 11:06:37 <FranciscoD> its an iterative process 11:07:16 <FranciscoD> Lets just improve our spec a little, and we'll finish the session 11:07:25 <mpduty> ok 11:11:11 <FranciscoD> http://paste.fedoraproject.org/96539/98337838 11:11:15 <FranciscoD> This is a better looking spec 11:11:21 <FranciscoD> Compare it to yours and see what I tweaked 11:12:28 <FranciscoD> #topic Tweaking the spec 11:14:07 <FranciscoD> A lot of changes are just cosmetic 11:14:13 <FranciscoD> The one major change is the use of the find_lang macro 11:14:32 <FranciscoD> find_lang is a script that looks for installed language files 11:14:48 <FranciscoD> all the *.mo files that were generated were language files 11:15:06 <FranciscoD> instead of having to manually specify them, which is tedious with larger software, you use the find_lang macro 11:15:20 <FranciscoD> it generates a list: %{name}.lang when used the way I do 11:15:27 <FranciscoD> and you pass that to the %files macro 11:15:38 <FranciscoD> telling the %files macro to also include the generated list 11:15:53 <FranciscoD> run rpm -E %find_lang to see what it stands for 11:16:40 <FranciscoD> Other than that, I just replaced some stuff with macros 11:16:47 <FranciscoD> and added some doc files: COPYING etc 11:16:57 <FranciscoD> which should be installed with the rpm 11:17:09 <FranciscoD> mpduty: any issues? 11:18:11 <niteshnarayanlal> FranciscoD,hi 11:18:23 <FranciscoD> niteshnarayanlal: hey 11:18:28 <FranciscoD> we're just finishing up the session 11:18:35 <niteshnarayanlal> I have been reading the above logs 11:18:39 <niteshnarayanlal> just a quick question 11:18:39 <FranciscoD> nice 11:18:42 <FranciscoD> go ahead 11:23:37 <niteshnarayanlal> sorry for the delay, still in office 11:23:54 <niteshnarayanlal> actually few days back I had done the packaging of few gems 11:24:26 <niteshnarayanlal> so basically what I did is to use an application to creat the sample spec 11:24:33 <niteshnarayanlal> in which I do the changes afterwards 11:24:39 <niteshnarayanlal> so my question is a bit generic 11:24:39 <FranciscoD> ok 11:24:40 <niteshnarayanlal> here 11:25:00 <niteshnarayanlal> do we have similar kind of applications for packaging diffenent things w 11:25:15 <FranciscoD> niteshnarayanlal: what application do you use to create the sample spec? 11:25:21 <niteshnarayanlal> yes 11:25:53 <FranciscoD> people have been trying to automate the procedure, but because the build systems are generally customized a litte etc., its not very easy to do 11:26:10 <FranciscoD> stuff like hello which are standard and simple can be automated 11:26:19 <FranciscoD> but bigger software generally requires some human intervention 11:26:39 <niteshnarayanlal> yes coz even the template spec has to be modified 11:26:47 <FranciscoD> always does 11:26:49 <niteshnarayanlal> for adding check section which is quite cruical to any spec 11:27:03 <FranciscoD> niteshnarayanlal: the thing is that not all software uses checks 11:27:14 <FranciscoD> %check comes in handy when the software has testcases 11:28:12 <niteshnarayanlal> true 11:29:02 <FranciscoD> Anything else? 11:29:05 <FranciscoD> Otherwise I'll close 11:29:12 <FranciscoD> we're just hitting the hour and a half mark 11:30:42 <FranciscoD> Ok, lets close 11:30:45 <FranciscoD> #endmeeting