10:05:33 <FranciscoD> #startmeeting ROS-RPM IRC session 2: RPM packaging basics
10:05:33 <zodbot> Meeting started Thu Apr 24 10:05:33 2014 UTC.  The chair is FranciscoD. Information about MeetBot at http://wiki.debian.org/MeetBot.
10:05:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
10:05:43 <FranciscoD> #meetingname ROS-RPM IRC session 2: RPM packaging basics
10:05:43 <zodbot> The meeting name has been set to 'ros-rpm_irc_session_2:_rpm_packaging_basics'
10:05:50 <FranciscoD> #topic Who's here?
10:05:57 <FranciscoD> .hellomynameis ankursinha
10:05:58 <zodbot> FranciscoD: ankursinha 'Ankur Sinha' <sanjay.ankur@gmail.com>
10:06:17 <FranciscoD> If you have an FAS, you can use the hellomynameis <fas account name> to introduce yourselves
10:06:31 <FranciscoD> (It's a good idea to have an FAS)
10:06:40 * FranciscoD hopes he isn't the only one here
10:06:45 <FranciscoD> banas: el ping
10:06:47 <banas> .fasinfo sarupbanskota
10:06:50 <zodbot> banas: User: sarupbanskota, Name: Sarup Banskota, email: sbanskota08@gmail.com, Creation: 2013-03-28, IRC Nick: banas, Timezone: Asia/Kolkata, Locale: en, GPG key ID: , Status: active
10:06:53 <zodbot> banas: Approved Groups: designteam cla_done cla_fpca
10:07:04 <FranciscoD> Anyone else?
10:07:16 <banas> lol, I have to be back in 3 minutes.
10:07:18 <banas> brb
10:07:23 <FranciscoD> XD
10:07:27 * FranciscoD will wait for banas
10:08:23 <banas> back!
10:08:30 <FranciscoD> mpduty: hey, just in time
10:08:32 * banas wonders if he took less than 3
10:08:35 <FranciscoD> .fas mpduty
10:08:35 <zodbot> FranciscoD: mohanprakash 'mohan prakash' <mpduty@gmail.com> - mpduty 'Mohan Prakash' <hamariteam@gmail.com>
10:08:42 <FranciscoD> banas: you took 1 ;)
10:08:46 <FranciscoD> Fine, lets start
10:08:47 <banas> hi mpduty! :D
10:08:49 <banas> FranciscoD: yay :D
10:08:55 <FranciscoD> #topic Important links
10:09:07 <FranciscoD> There are a couple of links you should keep around: bookmark them
10:09:10 <FranciscoD> http://fedoraproject.org/wiki/Join_the_package_collection_maintainers
10:09:15 <FranciscoD> http://fedoraproject.org/wiki/How_to_create_a_GNU_Hello_RPM_package
10:09:28 <FranciscoD> (We'll walk through the hello world one today)
10:09:32 <FranciscoD> http://fedoraproject.org/wiki/How_to_create_an_RPM_package
10:09:32 <banas> cool
10:09:49 <FranciscoD> http://fedoraproject.org/wiki/Category:Package_Maintainers
10:09:56 <FranciscoD> http://fedoraproject.org/wiki/Packaging:Guidelines
10:10:14 <FranciscoD> This is almost all the info you'll need
10:10:49 <FranciscoD> In special cases, you can look up http://www.rpm.org/wiki/Docs#PackagerDocumentation
10:11:04 <FranciscoD> And of course, when in doubt, send a mail to the fedora packaging committee and clear it up
10:11:09 <banas> cool, and this is not ROS specific, correct?
10:11:11 <mpduty> hi banas
10:11:17 <FranciscoD> banas: no, just general today
10:11:23 <banas> great
10:11:25 <FranciscoD> https://fedoraproject.org/wiki/Packaging_Committee?rd=Packaging:Committee
10:11:42 <FranciscoD> Ok, if everyone will open up the "how to create a hello world rpm" link
10:11:56 <FranciscoD> #topic Preparing your system
10:12:10 <FranciscoD> Of course, to build rpms you need to have the tools installed
10:12:22 <FranciscoD> luckily, we have a "fedora-packager" meta package that pulls in most things
10:12:32 <FranciscoD> so
10:12:38 <FranciscoD> #info sudo yum install fedora-packager
10:12:55 <FranciscoD> let me know once done
10:13:30 <banas> my internet is pretty bad, but I'll keep track!
10:13:35 <FranciscoD> Ok
10:14:01 <FranciscoD> This will install the rpmdevtools package
10:14:11 <FranciscoD> this contains a handy tool: rpmdev-setuptree
10:14:15 <FranciscoD> please run the command
10:14:33 <FranciscoD> #info rpmdev-setuptree sets up the required directory structure under ~/rpmbuild
10:14:43 <FranciscoD> once you've run it, cd into rpmbuild
10:14:47 <FranciscoD> and run: tree
10:14:49 <FranciscoD> see what you get
10:15:23 * banas is still waiting for fedora-packager to install
10:16:07 <FranciscoD> It should look like this: http://paste.fedoraproject.org/96510/33450513
10:16:43 <FranciscoD> Ask me to wait if you need to. We have quite a lot of time
10:17:14 <FranciscoD> Each directory serves a specific purpose
10:17:21 <FranciscoD> That should be quite obvious, right from their names
10:17:39 <FranciscoD> We generally don't need to tinker with the two BUILD* directories
10:18:05 <FranciscoD> #topic Lets package GNU hello world!
10:18:25 <FranciscoD> The sources directory is where we keep all our source code tar balls
10:18:33 <FranciscoD> tar balls, zips, patches, extra files
10:18:44 <FranciscoD> banas: still installing
10:18:47 <FranciscoD> mpduty: following?
10:18:51 <mpduty> yes
10:18:59 <mpduty> I've done this before
10:19:02 <FranciscoD> ah, great
10:19:08 <banas> FranciscoD: yeah! I have a question, I got something weird thrown on my screen. pasting
10:19:13 <FranciscoD> heh, go ahead
10:19:53 <FranciscoD> banas: how weird? :P
10:20:06 <banas> http://fpaste.org/96511/33477713/
10:20:36 <FranciscoD> uh oh
10:20:44 <FranciscoD> you have some messed up packages :/
10:21:00 <FranciscoD> you'll have to manually remove the older dups and then reinstall the new ones
10:21:09 <banas> argh, okay, you carry on, i;ll try to follow!
10:21:11 <FranciscoD> This is probably as a result of an update during the selinux bug
10:21:28 <FranciscoD> banas: http://fedoraproject.org/wiki/Common_F20_bugs#RPM_scriptlets_fail_during_updates
10:21:37 <FranciscoD> *sigh*
10:21:45 <FranciscoD> Ok, doesn't matter. I'll show you the outputs
10:23:20 <banas> FranciscoD: yup, I'm running some of them :)
10:23:25 <banas> yeah please do!
10:23:50 <FranciscoD> banas: I'd let you ssh into my system on a different user, but I don't think you can. I'm behind uni wifi too :/
10:23:58 <FranciscoD> Moving on.
10:24:03 <banas> yeah sure!
10:24:03 <mpduty> ok
10:24:05 <FranciscoD> cd into the source directory
10:24:16 <FranciscoD> wget http://ftp.gnu.org/gnu/hello/hello-2.8.tar.gz
10:24:25 <FranciscoD> we're just downloading the source
10:24:31 <FranciscoD> you can use your browser or whatever
10:24:52 <FranciscoD> Open up the tar: tar -xvf hello-2.8.tar.gz
10:24:54 <FranciscoD> take a look inside
10:25:05 <FranciscoD> More on tar at `man tar`
10:25:32 <FranciscoD> This is the source that will generate a binary for us
10:25:49 <FranciscoD> It has the usual structure that an autotools package has: configure script, make files etc
10:26:10 <FranciscoD> #topic build systems
10:26:27 <FranciscoD> What build systems do you folks know? Use info so we have a list
10:26:32 <FranciscoD> #info Autotools
10:26:35 <FranciscoD> what else?
10:26:43 <FranciscoD> #info cmake
10:26:48 <FranciscoD> #info python-setuptools
10:26:56 <FranciscoD> #info waf
10:27:07 <FranciscoD> #info catkin (ROS specific)
10:27:23 <FranciscoD> #info qmake
10:27:31 <FranciscoD> I'm sure there are a couple more
10:27:47 <FranciscoD> To build from the source, you need to know at least how to use these build systems
10:28:03 <FranciscoD> autotools looks simple, but when it comes down to patching build system files, it's a nightmare
10:28:04 <mpduty> ok
10:28:19 <FranciscoD> #topic the spec file
10:28:37 <FranciscoD> change directory to ~/rpmbuild/SPECS
10:28:50 <FranciscoD> rpmdev-newspec hello
10:29:07 <FranciscoD> rpmdev-newspec creates a blank spec for you from templates
10:29:23 <FranciscoD> rpmdev-newspec --help will show you more options
10:29:25 <mpduty> I guess the hello directory should be in the SOURCES
10:30:07 <FranciscoD> For example, you can use -t python xxx to generate a python spec template
10:30:07 <FranciscoD> mpduty: it should be
10:30:07 <FranciscoD> we don't need the uncompressed directory, rpm will uncompress it during the build process for us
10:30:17 <mpduty> ok
10:30:37 <FranciscoD> mpduty: check the output of tree ~/rpmbuild -L 2 -d
10:31:10 <FranciscoD> Open up the specfile using your favourite editor
10:31:17 <FranciscoD> vim/emacs/nano/peco/gedit/whathaveyou
10:32:00 <FranciscoD> It should look like this: http://paste.fedoraproject.org/96517/83355041
10:32:24 <FranciscoD> Now, until the %description part, everything is self explanatory
10:32:47 <FranciscoD> The buildrequires and requires is actually a little interesting
10:33:04 <FranciscoD> When you're building software from source, you might need certain libraries etc. that the source uses
10:33:18 <FranciscoD> So, since these are required while *building*, they are called "BuildRequires"
10:33:35 <FranciscoD> Similarly, some software is required to run the application once its been built
10:33:40 <FranciscoD> These are called "Requires"
10:33:46 <FranciscoD> Mostly, rpm can figure out Requires itself
10:34:01 <FranciscoD> Using logic to find the requires shared objects (library files) etc
10:34:19 <FranciscoD> However, for scripted languages sometimes, like python, we manually specify the "Requires"
10:34:23 <FranciscoD> Clear?
10:34:27 <mpduty> yes
10:34:49 <FranciscoD> The name of the specfile should be the same as the value in the Name: field
10:35:00 <FranciscoD> this is a must
10:35:00 <FranciscoD> fill up bits till %description
10:35:23 <FranciscoD> You can open up another terminal or nautilus to get info on the hello source code
10:35:26 <FranciscoD> The version is 2.8
10:35:31 <FranciscoD> See if you can find the license
10:35:58 <FranciscoD> The COPYING file or LICENSE file generally tells you the license
10:36:12 <FranciscoD> Note that the license in the headers of the source code also needs to be checked
10:36:24 <FranciscoD> There's another utility that does this for us: licensecheck
10:37:00 <FranciscoD> run `licensecheck -r .` inside the hello source directory
10:37:16 <FranciscoD> This will give you http://paste.fedoraproject.org/96521/33580613
10:37:33 <FranciscoD> So, the license field should have GPLv3+
10:37:39 <FranciscoD> since its GPLv3 and later
10:37:55 <FranciscoD> You can have multiple licenses too
10:38:09 <FranciscoD> "GPLv3+ and MIT" if required sometime
10:38:13 <FranciscoD> Moving on
10:38:18 <FranciscoD> The spec file uses some macros
10:38:36 <FranciscoD> For example, the %configure macro
10:38:55 <FranciscoD> rpm -E %configure will give you this: http://paste.fedoraproject.org/96523/35923139
10:39:05 <FranciscoD> A macro is just a text replacement
10:39:17 <FranciscoD> So, when rpm does it's building, it replaces %configure with *all* of that
10:39:46 <FranciscoD> More on macros here: http://www.rpm.org/max-rpm/s1-rpm-inside-macros.html
10:40:07 <FranciscoD> Now, try `rpm -E %setup`
10:40:27 <FranciscoD> you wont get an answer, because these are internal rpmbuild macros
10:40:59 <FranciscoD> The actual value of %setup is explained on the link I just gave you
10:41:15 <FranciscoD> It unpacks the SOURCE0, and enters the directory
10:41:36 <FranciscoD> (simply put)
10:42:05 <FranciscoD> #info Comments in specfiles are any lines that start with #
10:42:20 <FranciscoD> Comment out *all* the lines under %setup, till the end of the file
10:42:50 <FranciscoD> Then, save your file and in the terminal, run: cd ~/rpmbuild/SPECS; rpmbuild -ba hello.spec
10:42:54 <FranciscoD> See what you get
10:43:43 <mpduty> the hello.tar.gz should be in the rpmbuild directory right?
10:44:22 <mpduty> and the commands should be run from the rpmbuild directory?
10:44:35 <FranciscoD> mpduty: rpmbuild/SOURCES
10:44:41 <FranciscoD> mpduty: rpmbuild -ba hello.spec from the rpmbuild/SPECS directory
10:44:52 <mpduty> ok
10:45:06 <FranciscoD> mpduty: you can run the rpmbuild -ba command from any place, you'll just have to specify the complete path to it
10:45:21 <FranciscoD> cd ~; rpmbuild -ba rpmbuild/SPECS/hello.spec would work too
10:45:22 <mpduty> right
10:47:03 <FranciscoD> You should get something like this if you remove all the part after %setup
10:47:06 <FranciscoD> http://paste.fedoraproject.org/96527/83363871
10:47:28 <FranciscoD> Now, uncomment the stuff
10:47:33 * banas tried to play with some of the steps on that wikipage, but still gets the same errors. will have to examine after the meeting maybe!
10:47:52 <FranciscoD> run it again, see what you get
10:48:00 <FranciscoD> rpmbuild -ba hello.spec
10:48:15 <FranciscoD> This time, you'll see %configure being called
10:48:24 <FranciscoD> You'll also see %make and %make install being called
10:48:39 <FranciscoD> It should fail with errors, telling you that there are files that havent being packaged
10:49:06 <FranciscoD> The hello package luckily has a very simple build process
10:49:20 <FranciscoD> all one needs to do is `configure; make; make install`
10:49:27 <FranciscoD> Our template spec already has these steps
10:49:42 <FranciscoD> The %build section is where you provide commands required to build your software
10:49:56 <FranciscoD> In hello, this is %configure; and the make command
10:50:25 <FranciscoD> Similarly, the %install section is where you provide commands that INSTALL the built files to their correct locations
10:50:37 <FranciscoD> in the buildroot of course, you need root priviliges to use the actual locations
10:51:32 <FranciscoD> One of the most important sections is the %files section
10:51:43 <FranciscoD> In this section, you *list* out all the installed files that the final rpm should have
10:51:56 <FranciscoD> You have to be careful to only own directories that you need to
10:52:12 <FranciscoD> When the rpm is uninstalled, rpm will use this list to remove files and folders
10:52:23 <FranciscoD> so if your list is wrong, you might end up removing things from other packages
10:52:44 <FranciscoD> You can mention each file and directory individually, but that's a lot of work
10:52:47 <FranciscoD> So we use some shortcuts
10:53:00 <FranciscoD> https://fedoraproject.org/wiki/Packaging:RPMMacros?rd=Packaging/RPMMacros
10:53:24 <FranciscoD> From the error that you got, you'll see that something like this should include all the files
10:55:43 <FranciscoD> http://paste.fedoraproject.org/96527/83363871
10:55:58 <FranciscoD> Add it to your spec, rerun rpmbuild
10:56:05 <FranciscoD> it shouldnt' give any errors this time
10:56:32 <FranciscoD> banas: we can fix your system after the workshop
10:57:02 <banas> FranciscoD: yeah np :) I'll refer to the logs after that!
10:58:34 <FranciscoD> Ok, that's it then
10:58:39 <FranciscoD> you've built an rpm ;)
10:58:50 <FranciscoD> check rpmbuild/RPMS/x86_64 and you'll find your rpm there
10:59:40 <mpduty> Its all about writing the spec files properly it seems
10:59:47 <FranciscoD> mpduty: the spec is *everything*
11:00:02 <FranciscoD> if your spec is correct, your software will usually build like a charm
11:00:10 <mpduty> I'
11:00:13 <FranciscoD> unless there are build system issues, which do crop up from time to time
11:01:12 <mpduty> I'll have to go through different spec files for different kinds of rpm builds
11:01:19 <FranciscoD> mpduty: yes, thats what most of us do
11:01:26 <FranciscoD> After a while you learn how to handle them
11:01:36 <FranciscoD> Our spec, while complete, isn't the best spec
11:01:41 <mpduty> yes perhaps
11:01:56 <FranciscoD> For example, instead of using 'hello' everywhere, you can replace it with the %{name} macro
11:02:05 <FranciscoD> The Name: xxxx field defines it for us
11:02:41 <mpduty> I had tried with a c source file, the rpms was successful, not the rpm
11:02:55 <FranciscoD> Similarly, instead of using 2.8 in the SOURCE0 URL, you can use %{version}
11:03:04 <FranciscoD> mpduty: you mean the SRPM was successful?
11:03:13 <FranciscoD> you used rpm -bs
11:03:14 <FranciscoD> ?
11:03:18 <FranciscoD> instead of -ba?
11:03:22 <mpduty> yes
11:03:36 <FranciscoD> The src.rpm file only packs up all your build material
11:03:42 <FranciscoD> itll contain the spec, the source, the patches
11:03:53 <FranciscoD> everything that you need to build the rpm
11:03:57 <FranciscoD> therefore, its called a source rpm
11:04:08 <FranciscoD> you can use rpmls <path to a source rpm> to check its contents
11:04:26 <FranciscoD> to actually *build* the software, either binary or noarch, you have to use rpm -ba
11:04:39 <FranciscoD> that builds the srpm, and then tries to build the rpm from this srpm
11:04:41 <FranciscoD> clear?
11:04:49 <mpduty> yes
11:05:07 <mpduty> there must be dependency issues too?
11:05:19 <FranciscoD> http://paste.fedoraproject.org/96536/33750313
11:05:25 <FranciscoD> mpduty: depends on the software
11:05:33 <mpduty> ok
11:05:35 <FranciscoD> hello just needs C libs as far as I know
11:05:44 <FranciscoD> most of us already have it installed
11:05:54 <FranciscoD> more complicated software will have other deps
11:05:58 <FranciscoD> and you need to figure these out
11:06:02 <mpduty> ok
11:06:07 <FranciscoD> Mostly, the README or the INSTALL file has this information
11:06:21 <FranciscoD> or the configure script will fail, and tell you whats missing
11:06:27 <FranciscoD> and you add it, then re-run
11:06:30 <mpduty> ok
11:06:33 <FranciscoD> if it fails again, you see what else is required
11:06:37 <FranciscoD> its an iterative process
11:07:16 <FranciscoD> Lets just improve our spec a little, and we'll finish the session
11:07:25 <mpduty> ok
11:11:11 <FranciscoD> http://paste.fedoraproject.org/96539/98337838
11:11:15 <FranciscoD> This is a better looking spec
11:11:21 <FranciscoD> Compare it to yours and see what I tweaked
11:12:28 <FranciscoD> #topic Tweaking the spec
11:14:07 <FranciscoD> A lot of changes are just cosmetic
11:14:13 <FranciscoD> The one major change is the use of the find_lang macro
11:14:32 <FranciscoD> find_lang is a script that looks for installed language files
11:14:48 <FranciscoD> all the *.mo files that were generated were language files
11:15:06 <FranciscoD> instead of having to manually specify them, which is tedious with larger software, you use the find_lang macro
11:15:20 <FranciscoD> it generates a list: %{name}.lang when used the way I do
11:15:27 <FranciscoD> and you pass that to the %files macro
11:15:38 <FranciscoD> telling the %files macro to also include the generated list
11:15:53 <FranciscoD> run rpm -E %find_lang to see what it stands for
11:16:40 <FranciscoD> Other than that, I just replaced some stuff with macros
11:16:47 <FranciscoD> and added some doc files: COPYING etc
11:16:57 <FranciscoD> which should be installed with the rpm
11:17:09 <FranciscoD> mpduty: any issues?
11:18:11 <niteshnarayanlal> FranciscoD,hi
11:18:23 <FranciscoD> niteshnarayanlal: hey
11:18:28 <FranciscoD> we're just finishing up the session
11:18:35 <niteshnarayanlal> I have been reading the above logs
11:18:39 <niteshnarayanlal> just a quick question
11:18:39 <FranciscoD> nice
11:18:42 <FranciscoD> go ahead
11:23:37 <niteshnarayanlal> sorry for the delay, still in office
11:23:54 <niteshnarayanlal> actually few days back I had done the packaging of few gems
11:24:26 <niteshnarayanlal> so basically what I did is to use an application to creat the sample spec
11:24:33 <niteshnarayanlal> in which I do the changes afterwards
11:24:39 <niteshnarayanlal> so my question is a bit generic
11:24:39 <FranciscoD> ok
11:24:40 <niteshnarayanlal> here
11:25:00 <niteshnarayanlal> do we have similar kind of applications for packaging diffenent things w
11:25:15 <FranciscoD> niteshnarayanlal: what application do you use to create the sample spec?
11:25:21 <niteshnarayanlal> yes
11:25:53 <FranciscoD> people have been trying to automate the procedure, but because the build systems are generally customized a litte etc., its not very easy to do
11:26:10 <FranciscoD> stuff like hello which are standard and simple can be automated
11:26:19 <FranciscoD> but bigger software generally requires some human intervention
11:26:39 <niteshnarayanlal> yes coz even the template spec has to be modified
11:26:47 <FranciscoD> always does
11:26:49 <niteshnarayanlal> for adding check section which is quite cruical to any spec
11:27:03 <FranciscoD> niteshnarayanlal: the thing is that not all software uses checks
11:27:14 <FranciscoD> %check comes in handy when the software has testcases
11:28:12 <niteshnarayanlal> true
11:29:02 <FranciscoD> Anything else?
11:29:05 <FranciscoD> Otherwise I'll close
11:29:12 <FranciscoD> we're just hitting the hour and a half mark
11:30:42 <FranciscoD> Ok, lets close
11:30:45 <FranciscoD> #endmeeting