02:04:22 #startmeeting Fedora Classroom - Introduction to Koji Build system and Bodhi Updates system 02:05:20 #topic Basic Ideas 02:05:41 #link http://koji.fedoraproject.org 02:05:53 #link http://admin.fedoraproject.org/updates 02:06:51 I will get started now but feel free to ask questions in between if you like 02:07:36 Koji is the Fedora build system used by Fedora Project 02:08:38 Essentially it is what converts upstream source archives into RPM packages with the help of spec files as input from the package maintainers 02:08:49 In the first link, you can see a web interface to it 02:09:49 In the good old days when Fedora got started, the build system was run inside Red Hat and you could only see updates being pushed out in a bulk (via the daily rawhide report) for instance 02:10:19 With the merge of Fedora Core and Extras back before Fedora 7 release, a new build system was introduced 02:10:31 that made it possible for developers and users to see builds as they happen 02:10:46 of course, as with all things Fedora, it is free and open source 02:10:51 and available in the repository itself 02:11:31 Bodhi is the updates system used by Fedora. The web interface is available at http://admin.fedoraproject.org/updates 02:12:18 These are tools that are used primarily by Fedora package maintainers but provide a number of useful features for end users as well 02:13:28 So let me explain the basic workflow that ties these tools together 02:13:43 A Fedora package maintainer initiates a new build via Koji 02:14:09 let's say when there is a new upstream release of a project 02:14:38 The build is accessible via the web interface as soon as it is initiated 02:14:46 When the build is complete 02:15:07 the package maintainer can choose to either push it to updates or updates-testing repository 02:15:43 The recommended practice is to push it to updates testing repository first so that the people interested in testing it can have the opportunity to provide some feedback 02:16:08 There are occasions where the package maintainer has reasons to push it to updates repository directly as well 02:16:32 For example, a high risk security fix that is small enough will fall into the latter 02:17:18 The updates system is essentially what pushes the packages from the build system out in the mirrors in either repository 02:17:27 Any questions so far? 02:17:53 I'm curious about tasks vs builds on the koji page 02:18:42 ok. I will come to that in a bit 02:20:14 the simple explanation from my understanding is that tasks are how you initiate or manipulate a build 02:20:44 the updates system web interface 02:20:55 allows you to provide easy feedback to the maintainers 02:21:04 for example 02:21:13 If you are running Fedora 11 02:21:23 and you want to test the packages in the updates testing repository 02:21:25 you can do 02:21:34 yum --enablerepo=updates-testing update 02:21:54 you can see all the packages in the repo 02:21:55 via https://admin.fedoraproject.org/updates/F11/testing 02:22:09 and if you want to provide feedback on one of them 02:22:25 let's say 02:22:26 https://admin.fedoraproject.org/updates/F11/FEDORA-2009-8296 02:22:32 That particular update 02:22:37 has a single bug fix 02:22:41 if you were affected by it 02:23:01 and confirmed via the testing that it does indeed fix the bug 02:23:06 you can add a comment 02:23:23 just provide your email address 02:23:28 and select 02:23:37 works for me or does not work as depending on the case 02:23:41 and add your comment 02:24:05 The updates system by default pushes the package into updates repository once it gets three positive confirmations 02:24:08 * nirik notes this sort of testing is very very very valuable to maintainers. Please provide this feedback as much as you can. 02:24:16 these are called karma points 02:24:51 The maintainer of course can decide before the push whether he or she wants to take advantage of karma points and if so how many does it require 02:25:01 or choose to push it manually 02:25:12 it is a quicker alternative to providing feedback via bugzilla 02:25:25 the email address is useful because it is sometimes a conversation 02:25:32 if you say that it doesnt work 02:25:40 the maintainers would usually want more information from you 02:26:04 There is a common confusion with the updates system 02:26:11 that is the status called "pending" 02:26:27 when a update is pushed into either updates or updates testing repository by the maintainers 02:26:39 Someone from release engineering team 02:26:44 has to sign the package first 02:26:49 this is a manual process 02:26:54 and sometimes there is a delay 02:27:06 all the packages that have been pushed but not signed 02:27:09 is in the pending status 02:27:29 You can still access the package via Koji web interface 02:27:35 but it wont be available in the repository yet 02:27:42 Another common confusion 02:27:51 is that update notices are send out via mailing list 02:28:03 but the update is not in the mirror used by yum 02:28:36 this can happen because mirrors are not in the same schedule and yum picks a mirror that has yet to sync the update 02:29:01 Like nirik said the feedback that we get from users is very useful 02:29:11 for us as package maintainers 02:29:24 so provide more of it whenever you want 02:29:35 even a simple "works for me" is good to know 02:29:53 especially because we are in Fedora maintaining multiple releases and pushing updates across branches 02:30:05 and it is not always feasible to test every update extensively 02:30:42 After a package stays in updates testing repository for sometime (normally a week or so) 02:30:49 it is pushed into the updates repository 02:31:11 You can get notifications from many places 02:31:15 one is the mailing list 02:31:16 http://www.redhat.com/mailman/listinfo/fedora-package-announce 02:31:35 Another is rss feeds at http://planet.fedoraproject.org/infofeed/ 02:32:24 #topic Koji 02:32:43 Koji web interface like I said is accessible from http://koji.fedoraproject.org 02:32:52 One of the features used often by users 02:32:59 is early access to updates 02:33:04 before it hits the repository 02:33:15 or just to know what is the status of an update 02:33:40 let's say there is a Firefox update in http://firefox.com 02:33:48 and you want to know whether it is being built in Fedora 02:33:56 you can use the Koji interface to search for it 02:34:08 There is a pitfall here however 02:34:28 sometimes a feature update is only available in the development branch called Rawhide 02:34:41 and users try to get the update from there and end up breaking their system 02:35:14 so make sure you get the update intended for the release you wanted 02:35:31 and if you do this, be aware that it has not gone through the usual QA via updates testing repository 02:35:46 and you can end up with regressions 02:35:48 * nirik notes that they are also not signed. 02:36:18 can i ask a question? or wait till the end? 02:36:25 If you are just using the interface to keep track of updates, that is fine 02:36:29 sijis, feel free to ask 02:36:59 so looking at koji interface, how can i tell its for rawhide or not? 02:37:15 i assume i'm suppose to be looking at the 'recent builds' section. 02:37:17 that is a good question 02:37:29 let me give you some specific example 02:38:12 Let's say you are running Fedora 11 and want to know whats the latest version of Gnote built for your release 02:38:22 and you go to http://koji.fedoraproject.org/koji/ 02:38:27 use the search interface 02:38:41 You get the results 02:38:42 http://koji.fedoraproject.org/koji/packageinfo?packageID=8132 02:39:04 Most of the packages in Fedora use what is called the dist tag 02:39:08 fc12 of fc11 and so on 02:39:16 so a quick way to check 02:39:24 is look at the end of the package name 02:39:31 in this case 02:39:34 you will notice 02:39:50 that 0.6.1 is only available for Fedora 12 (Which is the development version) 02:40:00 and 0.5.3 is what is available for Fedora 11 02:40:16 If you click on 02:40:20 0.6.1 02:40:27 http://koji.fedoraproject.org/koji/buildinfo?buildID=125125 02:40:35 you see the tags? 02:40:44 that shows 02:40:49 dist-f12 02:40:52 yup. dist-f12, f12-alpha 02:40:56 that is a clear indicator 02:40:58 of which release 02:41:00 it is built for 02:41:05 that is set by the build system 02:41:27 dist-f11 for Fedora 11 02:41:32 it should be fairly obvious 02:41:40 what release it is being tagged 02:42:01 if you get a Fedora 12 tagged package and try to install it on a previous release 02:42:06 you will often end up breaking it 02:42:18 which tag is "rawhide"? 02:42:43 If Fedora 11 is the latest general release 02:42:49 dist-f12 02:42:51 is rawhide 02:43:09 ok 02:43:21 we dont tag anything as rawhide because then you would have to retag it before release 02:43:40 so rawhide is essentially the next Fedora release. 02:43:44 ! so gnote-0.6.1-1.fc12 is a rawhide release 02:43:47 correct 02:43:52 mj0vy, correct 02:43:57 mether, how to find whether gnote is in update-testing or rawhide or updates repository with koji? 02:44:13 if you go to 02:44:19 http://admin.fedoraproject.org/updates 02:44:22 and search for gnote 02:44:32 it would show you the status 02:44:37 the three statuses are 02:44:49 pending - release engineering needs to sign the packages manually 02:45:01 testing - it is in updates-testing repository 02:45:10 updates - It is in updates repository 02:45:30 there is actually 02:45:59 a bodhi and koji command line clients 02:46:23 yum install bodhi 02:46:25 and do 02:46:32 bodhi -L gnote 02:46:50 and you can find out the status of gnote across all the current branches 02:47:14 http://fpaste.org/bJiV/ 02:47:21 thats the same output 02:47:49 zer0c00l, does that answer your question? 02:48:02 mether, yes 02:48:08 excellent 02:48:39 i'm not sure if you've gone through this.. but how does bodhi and koji differ? 02:48:54 koji is the build system 02:49:02 and bodhi is the updates system 02:49:10 koji builds the packages 02:49:21 and bodhi is used to push the packages into the repositories 02:49:41 both of them have web interfaces and command line clients as well 02:50:03 Bodhi has also a metrics page which I find pretty interesting 02:50:14 These are available for every release 02:50:16 https://admin.fedoraproject.org/updates/metrics/?release=F11 02:50:20 is for Fedora 11 02:50:37 As you can see, we tend to push a large number of updates 02:51:18 interesting stuff. 02:51:22 Folks from the release engineering and infrastructure team have done a bunch of work to speed up the process of getting the updates to users whenever the package maintainers does a push 02:51:42 question .. 02:51:46 Josh Boyer from rel eng team explained some of the recent improvements in his blog post 02:51:47 http://jwboyer.livejournal.com/35243.html 02:51:50 BounceCat, ask 02:51:57 status:pending = "release engineering needs to sign the packages" .. do they check the package somehow before signing it? 02:52:25 no. they merely sign the rpm package so that you know it is from Fedora 02:52:37 ok 02:52:54 the testing is actually done by volunteers in the QA team or other users 02:53:09 so everyone can participate and help out 02:53:23 ! mether, for viewing, we should install only bodhi-client package, no? 02:53:49 yes there is no package named bodhi 02:54:01 mj0vy, yes. only the client is required for end users 02:54:09 i was wrong about the package name 02:54:13 it is called bodhi-client 02:54:16 instead of bodhi 02:54:57 I as a package maintainer get all the tools I need with the fedora-packager meta package 02:55:11 # yum install fedora-packager 02:55:15 if you want them all 02:55:36 As I was explaining 02:55:44 there is ongoing work 02:55:51 to improve the infrastructure 02:55:59 I will briefly explain them 02:56:01 https://fedorahosted.org/sigul/ 02:56:14 Sigul is a automated signing system 02:56:23 that basically is going speed up the process 02:56:28 of moving the packages 02:56:31 into the repository 02:56:45 currently someone from rel eng team has to manually sign 02:56:49 every single package 02:56:55 and sometimes it gets delayed 02:57:14 while the blog post I pointed out a bit earlier talked about the improvements they have done 02:57:24 Sigul should help it more 02:57:33 and it will be transparent to you as end users 02:57:36 Another thing 02:57:43 is what BounceCat what was asking about 02:57:58 release engineering doesnt actually do any checking on the package 02:58:02 because it is a small team 02:58:07 and there are a large number of updates 02:58:20 we get some feedback from the testers 02:58:26 and that is often not enough 02:58:39 so there has been some work 02:58:45 done on automating some of the common checks 02:58:52 https://fedorahosted.org/autoqa/ 02:59:02 you should be seeing it in action soon 02:59:32 common packaging mistakes can be caught and reported automatically 02:59:35 by this system 02:59:49 so testers can use their time to check for things 02:59:53 that cannot be easily automated 02:59:57 instead of grunt work 03:00:30 That's all I wanted to cover in this session but I will do others as followups related to this one 03:00:41 If you have more questions please ask 03:01:09 #topic Final Q&A 03:01:46 No more? 03:01:47 mether: It was very informative. Thanks. 03:01:56 Thank you all for attending then 03:02:00 thanks mether! 03:02:03 thank you 03:02:12 dumb question: how to speak the word "bodhi" ? eg sounds like? 03:02:13 mether: thanks! 03:02:29 BounceCat, Luke Macken who wrote the build system 03:02:37 is a bit of not so closet 03:02:50 fan of Buddhism 03:02:54 karma 03:02:56 bodhi 03:03:04 and some of his other projects 03:03:11 are named after the core ideals 03:03:12 buddha got wisdom under bodhi tree :) 03:03:28 http://en.wikipedia.org/wiki/Bodhi 03:03:30 thanks mether 03:03:42 mether, ahh ok. thanks for a great class btw !!! 03:03:47 Bodhi and Karma are Sanskrit words 03:04:02 Hint: Koji also has a interesting reason behind its name 03:04:04 check it up 03:04:12 BounceCat, zer0c00l welcome 03:04:35 like wrote the updates system 03:04:39 not the build system 03:04:54 I misspoke 03:05:07 Luke Macken wrote the updates system - Bodhi 03:05:13 alright 03:05:17 #end 03:05:30 #endmeeting