16:00:35 #startmeeting Server SIG Weekly Meeting (2016-01-26) 16:00:35 Meeting started Tue Jan 26 16:00:35 2016 UTC. The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:35 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:35 The meeting name has been set to 'server_sig_weekly_meeting_(2016-01-26)' 16:00:36 #meetingname ServerSIG 16:00:36 The meeting name has been set to 'serversig' 16:00:36 #chair sgallagh mizmo nirik stefw adamw simo danofsatx mhayden jds2001 16:00:36 #topic roll call 16:00:36 Current chairs: adamw danofsatx jds2001 mhayden mizmo nirik sgallagh simo stefw 16:00:45 .hello mhayden 16:00:46 mhayden: mhayden 'Major Hayden' 16:00:50 .hello sgallagh 16:00:50 .hello kevin 16:00:51 Hello 16:00:53 sgallagh: sgallagh 'Stephen Gallagher' 16:00:57 nirik: kevin 'Kevin Fenzi' 16:00:59 Dodji: Thanks for joining us today. 16:01:14 thank you for inviting me 16:01:19 .hello simo 16:01:20 simo: simo 'Simo Sorce' 16:01:20 .hello dmossor 16:01:26 danofsatx: dmossor 'Dan Mossor' 16:02:18 .hello Dodji 16:02:19 Dodji: Sorry, but you don't exist 16:02:23 :) 16:02:38 Dodji: it's FAS account 16:02:47 .hello dodji 16:02:48 Dodji: dodji 'Dodji Seketeli' 16:02:51 better 16:03:08 danofsatx: thanks 16:03:13 yw 16:04:23 We technically have quorum, but we're missing at least one person that I really want to be part of this conversation... 16:04:26 adamw: Are you around? 16:06:21 OK, well, I guess we may as well get started. 16:06:24 #topic Agenda 16:06:40 #info Agenda Topic: ABI/API Compatibility 16:06:55 Other topics for this week? 16:09:07 OK, guess not 16:09:15 #topic ABI/API Compatibility. 16:10:15 Dodji: Would you be willing to give a high-level overview of some of the work you've been doing, and then we can talk about how to take it and put it into practice for the Server Edition? 16:12:04 Dodji: ? 16:12:21 right 16:13:35 After we started looking into how to come up with an automatic scheme to detect incompatible ABI changes, we realized we needed a tool that would compare two binary *packages* and detect potentially problematic changes. 16:14:12 We came up with abipkgdiff (https://sourceware.org/libabigail/manual/abipkgdiff.html) 16:14:56 that tool takes two versions of a package, say glib-version-one and glib-version-two, along with the debug info packages of the two packages 16:15:13 and emits a report of the ABI changes that are potentially problematic. 16:15:57 In parallel, there has been work done in the taskotron infrastructure, to allow a task to retrieve: 16:16:34 1/ the latest previous *stable* package, for a given N-V-R 16:16:52 2/ the debug info package, for a given N-V-R. 16:17:46 Dodji: Can you show an example of what the output of that check looks like? 16:17:53 yes 16:17:58 Is it human-readable, or is other tooling needed for that? 16:19:16 http://paste.fedoraproject.org/314933/25139145 16:19:25 it's human-readable 16:19:59 so that's on the Glib package; comparing the rawhide one with the latest stable in F23 16:20:15 Cool 16:20:37 so 16:21:06 Provided someone stands up to write (and maintain) the taskotron task, I think it's now possible to write the task that will fire on each koji build 16:21:07 nice 16:21:23 Very, very cool 16:21:33 and compare the abi of the build that just completed against the latest stable build that matches 16:21:45 there are two catches, though. 16:21:57 One that I think is handled, and another one that is not yet handled. 16:22:03 I assume we'd also want to notify the creator of the build that the ABI had changed. 16:22:04 I'll start by the one that is handled. 16:22:20 Maybe even automatically untag the build so it doesn't hit the Rawhide mirrors in the nightly mash 16:22:31 1/ We need somehow to provide a way for packagers to cut false positives 16:22:35 (If it was intentional breakage, the packager could tag it back in) 16:22:46 Sorry, please continue. 16:22:55 sgallagh: right, I'll come to your valid points in a bit. 16:22:58 so, 1/ 16:23:41 For instance, in the case I just showed in the fpaste, that ABI change is a change to an implementation detail 16:24:17 why? because the change is on a type (a struct) that is *defined* somewhere that is not a public header file. 16:24:40 it says: 16:24:43 1 data member insertion: 16:24:43 'guint _GSocketPrivate::connected_read', at offset 256 (in bits) at gsocket.c:245:1 16:24:51 see, gsocket.c 16:25:03 Right, so an opaque struct grew a new member 16:25:07 right 16:26:35 http://paste.fedoraproject.org/314938/53825587 16:26:55 So we need a way for the package maintainer to acknowledge that certain changes are not public ABI 16:27:05 so, look at the output of the tool, when we tell it to not look at type changes that happen outside of *.h files 16:27:09 And that the size of the struct is not supposed to be relied upon. 16:27:45 so basically, just like you can provide valgrind with "suppressions", you can provide abipkgdiff with suppression specification files. 16:28:27 and the way it's implemented speficially for fedora is that, if a file ends up with *.abignore is found *in the package* abipkgdiff takes it as a suppression specification 16:28:37 and uses it as such. 16:28:44 Define "in the package" please? 16:28:52 Source: foo.abignore 16:28:56 (for instance) 16:28:58 OK, so in dist-git 16:29:01 in the .spec file 16:29:06 sgallagh: right. 16:29:06 Not in the produced binary RPM 16:29:31 sgallagh: ah, woops. My bad. sorry. It has to be in the binary RPM 16:29:40 Ah, hmm. 16:29:50 * Dodji confuses this with some other internal project he works on 16:29:51 sorry 16:30:05 I guess it could be a %doc ? 16:30:08 but if people want something else, I am open. 16:30:11 I guess it could be a %doc ? 16:30:12 yes 16:30:37 I wanted to provide *something* first. 16:30:40 I think we'd probably want to have a %abignore macro instead, really. 16:30:50 maybe rpmbuild could put it in the -debuginfo packages even? 16:31:12 yeah, that would work too. 16:31:17 Yeah, I'd rather see it in -debuginfo, then it's not adding unnecessary cruft to installed packages. 16:31:29 Dodji: so N-V-R has .abignore and N-V-R+1 too, which one do you pick up ? 16:31:52 simo: the +1 one. 16:32:15 I mean what if the suppressions should be different between X-.X+1 and X-1 -> X+1 is that possible ? 16:32:27 Dodji: my concern is if it wiill become a maintenance burden 16:32:31 simo: actually we only look inside the second package for *.abignore files 16:33:07 Dodji: so a packager needs to build a packge, get errors, create a new .abiignore file and then rev up the package 16:33:14 right 16:33:36 wouldn't it be better to pick .abignore from dist-git so that all you need is to commit it and then just rerun the tool w/o having to touch the RPM ? 16:33:40 simo: Well, we can also probably arrange it so that we have `fedpkg local --abicheck` or something 16:33:41 but the packager can see the error right away when building the package 16:33:48 abipkgdiff is an offline tool 16:33:55 just like rpmlint 16:34:06 sure 16:34:20 but truth is, people will end up going through the churn 16:34:29 yeah 16:34:31 so I'd rather have the file hosted externally to the RPM 16:34:35 * Dodji is opened to ideas :-) 16:34:43 simo: Right, but we can avoid the churn in koji at least 16:34:45 because after all some suppressions will be diff-version specific anyway 16:34:50 By making it part of `fedpkg local` 16:35:00 simo: right; I just wanted to provide "a way" at least. 16:35:05 sgallagh: I don't always run fedpkg local :) 16:35:07 I am not against better ways :-) 16:35:13 simo: tsk tsk 16:35:27 and then there is the second catch, that is not yet handled. 16:35:37 Dodji: the thing that makes me lean for externally hosted is that a suppression is by definition not a change of anything ni the code 16:35:40 Right, so let's assume that this is a solvable problem 16:35:45 I think we need a way to whitelist the packages that we actually want to check the ABI for. 16:35:47 so we go through a whole package build just to add a file in the rpm 16:35:50 sgallagh: running rpmlint abicheck etc post build is okay to me 16:35:55 just to suppress a tool used during build 16:36:07 sounds like a churn we should spare koji from 16:36:16 dgilmore: In the case of packages that take a long time to build, avoiding having to rebuild in koji seems sensible though 16:36:23 i.e, restrain the ABI check to core packages only 16:36:25 (Especially with ARM builds taking so long) 16:36:32 yeah 16:36:39 sgallagh: ehhh 16:36:52 Dodji: that again seem to be external data koji should retrieve from "somewhere" to me :) 16:36:54 Anyway, let's avoid prematurely optimizing this 16:37:10 simo: :) 16:37:22 A mechanism for telling the tool what to skip exists 16:37:24 simo: yes, and I am not sure where exactly 16:37:32 sgallagh: ah 16:37:32 like we have comps and all, maybe we have a virtual group of packages that defines the whitelist 16:37:36 Let's all agree that we can figure out an appropriate way to host it as we get down the path 16:37:41 sgallagh: useable from a taskotron task? 16:38:08 does taskotron have access to dist-git ? 16:38:15 Dodji: What I mean is that we don't have to solve this today. 16:38:20 okay 16:38:27 (because another way is again a file in dist-git that tells taskotron if it should run at all 16:38:32 2 birds one stone :) 16:38:41 sorry. I just listed the issues I see to have the system be implemented :) 16:38:50 so, notifications. 16:38:55 Dodji: No, that's good. We will absolutely follow up on it. 16:39:07 I think we could use the same notification mechanisms used for the taskotron tasks today 16:39:10 (I just don't want to lose the remaining 20 minutes on that rat-hole) 16:39:15 I.e, the that runs rpmlint. 16:39:29 the *task* that runs rpmlint, sorry. 16:39:45 it sends me notifications on IRC, even. 16:39:53 Does that run on Koji builds? 16:39:58 yes. 16:40:01 I thought rpmlint only ran on Bodhi updates 16:40:10 sgallagh: that's depcheck and upgradepath 16:40:12 hmmh, now I am not sure anymore :-) 16:40:19 Huh... I don't see those rpmlint messages. 16:40:23 isn't it all flowing into fedmsg ? 16:40:25 Maybe my fedmsg isn't set up to send them 16:40:27 rpmlint is run on every koji build, notifications are handled through fmn 16:40:39 right, thanks tflink 16:40:46 Good to know 16:41:06 OK, if it's going through fedmsg, that makes notification (relatively) easy, doesn't it? 16:41:26 at least standardize-able, if not easy 16:41:51 the config in fmn may end up needing some tweaking, though 16:42:31 so, that's all for the work we did. 16:42:40 Dodji: So what is the open question on notifications? 16:43:05 sgallagh: ah, no, on notification, I was actually replying to your question at the beginning 16:43:23 I assume we'd also want to notify the creator of the build that the ABI 16:43:23 had changed. 16:43:46 sgallagh: I didn't have any open question on notification myself. 16:44:01 my open question, though, is who is going to take care of the Task? 16:44:11 the taskotron task, that is. 16:44:26 *everyone suddenly looks urgently busy* 16:44:43 lol 16:44:45 What language are taskotron tasks written in? 16:45:02 they can be written in just about whatever. for the moment, we support python the most 16:45:30 so long as it can return results in a form that we can understand, it could be written in just about anything 16:46:07 for the record, I'll provide support through hand holding and code to anything needed on the libabigail side. 16:46:10 I am committed to this. 16:46:49 bconoboy: I'm going to try to drag you into this conversation, if only because I know this topic is relevant to your interests as well 16:46:57 And you might be able to find a resource to help with this :) 16:47:25 fwiw, I'm doing a workshop on writing taskotron tasks @ devconf 16:47:50 #info tflink is running a workshop on writing taskotron tasks at DevConf.cz 16:48:13 * bconoboy wakes up 16:48:15 #info Help wanted for writing a taskotron task for automatically running ABI checks on important packages 16:48:35 sgallagh: was there a specific question for me? 16:48:36 I am going to be at Fosdem, but not at devconf, sigh. 16:48:57 seems like the scope is not limited to server 16:48:58 * Dodji always misses the cool stuff. 16:49:03 masta: indeed. 16:49:05 but this is neato 16:49:17 #info Server SIG should work with Dodji to resolve open questions about whitelisting packages to be scanned as well as mechanisms for hosting the suppression file. 16:49:46 masta: Most definitely not, but Server SIG has been talking about wanting to stabilize its advertised ABI for some time now. 16:49:55 So this is a direct benefit to our goals. 16:50:12 ah! yes an advertised abi is nice feature for server 16:50:41 s/nice/mandatory, IMHO :) 16:50:42 sgallagh: ah. Also. For the advertising. I didn't talk about this, but we do have a format for specifying an ABI, in libabigail. 16:50:52 Oh, interesting 16:50:56 the tools' name is abidw 16:51:18 you say 'abidw the-binary > the-binary.abi' 16:51:18 What exactly does that mean? 16:51:25 "specifying an ABI" 16:51:42 I would think a stable ABI is equally applicable to Desktop/Workstation users, possibly more so since there are more moving parts 16:51:45 What does that look like? 16:51:47 the-binary.abi is then a file which contains the description of all the functions, variables *and their types* 16:51:58 sgallagh: it's an XML file, sorry. 16:52:02 bconoboy: I'm not disagreeing, but someone has to drive it. 16:52:12 Dodji: Well, at least you apologize ;-) 16:52:19 sgallagh: but then, you can do: "abidiff foo.abi foo" 16:52:28 sgallagh: to compare a binary against a given abi 16:52:35 just like you compare the abi of two binaries. 16:52:48 I'm guessing this is an intermediate representation you use when comparing the binaries? 16:52:49 yeah I was wondering how the ABI graph would be expressed. xml, json, etc 16:53:02 sgallagh: yes 16:53:17 sgallagh: it's the "prettyfied" dump of the internal representation used by libabigail 16:53:20 This could still be useful in generating a skeleton for an API specification I suppose 16:53:35 s/still/also/ 16:53:47 sgallagh: completely agree 16:54:00 bconoboy: To which part? 16:54:05 some has to drive it 16:54:40 sgallagh: you asked what it looked like: http://paste.fedoraproject.org/314950/53827262 16:54:43 I'm catching up on scrollback, but basically I think the abi we ship a release with is one we should be checking updates against, minimally 16:55:12 masta: ^^ an example of the abi graph representation 16:55:16 Dodji: Well, at least it's more readable than YAML or PERL 16:55:26 sgallagh: :) 16:56:04 Dodji: Thank you again for talking with us about this today. We're going to want to follow up (probably after DevConf). 16:56:04 it's very very detailed though. 16:56:23 sgallagh: you are welcome. I am looking forward to it. 16:56:38 good stuff Dodji. Thanks. 16:56:41 Unfortunately, we're at the top of the hour and I have another meeting I need to get to. 16:56:45 nirik: thanks :-) 16:56:56 sgallagh: no pb. I talk too much, I know. 16:56:59 * Dodji has a meeting too. 16:57:02 Dodji: I'll see what I can do (or convince bconoboy to do) about finding someone to own that taskotron task 16:57:09 thanks. 16:57:31 I think the advantages to it are obvious 16:57:45 * nirik nods 16:57:47 #topic Open Floor 16:58:00 Two remaining minutes for Open Floor. Anyone have anything here? 16:58:09 * nirik has nothing off hand 16:59:21 Thanks for coming, everyone. 16:59:59 #endmeeting