21:00:14 <ignatenkobrain> #startmeeting Rust SIG (2017-02-15) 21:00:14 <zodbot> Meeting started Wed Feb 15 21:00:14 2017 UTC. The chair is ignatenkobrain. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:14 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 21:00:14 <zodbot> The meeting name has been set to 'rust_sig_(2017-02-15)' 21:00:18 <ignatenkobrain> #meetingname rust-sig 21:00:18 <zodbot> The meeting name has been set to 'rust-sig' 21:00:21 <ignatenkobrain> #chair ignatenkobrain jistone 21:00:21 <zodbot> Current chairs: ignatenkobrain jistone 21:00:26 <ignatenkobrain> #topic Agenda 21:00:30 <ignatenkobrain> #link https://docs.pagure.org/fedora-rust.sig/meetings/2017-02-15.html 21:00:32 <ignatenkobrain> #info (1) Roll Call 21:00:35 <ignatenkobrain> #info (2) Per-target dependencies 21:00:38 <ignatenkobrain> #info (3) Initial patching of crate 21:00:41 <ignatenkobrain> #info (4) Open Floor 21:00:43 <ignatenkobrain> #topic Roll Call 21:00:47 <ignatenkobrain> .hello ignatenkobrain 21:00:48 <zodbot> ignatenkobrain: ignatenkobrain 'Igor Gnatenko' <ignatenko@redhat.com> 21:00:52 <Pharaoh_Atem> .hello ngompa 21:00:53 <zodbot> Pharaoh_Atem: ngompa 'Neal Gompa' <ngompa13@gmail.com> 21:01:07 <jistone> .hello jistone 21:01:08 <zodbot> jistone: jistone 'Josh Stone' <jistone@redhat.com> 21:01:26 <ignatenkobrain> hey Akien and luke_nukem ;) 21:01:50 <Pharaoh_Atem> yay, my hello works now 21:01:55 <ignatenkobrain> Pharaoh_Atem: :D 21:02:08 <luke_nukem> Hello Akien, ignatenkobrain, Pharaoh_Atem 21:02:12 <ignatenkobrain> okay, don't expect anyone else to come, so let's start 21:02:14 <ignatenkobrain> #topic Per-target dependencies 21:02:19 <ignatenkobrain> #link https://pagure.io/fedora-rust/rust2rpm/issue/2 21:02:22 <ignatenkobrain> #link https://pagure.io/fedora-rust/rust2rpm/issue/2 21:02:59 <luke_nukem> This issue is specific to crates and compilation of them? 21:03:13 <ignatenkobrain> luke_nukem: mostly to Requires/BuildRequires 21:03:28 <ignatenkobrain> thing is that we don't need windows dependencies for our builds and cargo already aware of this 21:03:46 <ignatenkobrain> [target.'cfg(unix)'.dependencies] 21:03:47 <ignatenkobrain> libc = "0.2.6" 21:03:47 <ignatenkobrain> [target.'cfg(windows)'.dependencies] 21:03:47 <ignatenkobrain> kernel32-sys = "0.2.1" 21:03:50 <ignatenkobrain> for example 21:03:55 <Pharaoh_Atem> we only need them if we're setting up cross-compiling, but that's not a case we're handling right now 21:04:08 <jistone> option 1, package and require those anyway -- paying the (one-time) cost to struggle with winapi etc 21:04:29 <luke_nukem> ignatenkobrain: Would dealing with this now, enable use of (lay ground work for) cross-compiling later 21:04:58 <ignatenkobrain> jistone: luke_nukem: but this means all that stuff will be pulled into buildroot 21:05:06 <ignatenkobrain> which will make it very slow 21:05:10 <jistone> yes 21:05:20 <ignatenkobrain> and we are far far far away from cross-compilation 21:05:36 <ignatenkobrain> there's even RFC for that syntax: https://github.com/rust-lang/rfcs/blob/master/text/1361-cargo-cfg-dependencies.md#detailed-design 21:05:36 <ignatenkobrain> Writing tokenizer shouldn't be hard 21:05:48 <jistone> "very slow" - do you think it's really so bad? 21:06:04 <ignatenkobrain> jistone: metadata is quite huge, so yes. 21:06:19 <ignatenkobrain> metadata == rpm-md 21:06:41 <ignatenkobrain> I even tried to start packaging kernel32-sys and it has some number of dependencies 21:06:49 <ignatenkobrain> I just gave up 21:07:01 <jistone> option 2, patch these deps out, like #20 proposes 21:07:22 <jistone> option 3, write a proper tokenizer 21:07:32 <ignatenkobrain> jistone: you mean to completely strip that stuff from md? (2) 21:07:32 <jistone> option 4, write rust2rpm in rust and use the cargo crate 21:07:52 <jistone> I mean strip stuff from Cargo.toml 21:08:10 <ignatenkobrain> yeah 21:08:19 <luke_nukem> I would lean towards Option 4. 21:08:44 <jistone> luke_nukem, we've avoided that so far because it presents bootstrapping challenges 21:08:59 <jistone> need the tool to make crates needed to build the tool... 21:09:02 <ignatenkobrain> yeah, bootstrapping is not trivial 21:09:12 <ignatenkobrain> and basically cargo itself is a tool as well 21:09:15 <ignatenkobrain> brr 21:09:16 <ignatenkobrain> crate 21:09:17 <luke_nukem> I've experienced that hassle myself, yeah.. 21:09:32 <ignatenkobrain> so we will rebuild stuff at least 3 times? 21:09:41 <luke_nukem> cargo now requires cargo+rust to build, whilst rust requires cargo+rust... 21:10:03 <ignatenkobrain> and then on top of it we build rust2rpm and rebuild cargo once more :D 21:10:49 <luke_nukem> ignatenkobrain: But cargo is only a tool, it doesn't insert any code/changes in to compilation. 21:10:49 <ignatenkobrain> so I woul go (2) if no one could do (3) in very short period 21:11:22 <ignatenkobrain> luke_nukem: it is tool + crate AFAIK, so depgen needs to be present 21:11:46 <ignatenkobrain> jistone: btw, thanks for nice summary ;) 21:12:29 <jistone> at the moment, for cargo I include a cargo-$version-vendor.tar.xz for its crates 21:12:50 <jistone> and because of bootstrapping, I'm not certain we should ever do otherwise... 21:12:51 <jistone> we'll see 21:13:42 <luke_nukem> (rust, bootstrapped with rust+cargo bootstrap) cargo (built with fresh rust, +cargo bootstrap), rust (rebuild with fresh rust, and cargo)? 21:15:06 <jistone> and one more cargo with fresh rust+cargo 21:15:46 <luke_nukem> Getting back to the issue: Option 2 is fastest route? If that path is taken, what is the effect on future option 3 or 4? 21:16:28 <ignatenkobrain> (2) is solution "right now" 21:16:35 <jistone> we'd just stop patching it out when 3/4 works 21:17:07 <luke_nukem> jistone: Yes, sorry left that out. So once that's built, then the rust written crate2rpm can be built, yes? 21:17:12 <luke_nukem> And only once? 21:18:01 <jistone> depends on what that crate2rpm depends on. if only cargo, then yeah, we're done 21:18:04 <jistone> (I think) 21:18:14 <Pharaoh_Atem> I think it would only be cargo 21:18:23 <ignatenkobrain> jistone: it will depend on semver and such 21:18:29 <ignatenkobrain> so only cargo is not enough ;) 21:18:31 <Pharaoh_Atem> semver is part of rust lib, isn't it? 21:18:34 <ignatenkobrain> definitely some http stuff and etc. 21:18:48 <ignatenkobrain> Pharaoh_Atem: no, it's different crates (after all) 21:18:54 <Pharaoh_Atem> ah 21:18:55 <jistone> ignatenkobrain, I'm hoping the cargo crate can handle that for us, but I haven't looked at its api 21:19:54 <luke_nukem> (2) followed by (4), fallback being (3)? 21:20:35 <jistone> ignatenkobrain though a parser would be easy (3) 21:20:42 <jistone> I'm more skeptical 21:20:57 <jistone> but he's been doing most of the work so far, so ? 21:21:00 <ignatenkobrain> I prefer (2) -> (3) 21:21:19 <ignatenkobrain> since yeah, I mostly do all the stuff :D 21:21:36 <ignatenkobrain> apart from rustc/cargo where Josh does great job 21:21:40 <luke_nukem> Sounds fair to me 21:22:31 <ignatenkobrain> #agreed we will just drop all non-fedora target dependencies manually from Cargo.toml 21:22:42 <ignatenkobrain> #topic Initial patching of crate 21:22:46 <ignatenkobrain> #link https://pagure.io/fedora-rust/rust2rpm/issue/20 21:22:52 <ignatenkobrain> So, here's patching itself :D 21:23:10 <ignatenkobrain> my problem with packaging libc was something like: 21:23:21 <ignatenkobrain> `cargo metadata` fails due to workspace 21:23:36 <jistone> I'd like to solve that particular case, and there's an action on me from last week 21:23:39 <ignatenkobrain> (this issue is on Josh's tracking) 21:23:43 <jistone> but we'll want patching in general anyway 21:23:51 <jistone> right 21:24:03 <ignatenkobrain> yeah, we'll need to do patching anyway 21:24:18 <ignatenkobrain> my crazy idea was to split generation action into multiple steps 21:24:41 <ignatenkobrain> like Download -> Unpack -> Get diff -> Run generation 21:24:54 <ignatenkobrain> though not sure how to do this better from user's POV 21:25:02 <ignatenkobrain> how would you like to do such initial patching? 21:25:19 <ignatenkobrain> e.g. as we discussed couple of lines above to remove dependencies from Cargo.toml 21:26:06 <jistone> can you have these as separate actions, but a meta-action to do them all for the non-patched case? 21:26:35 <ignatenkobrain> in theory yes, but what would be separate actions? 21:26:53 <ignatenkobrain> just interested how it should look like from user side 21:28:07 <jistone> perhaps: rust2rpm fetch foo; (figure out patching); rust2rpm apply foo foo.patch; rust2rpm generate foo 21:28:08 <luke_nukem> ... Download, pull cargo.toml from archive, patch and diff, generate rpm and spec to include the generated patch? 21:28:48 <jistone> maybe just two, fetch and generate, where the latter can take patch arguments 21:28:58 <luke_nukem> Is rust2rpm requiring the user to supply a patch, or generating it itself? 21:29:12 <ignatenkobrain> luke_nukem: that's what we're trying to figure out here 21:29:16 <luke_nukem> ah.. 21:29:32 <jistone> if it can be automatically patched, you don't need multiple steps 21:29:46 <jistone> but that's only true for known "bad" cases like workspaces 21:30:00 <Pharaoh_Atem> it'd be better if it could generate patches, but it might also need to have a no-patching mode to support manual patching 21:30:15 <ignatenkobrain> jistone: workspaces are not always break stuff ;) 21:30:24 <jistone> there could be something more subtle, like an optional dep with patent concerns, which we need to block 21:31:08 <luke_nukem> Perhaps two paths if possible "rust2rpm gen-with-patch" which would auto-gen the patch, "rust2rpm gen --patch new.patch" to specify a patch? 21:31:12 <ignatenkobrain> probably rust2rpm fetch foo would download .crate, then rust2rpm prep would unpack it, then you change files inside that dire and rust2rpm generate will generate patch and apply it? 21:31:24 <jistone> we'll also want to sometimes strip whole sources, like the bundled openssl in openssl-sys crate. a patch is ugly for that 21:31:29 <Pharaoh_Atem> sounds like a quilt workflow... 21:31:44 <ignatenkobrain> jistone: that's what rm -rf is for in %prep 21:31:45 <ignatenkobrain> ;)_ 21:32:14 <jistone> sure, just not sure if you want rust2rpm to insert that, or always add it manually 21:32:15 <ignatenkobrain> rust2rpm is not supposed to generate 100% good spec 21:32:19 <jistone> ok 21:32:41 <Pharaoh_Atem> I don't think we can get to pyp2rpm levels of automated packaging just yet anyway 21:32:42 <ignatenkobrain> it's mainly to get BuildRequires and boilerplate stuff 21:32:59 <ignatenkobrain> pyp2rpm doesn't solve this problems either 21:33:17 <ignatenkobrain> Pharaoh_Atem: and automatic packaging we already have in rust2rpm 21:33:25 <ignatenkobrain> it generates buildable spec 21:33:29 <Pharaoh_Atem> yes it does 21:33:51 <luke_nukem> I need clarification: rust2rpm, the purpose is to be used by another spec/build, to generate a new spec/build? Or is it to be used by a user to generate a one off package? 21:34:07 <Pharaoh_Atem> we just don't have the empirical build data to make sure it handles every corner case yet :) 21:34:15 <ignatenkobrain> luke_nukem: what do you mean? 21:34:39 <jistone> it's a tool for packagers, we humans 21:34:47 <luke_nukem> Gotcha 21:35:36 <jistone> generate a spec which one can then take to review and get into the distro proper 21:35:43 <ignatenkobrain> we're not trying to implement any AI here like to determine proper license files, remove bundled deps and etc. 21:35:51 <Pharaoh_Atem> not yet, anyway :P 21:35:53 <jistone> tangent - any thoughts on how to update deps later? 21:36:07 <ignatenkobrain> jistone: so I had in mind `rust2rpm diff ` 21:36:11 <jistone> cool 21:36:18 <ignatenkobrain> so it will do diff between 2 versions 21:36:25 <ignatenkobrain> in terms of description/metadata 21:37:05 <luke_nukem> ignatenkobrain: to find out what packages need updating/rebuilding? 21:37:30 <ignatenkobrain> luke_nukem: and how you should change your spec, yes 21:38:03 <jistone> e.g. foo-1.1 now requires bar, have to add that 21:38:21 <ignatenkobrain> yup 21:39:00 <luke_nukem> and bar-1.2 is still using an older zyx package, so it also tells us which version it requires? 21:39:25 <jistone> I think we only have to look at that for bar.spec 21:39:34 <jistone> rpm can handle transitive dependencies for us 21:39:50 <ignatenkobrain> s/can/does/ ;) 21:41:18 <ignatenkobrain> probably `rust2rpm generate-custom` will download crate and drop us shell in unpacked sources 21:41:29 <ignatenkobrain> once you EOF, it generates diff and applies patch 21:41:32 <ignatenkobrain> what do you think? 21:41:47 <luke_nukem> Hmm.. So Cargo.toml specifies the crates and versions required, crate2rpm diff will give this as BuildRequires rust-xyz == version? 21:42:04 <ignatenkobrain> luke_nukem: usually >=, but yes 21:43:07 <ignatenkobrain> if we follow generate-custom as I described above, then we can't easily do diff =( 21:43:08 <luke_nukem> ignatenkobrain: ^^ that sounds like a workable soultion 21:43:21 <ignatenkobrain> although we can't do it either 21:43:48 <ignatenkobrain> stuff is too fragile 21:43:52 <jistone> auto-diff seems too fancy to me 21:44:40 <ignatenkobrain> actually we can have --patch ... which will always apply some patch 21:44:47 <jistone> rust2rpm prep (or whatever); rust2rpm generate --patch foo.patch 21:44:49 <ignatenkobrain> so should be easy to automate stuff 21:44:56 <jistone> you can do the latter directly if you need no patches 21:45:32 <ignatenkobrain> jistone: what do you think about (10:41:17 PM) ignatenkobrain: probably `rust2rpm generate-custom` will download crate and drop us shell in unpacked sources (10:41:28 PM) ignatenkobrain: once you EOF, it generates diff and applies patch 21:45:48 <ignatenkobrain> or wait, auto-diff was comment about that? 21:45:52 <jistone> yes 21:46:09 <Pharaoh_Atem> that's basically the quilt workflow 21:46:30 <jistone> hmm, can we literally use quilt? 21:46:34 <luke_nukem> 'rust2rpm prep', do the work required in the workspace, then 'rust2rpm generate', this would create a second workspace, diff between the two, then generate rpm spec + patches? 21:46:47 * jistone hasn't used it before 21:47:02 <ignatenkobrain> the problem with multiple independent steps is that you have to keep track of directories and so on 21:47:08 <ignatenkobrain> which means more code :D 21:47:22 <jistone> you don't have to care with the way I wrote it 21:47:37 <jistone> an explicit --patch means there isn't any state between steps 21:47:39 <ignatenkobrain> while if you are dropping user into shell, you know when user finishes with it 21:48:17 <ignatenkobrain> well, rust2rpm-prep is basically tar xf 21:48:32 <jistone> with a fetch beforehand, if needed 21:49:05 <Pharaoh_Atem> jistone: I don't use quilt, but my understanding is that Debian and OpenSUSE guys use it a lot 21:49:44 <jistone> if any existing tool will already do this, we can avoid NIH 21:49:55 <Pharaoh_Atem> here's Debian's guide about quilt: https://wiki.debian.org/UsingQuilt 21:49:58 <jistone> worth investigating, at least 21:50:25 <Pharaoh_Atem> quilt might not be the right choice, but at least it could be inspiration on how to do this right 21:50:38 <jistone> yeah 21:51:00 * ignatenkobrain wants to avoid large patching, everything has to go to upstream 21:51:12 <ignatenkobrain> apart from our workarounds :D 21:51:33 <jistone> right, keep hacks local, but otherwise upstream-first applies as usual 21:51:40 <Pharaoh_Atem> terrifyingly, quilt is written in shellscript 21:51:47 <ignatenkobrain> Jesus 21:51:54 <Pharaoh_Atem> http://git.savannah.nongnu.org/cgit/quilt.git/tree/ 21:51:56 <luke_nukem> really the only patching that should be getting done is just the bare minimum to package, right? Which in this case is some platform/target config related stuff 21:52:04 <Pharaoh_Atem> yes 21:52:14 <luke_nukem> Pharaoh_Atem: That *is* terrifying. 21:52:16 <ignatenkobrain> luke_nukem: yeah, all other patching is our usual RPM patching 21:52:27 <jistone> ignatenkobrain, do you have enough ideas to go experiment with this? 21:52:55 <ignatenkobrain> jistone: I think I have at least 2-3 ideas, so will try them out 21:52:58 <luke_nukem> So, in this case, the tool has a relatively simple job? 21:53:27 <Pharaoh_Atem> yes 21:53:34 <Pharaoh_Atem> really, there's only two cases of patching: 21:53:41 <luke_nukem> ignatenkobrain: I'm happy to colaborate if you need another hand. Might take me a day or two to get everything in place and familarize myself. 21:53:47 <Pharaoh_Atem> 1. incompat (such as the workspace in libc) 21:54:03 <Pharaoh_Atem> 2. ripping out patented stuff (such as codec stuff) 21:54:06 <ignatenkobrain> #action ignatenkobrain to experiment with approaches, compare them and choose the best 21:54:45 <ignatenkobrain> so, I think we're done with this at least for today 21:54:48 <ignatenkobrain> #topic Open Floor 21:55:20 <ignatenkobrain> anyone has something to talk about ? 21:55:24 <ignatenkobrain> we have ~5 minutes 21:55:24 <jistone> testing and karma would be welcome on the rust updates in f24, f25, el7 21:56:28 <jistone> belated welcome and thanks to luke_nukem for expanding our distro breadth :) 21:56:31 <ignatenkobrain> #info test and leave karma for rust updates 21:57:12 <luke_nukem> Thank you for inviting me to take part :) 21:57:41 <Pharaoh_Atem> now we have people from Fedora, Mageia, and openSUSE :D 21:57:54 <ignatenkobrain> just quick question, everyone is happy to managing meetings via pagure / announcements etc. ? ;) 21:58:04 <Pharaoh_Atem> I don't have a problem with it 21:58:20 <Pharaoh_Atem> only annoying thing is that there's no decent content search in it, but for this, it's fine 21:58:39 <luke_nukem> ignatenkobrain: Worked well for me. The countdown timer/format/timezones. Worked well. 21:58:54 * ignatenkobrain is happy then 21:59:39 <Pharaoh_Atem> I'm glad you post all the time zones 21:59:47 <Pharaoh_Atem> makes it much easier for me to put it into my calendar 21:59:49 <luke_nukem> ^ very helpful 21:59:59 <luke_nukem> Also iCal export was good 22:00:10 <ignatenkobrain> iCal export is great in fedocal 22:00:14 <ignatenkobrain> so, thanks everyone for attending and see you on #fedora-rust and here next week! 22:00:14 <ignatenkobrain> #endmeeting