16:05:19 #startmeeting go_sig_meeting 16:05:19 Meeting started Wed Dec 5 16:05:19 2018 UTC. 16:05:19 This meeting is logged and archived in a public location. 16:05:19 The chair is Pharaoh_Atem. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:05:19 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:05:19 The meeting name has been set to 'go_sig_meeting' 16:05:31 Pharaoh_Atem, this is the part where we wait for others like jcajka to arrive 16:05:35 #topic Roll call 16:05:42 #topic Roll Call 16:05:56 interesting, the bot isn't responding to me 16:05:59 .hello2 16:06:00 jcajka: jcajka 'None' 16:06:09 .hello ngompa 16:06:10 Pharaoh_Atem: ngompa 'Neal Gompa' 16:06:20 .hello2 nim 16:06:21 nim: nim 'None' 16:06:31 it is not responding to topic IIRC 16:06:41 try #chair 16:06:55 #chair Pharaoh_Atem jcajka 16:06:55 Current chairs: Pharaoh_Atem jcajka 16:07:57 so is that it? 16:08:01 no one else? 16:09:03 bhavin192 pgier deparker 16:09:07 ^^ 16:09:19 zyga, carlwgeorge ? 16:09:24 Pharaoh_Atem, it would be awfuly nice to have eclipseo with us, but I haven't seen him lately 16:09:34 yeah, he kinda disappeared... 16:09:52 hey 16:10:02 meeting? 16:10:06 Yep 16:10:08 say hello! 16:10:09 Yes! 16:10:13 nice, thank you for pinging me 16:10:34 you can use either .hello2 or .hello 16:10:44 .hello zyga 16:10:45 zyga: zyga 'Zygmunt Krynicki' 16:11:08 oh am i in this sig now? :D 16:11:14 if you want to be :) 16:11:20 sure why not 16:11:29 is there a badge? 16:11:31 ;-) 16:11:32 i've been doing more and more go pkgs lately 16:11:39 for reference, .hello2 uses your nick to map to fas, while hello will let you pass fas id as argument 16:11:48 zyga: maybe one day! 16:11:49 what's the agenda? 16:11:55 I'll get to that in a moment 16:12:08 I can explain my desire to improve cross distro packaging via indigo project 16:12:31 (which is not ready but sufficiently useful to discuss, perhaps) 16:12:31 anyway, onto the next bit 16:13:05 #topic Agenda: https://pagure.io/GoSIG/go-sig/issue/20 16:13:23 The agenda for this meeting: https://pagure.io/GoSIG/go-sig/issue/20 16:13:37 that's the heavy subject for today 16:13:47 #undo 16:13:47 Removing item from minutes: 16:13:55 that's actually not the agenda :D 16:14:07 yep :) 16:15:03 #topic Voting rules for Go SIG proposal: https://pagure.io/GoSIG/go-sig/pull-request/15 16:15:16 this was the item from last time that jcajka was working on 16:15:50 we all approved it because we have confidence in jcajka 16:16:21 Okay, then 16:16:27 I have forgotten to sent out the notice, but I would merge it if you all agree 16:16:46 Why don't we just go ahead and vote now for it? 16:16:51 I'm gravy with it: +1 16:17:06 we probably don't have quorum 16:17:16 based on the rules :) 16:17:23 * jcajka +1 16:17:28 the rules don't apply yet, unless we want to do that now :P 16:17:48 yes you really need to lower the quorun unless more people join the meeting each month ;) 16:18:04 or just stick to the in ticket voting, wich could work 16:18:07 * nim +1 16:18:16 looks good to me, so +1 pending my addition to the sig 16:18:36 that's easy enough to take care of right now, I think 16:18:59 just followed process and commented on https://pagure.io/GoSIG/go-sig/issue/1 16:19:39 carlwgeorge, https://pagure.io/group/GoSIG 16:20:20 I've added carlwgeorge to the SIG FAS group 16:20:33 carlwgeorge, you can even have two seats! Really what name do you prefer? 16:20:50 carlwgeorge, with or without w? 16:22:21 so we have four votes so far 16:22:21 Pharaoh_Atem, I also added pagure-side (twice) 16:22:41 šŸ‘ 16:22:53 with the w, without was my old fas that is deactivated 16:23:35 at this point, I guess we'll consider it good, but jcajka, we might want to lower the minimum bar because it might get hard to have five people every time 16:24:15 I think 3 would be realistic for now 16:25:39 I would say we will be better to stick to the in ticket voting, rather then lowering quorum as it would allow meeting hijacking, i.e. someone just pushing his agenda when everyone is on PTO or so 16:25:58 okay 16:26:22 do we have a proposal for ticket voting? 16:26:23 bypassing feedback from everyone 16:26:31 it is in the voting rules 16:27:02 Pharaoh_Atem: would you merge the PR, if we agree 16:27:08 sure 16:27:33 #agreed Voting rules will apply, we'll primarily do ticket voting until we can have quorum regularly in meetings 16:27:48 Merged 16:28:04 jcajka, honestly, Iā€™m pretty sure metting or ticket, it will be hard to find people to review and give an opinion on things 16:28:04 thanks :) 16:28:52 it won't be easy, but we shouldn't be doing arbitrary stuff, not agreeing on something is also some form of statement 16:29:20 and we can ask FESCO, FPC to decide 16:29:56 #topic Updating and Clarifying Go Packaging Guidelines Draft: https://pagure.io/GoSIG/go-sig/issue/3 16:30:36 Where are we on this? It's a prerequisite for moving forward for almost everything for Go packaging 16:31:55 my opinion is that the wiki page can be removed 16:32:12 do we have a replacement? 16:32:20 it's too long and I've been told in no uncertain terms by FPC members they would never review it 16:32:51 jcajka: what do we want to do here? 16:32:59 so I'd rather have people review the new clean macro project 16:33:05 from my point of view this is also blocker for issue #20, I very much appreciate work that is nim doing, but we need some specs that his implementations are adhering to(I haven' seen any unite tests for what he implemented) 16:33:23 check what they want to be changed or not 16:33:44 and then push the documentation in there to FPC when we all agree that's what we want to do 16:35:01 to be honest, I have no idea what everything does yet either 16:35:13 jcajka, can you point me to unit tests for other macros? The unit test seems to be if it builds the stuff we want in koji, it works 16:35:15 and I'm not sure how well this plays nicely with mixed language packages 16:36:09 nim: could you point me ti o what are you macros implementing/supposed to do? 16:36:20 are your macros 16:37:01 s/ti o/to docs... sorry 16:37:40 jcajka, there are long lines of description of the intent before each macro 16:37:41 https://pagure.io/go-rpm-macros/blob/master/f/rpm/macros.d/macros.go-rpm#_152 16:38:01 could you put it in to the guidelines wiki? 16:38:03 * zyga goes away for now, ping me if anyone wants to learn about or discuss indigo 16:38:08 and you have high-level templates that describe how it's all supposed to work together 16:38:39 https://pagure.io/go-rpm-macros/blob/master/f/templates/rpm 16:39:07 that is not definition of what is it supposed to do, it is what it does, which might be two totally different things 16:39:36 jcajka, really, no. That will take ages to rewrite wiki-style, and then FPC will tell you they expect another format, and it will all end up desynced from the actual code 16:40:12 jcajka, we have a nice repository now, if you want to change the templates of explanations just do a commit 16:40:26 I want guidelines... 16:40:53 jcajka, I know but everyone else seem to want another form of guidelines 16:41:11 what have you created is awesome, but also docs side needs to be done, I know it is hard and kind of unpleasant(at least to me), but it needs to be done 16:41:17 jcajka, when i write your kind of guidelines they accuse me of not documenting because they find them useless 16:42:04 I know what your implementation is doing, but I have no idea what is should do... 16:42:54 jcajka, well from my point of view it should do what it's doing, otherwise that's a bug 16:43:07 what is the it? 16:43:29 some whim of yours? something that fedora needs? 16:43:32 jcajka, and the documented templates should work when fed into rpm mock koji copr otherwise it's a bug too 16:43:57 many things could "work", but still be broken 16:44:20 jcajka, something that I found useful when packaging several hundreds of spec files 16:45:08 as much I hate this academic discussion, we are striving to have some sort of canonical rule set that is more broder that someone one individual found working 16:45:24 and there are decent reasons to document intended behavior 16:45:38 because we may need to change it, or expand it, depending on how things change in the future 16:45:46 Pharaoh_Atem, the packaging templates are the intended behaviour 16:45:49 that is reason why it haven't been done by us in the past as it is quiet a big undertaking 16:46:18 welp, I g2g 16:46:20 jcajka: the floor is yours 16:46:32 Pharaoh_Atem, because the intended behaviour is to produce working packages, so it starts with the spec you want to write 16:46:44 nim: you might get annoyed by us and leave an now we have no idea what your take has been 16:47:35 jcajka, again, please read the templates or the code comments, everything is explained there 16:48:17 nim if you can put the templates in to some form of CI(along with all the use case) it will be enough, if you don't want to write "docs" 16:48:43 but I have no idea if that is even possible for macros... 16:49:41 jcajka, that's why I wrote the Fedora CI was just building packages all day long in koji and copr and opening tickets when the result does not please someone 16:50:30 jcajka, you do not want guidelines you want a formal specification 16:50:40 jcajka, but no one is going to write that 16:51:06 jcajka, even if someone wrote that no one is going to check manually an implementation conforms 16:51:41 jcajka, and practically, this is all so low-level I don't see how to do unit tests on it 16:51:44 nim: so you think that having some ad-hoc obscure macros set is better? 16:52:18 jcajka, the best thing you could do is give QA people requirements hand perfome tests on the produced packages 16:52:42 tbh I have expected that this will end up this way... my main reason why I have been so pessimistic about it from start 16:52:59 nim we can go in direction that nobody will ever use such macros... 16:53:19 jcajka, I think it is way better because it produces actual working packages and an hypothetical formal spec does not 16:53:38 do what happens if your macros do not work? 16:53:42 hey, do you guys want to know about indigo? 16:53:47 as opposed to macros? 16:53:57 I'm not following the SIG work closely 16:54:05 I'm an upstream that struggles with go packaging 16:54:06 jcajka, you're asking for things no one does in Fedora. And yet it is used. Whay? Because it produces working packages and that's what people need 16:54:13 and I can share my upstream perspective on packaging 16:54:25 and explain why I decided to try to improve the status quo by making indigo 16:54:56 jcajka, if the macros do not work it's a bug and it needs to be fixed. Like for any other software 16:55:18 nim: with the old macros we have also working package, based on your logic we don't need your work 16:55:42 nim and how fixed? what is the expected "working" state 16:55:50 jcajka, if you hit a problem with any other fedora or red hat macro you open a bug 16:56:09 and it is magically fixed... 16:56:28 jcajka, and someon interacts with you in the ticket and fixes stuff or not 16:56:37 jcajka, that's the bazaar 16:57:11 jcajka, I had things I needed fixed, and upstreams didn't want to fix it, so I fixed it my side 16:58:00 jcajka, and a formal spec would not change anything because if people do not want to do stuff, they won't do it spec or not. That's the difference between community and paid-fore development 16:58:39 zyga: I think we'd be certainly interested in Indigo 16:58:56 * Pharaoh_Atem is not back, just briefly glancing here 16:59:01 jcajka, you never *needed* my work, because anything you automate in a macro, you can also do manually 16:59:12 jcajka, that's just a lot of work 16:59:25 jcajka, a lot more work 16:59:50 zyga, I'm interested too, please wait a sec we finish this 17:00:06 sure :) 17:00:42 jcajka, so basically, the whole point and intent of the macros, it to permit the documented spec templates 17:01:04 as you mentioned somebody needs to do the work and docs enables us to do the work even if you are unwilling as we know what the macros should do 17:01:10 jcajka, where you package go software in a few dozen lines easy to check 17:01:13 or unable 17:01:32 jcajka, instead of writing hundreds of manual lines full of human errors 17:01:52 we have hundred of lines of macros with human errors... 17:02:34 jcajka, you have hundred of lines of macros where the code is documented and tracked 17:02:40 difference is when bugs are found they can be fixed in one spot rather than many spec files 17:02:51 IMO this is not productive let's see in issues 17:03:20 jcajka, and FYI the latest round of simplifications permitted something like 13000 removals in our internal repository 17:03:38 and what in fedora? 17:04:31 I know that you care about your internal project when you have been flooding me with request to fix stuff for you, but do you about Fedora? 17:04:43 jcajka, there is not reason for Fedora to be different, when you have hundreds of Go specs to manage like Fedora has, any simplification not matter how trivial it seems has a huge multiplication factor 17:05:57 nim, zyga: what about indigo :) 17:06:00 jcajka, and my internal project is pretty much the same as Fedora 17:06:08 so 17:06:18 hey, I'm Zygmunt 17:06:23 my day job is to write go code 17:06:32 sometimes I work on making that available as a system package 17:06:39 jcajka, except it serves as testbed for all the things you do not want to look at before they are solid 17:06:43 then I need to wear the downstream hat 17:06:54 I had experiences with fedora, debian, suse, ubuntu and arch 17:07:08 in all those places go has some sort of infrastructure that "aids" in the build process 17:07:19 #topic Indigo 17:07:28 jcajka, well they are solid, since hundreds of Fedora packages build at first pas because of the testbed testing https://pagure.io/go-rpm-macros/blob/master/f/templates/rpm 17:07:32 my experiences were that it was often hard to understand what's behind a rpm macro or a debhelper machinery 17:07:58 perhaps with naive optimism I was driven to create a new project that aims to make golang packaging sane from the point of view of an upstream developer 17:08:18 the main problem with downstream packaging was reproducing the environment that lead to test or build failure 17:08:21 nim: let's discuss that in issues 17:08:46 as an upstream developer you are used to just using GOPATH, building and testing code 17:09:18 at least in opensuse I was forced to use macros that expanded to very very verbose code that did things I would never do as an upstream developer 17:09:26 not because of particular distribution policy 17:09:32 but because of someone just implementing it that way 17:09:42 :) 17:10:02 I recognise the need for distribution helpers to handle things like build policy, hardening flags, collecting debug data and alike 17:10:20 but I wanted to provide that in a way that doesn't feel alien to people outside of a packing circle 17:10:36 and in a way that allows upstream developer tasked with fixing a build failure on $distribution be comfortable with that task 17:10:40 indigo is a go indirection layer 17:11:02 it can be used as go binary to build, test or install 17:11:06 it can be used directly 17:11:25 and based on my recent experiences I will also make it so that it can be used as $(indigo build), that is to expand to simple shell commands directly 17:11:40 indigo is a mechanism for injecting distribution specific build policy into a go package 17:12:01 the idea is that indigo reads a declarative policy, in my idea it would be coming with a particular toolchain package or a meta package 17:12:12 and expands that policy to changes to various commands 17:12:18 in suse that is as simple as -buildmode=pie 17:12:23 in fedora it can be more comprehensive 17:12:40 I am happy to accommodate all needs of each distribution willing to use it 17:12:58 version 0.3 that I would like to release this weekend will support all concepts used inside suse 17:13:18 indigo is written in go, has tests and some documentation (I need to write some more) 17:13:34 I'm working on adopting it for packages that I need to maintain 17:13:43 and I'm interested in collecting feedback 17:13:49 indigo doesn't replace macros that distributors use 17:13:53 zyga: do could you post link to your project? 17:13:56 but it should, if done properly, reduce their scope 17:14:01 sure, gitlab.com/zygoon/indigo 17:14:11 zyga: thanks 17:14:12 note that I haven't pushed 0.3 changes yet, so this is somewhat older than what I have locally 17:14:39 and the most important part, allow developers to feel comfortable with this distribution downstream packaging world 17:15:00 it's also supposed to be sane, the output of "go build" is not very long verbose and alien, it is just go build with extra build flags 17:15:06 I'm sure some of my assumptions are wrong 17:15:16 some things need extra logic to handle what distributions want to do 17:15:18 but that's the main idea 17:15:31 that's that, I'm happy to answer any questions 17:15:43 oh 17:15:56 indigo has been slowly going into opensuse, not formally and officially but just as another way to build 17:16:01 I don't believe anyone else is using it 17:16:06 but it got into devel:languages:go 17:16:21 I'm trying to gather interest from the opensuse golang maintainers but that team has been volatile recently 17:16:29 I did get encouragement to do new things 17:16:40 as the state of their packaging macros was in some disarray 17:16:52 to the point where maintainers recommended against using them 17:16:53 that's it 17:17:04 * zyga thanks everyone for listening 17:17:13 thanks zyga 17:17:45 zyga: thanks it is really interesting, i have been playing in the past with idea about something similar more focused on extending go get to understand the packaging of distros, i.e. installing deps from distro packages, haven't done much in that direction 17:18:08 jcajka: I decided not to attempt to do that because it's bigger in scope than I felt comfortable with 17:18:22 the problem with that is that you either allow vendoring or not allow vendoring 17:18:31 and that impacts design very strongly 17:18:46 I'm looking at the go module system as a possible future solution for the problem 17:18:59 where at least distributors can manage it with more explicit mapping 17:19:11 zyga: I have not gone that far, but I still thing that there should be no vendoring in ideal world 17:19:29 I'm thinking in same direction, but haven't looked at it yet 17:19:40 jcajka: realistically as an upstream I need to deal with both preferences 17:20:07 jcajka: one thing I would love is to see more version awareness in the module system going forward 17:20:11 as now upstreams don't know what they require 17:20:19 since they always require what's on their disk 17:20:21 which is not great 17:20:23 but that's reality 17:20:39 once that is in place the go module system seems like a natural boundary for packaging in distributions 17:20:43 but that could take a few years 17:20:46 zyga, yes the root of the problem is being to translate the messy way upstreams release Go projects, in the regular and formal description rpm expects 17:20:57 and I was determined to build something smaller that would help me with daily issues 17:21:03 yup, it will be long way to get rid of it and in some instances when there is careful thought it is indeed useful 17:21:18 zyga, we've almost not worked of enhancing the go build part 17:21:39 zyga, it's two lines of macro definitions with default build flags 17:22:01 nim: I can share one interesting perspective of how opensuse packaging used to work (or still works to some degree) 17:22:11 zyga, most of the work is to get to the point where those two lines can be used, and then to pick the result 17:22:13 installing a toolchain gives you globally set GOPATH, GOARCH and the usual suspects 17:22:28 so you can have a project that "go test"s just fine 17:22:47 but fails when built in obs (like koji but in suse) 17:22:58 some things also depend on if you are root or not 17:23:07 specifically GOBIN is set then 17:23:36 the way the system is prepared for building a package was also somewhat inefficient and alien to what upstream work accustomed me to 17:23:44 so one thing I can do with indigo 17:23:50 is to "indigo build" my tree 17:23:56 and immediately see if it would pass in upstream context 17:24:16 if I'm unaware of using a dependency that is not packaged 17:24:19 or packaged but incompatible 17:24:24 that's an asset to my daily work 17:24:39 in addition, I could do this without extra compexity of figuring out what %gobuild does today 17:24:42 how to expand it 17:24:51 how to follow the set of macros and shell scripts that do weird stuff 17:25:10 (and believe me, in suse it was very weird lately, where the system changed three times, or so) 17:25:17 zyga: we don't touch that stuff(in most case) in Fedora, so we have more upstream experience, you need to set it by hand in most cases 17:25:22 so it was mainly a mental compatibility layer 17:26:19 I will work on the 0.3 release, I would love to bring that to fedora even though at this stage it's not of much use 17:26:26 I would also be happy to maintain the package 17:26:38 zyga: I trust you, I have/had to deal with the complex systems when debugging or porting package for different arches, that makes me thing that even current system of macros(that are rather simple and naive) in Fedora is fairly complex for outsiders 17:26:42 and start to toy with expressing some of the policy this way 17:27:02 zyga, we have (or will have) very complex macros to identify the tar.gz corresponding to an import path, unpack it cleanly, so go build can be run 17:27:03 I believe you... 17:27:17 zyga, but we do not mess with the go compiler 17:27:37 the only thing I would like to add to the go compiler is that it could ship the policy file 17:27:43 zyga, at least, that's jcajka's role, and I'm not aware he is messing with it 17:27:45 so installing a toolchain could teach indigo how to behave 17:27:54 the policy file is a json file with some declarations 17:28:17 this is the indigo 0.2 spec file from opensuse https://build.opensuse.org/package/view_file/devel:languages:go/indigo/indigo.spec?expand=1 17:28:25 I did some tricks there to optimise the build process 17:28:42 it doesn't matter here but it matters for larger packages (with vendorized deps) 17:28:50 especially when you are trying to shave seconds from C I 17:29:01 the trick is that there's no %setup 17:29:17 and that the source tarball is unpacked into the right spot straight away 17:29:21 this way there's less IO 17:29:28 and faster iterations in CI 17:29:29 zyga: yup, I'm trying to keep the experience as close to upstream, when you install the compiler(so you are kind of clueless, but in the upstream way, not with some set of EXOTIC env vars) 17:29:38 yep 17:29:52 jcajka: I like the idea where $(indigo build) expands to something that is appropriate 17:29:54 and understandable 17:30:19 indigo can also tell you about the policy but this is more direct and easier to find out about :) 17:30:22 anyway, that's all I have 17:30:27 please leave me questions if you have any 17:30:32 I'm happy to collect feedback 17:30:50 it's a background project but something I firmly believe in as a step, at least now with the current set of the world :) 17:30:55 I need to go AFK 17:31:02 I should be back in ~3 hours 17:31:19 zyga, practically we do not have much io either, we do use setup but we contruct the intermediary GOPATH tree via symlinks to the archive roots, so it's cheap and fast 17:31:57 what to do with the result to beat it into shape is something esle 17:33:42 zyga: I will bug you about how add Fedora build rules and hot to use it in general, etc. I hope over the courser of late Dec I should have some more time 17:34:14 Thank you 17:35:46 zyga, it would be nice if you could take a look at what we do and tell us where you think we are doing wrong 17:36:26 zyga, but, as far as I see, you're working on the only part we have not tried to automate heavily yet 17:37:23 zyga, in Fedora 17:37:29 Gladly. Could you please share some pointers to what to look at? 17:38:16 zyga: you have a macro layer here https://pagure.io/go-rpm-macros 17:38:40 zyga, and a code analysis engine here: https://pagure.io/golist/ 17:38:54 that is not what is in production in the moment 17:39:19 but it would be great to hear about it 17:39:31 zyga, and ideal clean specs here https://pagure.io/go-rpm-macros/blob/master/f/templates/rpm 17:40:45 here is current state https://src.fedoraproject.org/rpms/dep/blob/master/f/dep.spec for some rather not too complex package 17:41:26 I fear the macro layer won't be too interesting for you or jcajka, it's a lot of looping over rpm variables to generare uninteresting rpm boilerplate 17:41:49 Thank you. I will read that later today 17:41:56 it's interesting to me as go packager because it means I do not have to write this tedious boilerplate, or check it for mistakes 17:43:23 we are well over the time, let's close the meeting 17:43:38 zyga, what would be nice is to reduce the mismatch between the go tool conventions and what rpm expects, so the macro adaptation layer can shrink over time 17:44:27 thanks everybody for coming zyga for talking about indigo and Pharaoh_Atem for chairing this meeting 17:45:13 #endmeeting