16:04:51 <nim> #startmeeting golang_sig
16:04:51 <zodbot> Meeting started Wed Feb 27 16:04:51 2019 UTC.
16:04:51 <zodbot> This meeting is logged and archived in a public location.
16:04:51 <zodbot> The chair is nim. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:51 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:04:51 <zodbot> The meeting name has been set to 'golang_sig'
16:05:21 <nim> #topic Roll Call
16:05:27 <nim> .hello2 nim
16:05:28 <zodbot> nim: nim 'None' <nicolas.mailhot@laposte.net>
16:06:02 <eclipseo> .hello2
16:06:03 <zodbot> eclipseo: eclipseo 'Robert-André Mauchin' <zebob.m@gmail.com>
16:06:07 <Son_Goku> .hello2 ngompa
16:06:08 <zodbot> Son_Goku: Sorry, but you don't exist
16:06:14 <Son_Goku> .hello ngompa
16:06:15 <zodbot> Son_Goku: ngompa 'Neal Gompa' <ngompa13@gmail.com>
16:06:35 <nim> QuLogic, deparker: want to joing the golang SIG meeting?
16:06:40 <nim> join?
16:07:21 <nim> now we wait till 17h10 I think to give people time to arrive
16:07:32 <nim> eclipseo, nice to see you with us again
16:08:00 <eclipseo> yeah sorry for last month I had a little break
16:08:34 <nim> eclipseo, we all missed for one reason or another all the meetings since november I think :(
16:08:39 <eclipseo> I've been bringing myself up to speed with reviews and now I'm back into circular Go deps again
16:09:13 <nim> yeah we all see you are alive :)
16:10:02 <Son_Goku> heh
16:10:26 <Son_Goku> this month has been an "everything is on fire" month for me
16:11:10 <nim> end of Christmas break for everyone I fear
16:11:30 <Son_Goku> well, hardware failure of one of our servers is not a fun situation to recover from
16:11:47 <Son_Goku> err $DAYJOB's servers
16:12:11 <nim> I'd love to have one of those, I'm stuck in the provisionning phase at $DAYJOB
16:13:34 <nim> anyway, time is up, let's go
16:14:15 <nim> #topic where do we want to go now (aka https://pagure.io/GoSIG/go-sig/issue/20)
16:14:55 <nim> that's the only issue marked as meeting in pagure
16:15:00 <nim> it's getting stale
16:15:28 <Son_Goku> we also haven't had a meeting in a month ;)
16:16:18 <nim> I think we've all seen by now that neither jchaloupka nor jcajka were keen on involving themseves at this stage
16:16:21 <eclipseo> at what part of the 11 points ane we?
16:16:45 <Son_Goku> well, I'm pretty sure that jchaloupka is basically dead to the world right now
16:16:52 <Son_Goku> I haven't seen hide or hair of him in months
16:17:28 <nim> right now the redhat-rpm-config stuff is done 99% of the way, someone need to convince the redhat-rpm-config maintainers to finish the little missing bit
16:18:15 <nim> that means we could move to 2 or even force resolution on 1 by moving to 2
16:19:05 <nim> QuLogic has been fixing golist
16:19:13 <nim> I've been fixing the macro packages
16:19:35 <nim> but I still have no idea how we could get them in the distro without support from jcajka
16:20:02 <nim> as they need to replace the current stuff to be useful
16:20:07 <nim> any idea here?
16:20:33 <nim> Son_Goku, you move more in distro cricles than us, how can we get there?
16:20:40 <nim> circles
16:20:54 <eclipseo> But when this get implemented, do we need to rewrite all our SPECs or is this packaward compatible?
16:21:18 <nim> this is intended to be backwards compatible
16:21:29 <Son_Goku> if it's backwards compatible, jcajka will help us
16:21:29 <nim> and both QuLogic and my tests are green
16:21:40 <Son_Goku> and I can talk to him about making that happen
16:21:51 <Son_Goku> he's been very leery after the last few adventures
16:21:57 <nim> but the proof is in the pudding
16:22:18 <eclipseo> if everything is double checked, I think he will agree
16:22:18 <Son_Goku> as for the other distros, I can reach out to my counterparts in openSUSE and get a dialogue going
16:22:42 <Son_Goku> it'll be trivial to get Mageia to go along, since Fedora and Mageia are pretty close as-is
16:22:54 <Son_Goku> someone should reach out to Akien, who is both Fedora and Mageia
16:22:59 <Son_Goku> (and not me ;) )
16:23:53 <nim> eclipseo, nothing is perfectly checked unfortunately, you and others churn out packages too fast for anyone to retest all the time
16:25:02 <nim> Son_Goku, last time the only thing that broke is an alias that jchaloupka left in despite redhat-rpm-macros maintainer to nuke it :(
16:25:27 <nim> Son_Goku, so are you ready to do it?
16:25:31 <eclipseo> Currently I'm checking rclone deps, for which upstream si moving fast in term of releases. So in any case, I'll revisit them when the new macros are in place. The only issue is that a lot of this is circular dependencies hell.
16:25:48 <nim> Son_Goku, what do you want from QuLogic or me to check things works before the next step?
16:26:12 <Son_Goku> I think at this point we just need jcajka's signoff
16:26:39 <nim> Son_Goku, yes he and jchaloupka own the packages that need changing/retiting
16:26:42 <nim> retiring
16:26:45 <eclipseo> Would be great to have them all implemented, as I don't really wamt to continue bushing updates that will be obsoletes once the new macros are in
16:27:15 <Son_Goku> and jcajka's signoff is important to get this stuff working for EL8 too
16:27:19 <Son_Goku> as he's our connection to the inside
16:27:55 <nim> eclipseo, they won't be obsolete, they just can not use the enhancements that make spec wwriting easier
16:28:34 <nim> but anyway another reason we need this done is that Go modules will be on by default in Go 1.13
16:28:46 <nim> and that requires scary tooling reworking
16:29:11 <nim> and you can't really rework what's in Fedora right now
16:29:47 <eclipseo> We need jchaloup for: https://pagure.io/golist/issue/1
16:30:36 <nim> eclipseo, QuLogic has fixed it in a PR
16:30:49 <nim> https://pagure.io/golist/pull-request/16
16:31:10 <nim> eclipseo, that's why we need this code in a clean package that QuLogic can update and maintain
16:31:12 <eclipseo> yes that what I had in mind too when I investigated the bug
16:31:17 <Son_Goku> okay, I gotta go to work now
16:31:21 <eclipseo> this should be committed
16:31:22 <Son_Goku> I'll be back in ~15 minutes
16:31:34 <nim> Son_Goku, ok see you in 15min
16:32:19 <nim> eclipseo, we're all in a 4th dimension land where the previous tooling maintainers have gone to greener pastures and the new ones are not official yet
16:32:58 <nim> anyway *if* Son_Goku can  make jcajka and rehat-rpm-config maints to move
16:33:25 <nim> it'll all just be a mass rebuild, probably in a side branch, to check what works or not
16:33:25 <Son_Goku> I got the file triggers for ldconfig to happen
16:33:31 <Son_Goku> if I can do that, I can do almost anything
16:34:02 <nim> Son_Goku, the remaining stuff is simple and trivial, that's why they are bikeshedding it to death and not merging
16:34:15 <Son_Goku> bbs
16:34:16 <eclipseo> on what branch will the new macros be active? From F29 to F31 or only F31?
16:34:45 <nim> eclipseo, probably, just devel till the first mass rebuild is ok
16:34:54 <nim> eclipseo, and then pushed to stable
16:35:25 <nim> eclipseo, I don't think it's wise to push to stable before we've done at least one serious rebuild in devel
16:36:32 <eclipseo> ok
16:36:35 <nim> eclipseo, and don't think anything will break, and if something breaks it will be a corner case easy to fix, but that kind of initial bootstrap fixing is easier in devel
16:36:58 <nim> now the scarry stuff
16:37:00 <nim> modules
16:37:30 <eclipseo> i still haven't understood them so I'm not much help here
16:37:38 <nim> I'll explain
16:37:51 <nim> for you and others that may read the meeting CR
16:37:59 <eclipseo> I'm not sure how it will benefit Go packages except for thing like Kubernetes
16:38:11 <nim> basically, in just a few months, Google will release Go 1.13
16:38:22 <nim> at that point modules will be on by default
16:39:00 <eclipseo> what will it change for us?
16:39:07 <nim> that means nothing we have in the distro will build anymore unless we manage to create module packages in the meanwhile
16:39:31 <nim> or that we diverge from upstream and force non-module mode Fedora-side
16:40:14 <nim> because upstream decided modules and traditional builds can not cohexist and do not use the same files on disk
16:40:49 <eclipseo> how can we handle it then?
16:41:02 <nim> so we need to decide, probably by next meet, if we want to go the diverge-from-upstream route or get module-ready
16:41:31 <eclipseo> what would be "module-ready" entail?
16:41:47 <eclipseo> it seems to me that it's yet aother way to bundle
16:41:49 <nim> module-ready would mean:
16:42:42 <nim> generate golang-foo-module or golang-foo-module-devel packages containing the module files
16:43:55 <nim> generate the deps in rpm for those module files
16:44:12 <nim> index thos module files when installed on disk
16:44:40 <nim> handle all the cases where the module limit is not the same as our current devel packages
16:45:00 <eclipseo> the issue i see if we diverge from upstream is tat at one point it will not be supported anymore and we will have issue while reporteng bugs to upstream bicause we don't use the official build method
16:45:34 <nim> for example upstream decided testing code was an integral part of modules, module deps include the depos needed by testing code, you can not ifdef it
16:46:15 <eclipseo> this is problematic for circilar deps issues
16:47:02 <nim> eclipseo,  I agree we have to go the module route sonner than later unless we want to be keft on the side by upstreams
16:48:02 <nim> but anyway, forgetting circular deps, because those are things upstreams are supposed to take care of
16:48:29 <nim> we have the problem that Google is pushing hard for modules but its tooling is very immature
16:48:47 <nim> and basically only handles the case where go gets everything from the internet
16:49:12 <nim> so we need to write our own code to create the module files
16:49:49 <nim> we need to write our own code to index the module files
16:50:14 <nim> and we need to convert module metadata to rpm metadata
16:51:13 <nim> I think I know how to do the conversion thing
16:51:23 <nim> for simple clean module requires
16:51:56 <nim> so it will work for 90% of the projects and fail badly for the 10% that try clever weird things
16:52:25 <nim> creating the module files is just copying the project files in the correct place and zipping it
16:53:37 <nim> but since upstream does not provide an authoritative way to do this, that means we'll have to rely on golist to identify project files
16:53:56 <eclipseo> modules as defined in go.mod files always reference a specific version, while what we package are often the latest version only, how will we handle this?
16:54:36 <nim> that means we can not forget https://pagure.io/golist/issue/2 and we really need someone to fix it
16:55:27 <nim> eclipseo, as far as i understand things one of the main point of modules was to define the upgrade logic
16:55:41 <nim> eclipseo,  so for all projects that use semver
16:56:58 <nim> eclipseo, the go compiler should accept foo a.b.c instead of foo x.y.z as long as a=x and b.c >= y.z
16:57:11 <nim> that's the easy nice thing in modules
16:57:38 <nim> eclipseo, otherwise we just do compat packages
16:58:10 <nim> eclipseo, another nice thing of modules is that compat packages should ont clash on-disk as in GOPATH mode
16:58:20 <deparker> nim, sorry I saw this kind of late, I'm on pacific time... anyways I assume the go sig meeting is over but I'd very much like to join in the future... is there someone who could ping me with a calendar invite or something?
16:58:40 <nim> deparker, it's still on and you're welcome
16:58:50 <nim> deparker, we were discussing modules
16:58:56 <deparker> nim, I have back to back meetings this morning right now unfortunately :(
16:59:24 <deparker> could I be invited to future meetings so I can make sure I'm available?
16:59:51 <nim> deparker, the meeting bot sends and invite one week before to the golang mainling list
17:00:09 <deparker> nim, ack
17:00:11 <nim> deparker, just read the ML, it's very low-traffic
17:00:34 <nim> deparker, we count on you!
17:00:57 <nim> anyway, to get back to modules
17:01:13 <nim> mapping deps seems doable easy peasy
17:01:33 <nim> generating module files needs someone to finally fix https://pagure.io/golist/issue/2
17:01:50 <nim> and indexing module files: I have no idea
17:02:25 <nim> I've not been able to locate the index spec so far, not the code that generates it on the Google servers
17:02:55 <nim> eclipseo, so that's all the things that need to be done
17:03:11 <nim> eclipseo, just before we get to the real-world testing
17:03:29 <nim> and see all the ways upstreams manage to break our assumptions
17:03:46 <eclipseo> I don't see us being 1.13 ready when it's published
17:03:55 <nim> that's why at this stage I'm pretty pessimistic on us being ready
17:04:04 <eclipseo> I can't help much with it as I'm not a programmer
17:04:23 <nim> eclipseo, NP, I'm not really a programmer either
17:04:40 <nim> I just do it because it needs to be done by someone
17:06:31 <nim> and, some upstreams won't be ready by 1.13 time I think, so the real deadline is 1.14 (even though it sucks to diverge from Golang upstream for one version)
17:07:04 <nim> but we won't be ready for 1.14 either without packaging prototypes for 1.13
17:07:36 <nim> which means getting all the stuff in https://pagure.io/GoSIG/go-sig/issue/20 out of  the way first
17:07:58 <nim> was I more or less clear or do you have other questions?
17:09:01 <nim> eclipseo, I was all assuming it would be way easier before discovering upstream had made an hard break between modules and non-modules
17:09:21 <nim> eclipseo, and not provided anything for people not living in go get land
17:10:15 <eclipseo> no that was clear, I just don't see how everything will fit within our build system in the end
17:10:41 <nim> eclipseo, that means I wasn't clear :)
17:11:06 <nim> eclipseo, basically, in a target module world
17:11:50 <nim> eclipseo, all the current devel packages are useless and probably gone
17:12:15 <eclipseo> That I did understand
17:12:37 <nim> eclipseo, and you have module/module-devel packages that do the same thing, and contain module files (zip files)
17:12:59 <nim> eclipseo, and you can not mix modules and non-modules in a build
17:13:35 <eclipseo> but some old package will never move to the module thing
17:14:04 <eclipseo> so does it mean we will need to keep the old system and the new system together?
17:14:14 <eclipseo> the overhead would be crazy
17:14:46 <nim> eclipseo, the first step is being able to generate module packages
17:15:22 <nim> eclipseo, I can automate easily the generation of the mirror old-style devel package from the module package if necessary
17:16:00 <nim> and then we decide when we turn old-strype generation off because it's just wasted repo space
17:16:29 <eclipseo> what are other distros doing? are we the only ones unbundling everything?
17:16:39 <nim> no idea
17:17:12 <nim> IMHO everyone is at the "drat, 1.12 was released, modules will be on in 1.13, how do we do it now?"
17:17:55 <nim> ie modules cease to be just a nice theoritical idea, and people realise Google didn't ptovide the needed tooling for distros
17:18:45 <nim> module were supposed to remove vendor and force evryone to unbundle
17:18:46 <eclipseo> maybe we shoild run experiments with 1.12 with GO111MODULE=on
17:19:07 <nim> unfortunately you can't
17:19:22 <nim> before the point where you generate module packages
17:19:42 <nim> because if you don't generate your own module packages
17:20:11 <nim> GO111MODULE=on will just cause the go compiler to download stuff from the internet
17:20:50 <nim> and sure it will work (as in "it builds") but you'll have no idea where the module was generated and how
17:21:16 <nim> that's the "upstream decided to do a hard break" part
17:21:51 <nim> you can't take a codebase, with a working GOPATH, and convert one of the deps manually to cjeck how it goes
17:22:16 <nim> you need to put everything in its own module to start checking
17:22:33 <nim> or download blyindly pre-made modules from the internet
17:23:03 <nim> or vendor everything (which will work with GO111MODULE=on, except you won't be using module mode at all)
17:24:09 <nim> that's why fist step is to start generating modules, forgetting about all the tricky stuff like circular deps
17:24:40 <eclipseo> there is this also https://blog.golang.org/modules2019#TOC_5., which would need to be bypassed I guess
17:24:48 <nim> and once you have modules working in the simple case, iterate to see just how far you can go with the simple case
17:24:59 <nim> and what patterns require packager workarounds
17:26:05 <nim> eclipseo, we'll just nuke the sum file I think
17:27:07 <nim> at first at least
17:28:04 <nim> will probably have to because otherwise it's not possible to patch anything at the distro level
17:28:57 <eclipseo> So now we will need to rebuild our current packages with new macros, but then in August we will need to reinvent everything for Go 1.13
17:29:03 <eclipseo> great :(
17:29:05 <nim> and we will need to patch because I don't think upstreams will start magically releasing perfect code that never needs patching
17:29:42 <nim> eclipseo, progress!
17:30:10 <nim> eclipseo, more seriously, the new macros are just incremental enhancements
17:30:43 <nim> eclipseo, the module thing is a lot more scary, because it adds constrains on upstreams and us
17:31:03 <nim> and sure the end result will be better, but
17:31:30 <nim> I'm pretty sure that any spec, that does heavy surgery to workaround problems
17:31:40 <nim> won't survive modules without deep changes
17:32:11 <nim> that's why the new macros will help us a lot here
17:32:27 <nim> by helping to remove all the boilerplate
17:32:53 <nim> so it will be clearer if a spec does tricky things that will need a modulectomy
17:34:42 <nim> basically, once we get our tooling to work
17:35:07 <nim> it will be trivial to generate module, or devel, or module+devel from clean upstreams
17:35:23 <eclipseo> ok, I hope so
17:35:32 <nim> and unclean upstreams will have to either move to modules, and clean their act in moving, or die
17:36:30 <nim> but, there will probably at least six months of heavy churn and dieout
17:37:17 <nim> and what's unclear at this point, is how many upstreams will clean up their act
17:37:38 <nim> and how many will find ways to continue doing garbage in module mode
17:37:55 <nim> just different garbage that will require different workarounds on our part
17:38:37 <nim> just for reference, the upstream ticket on providing some module tooling https://github.com/golang/go/issues/28835
17:39:23 <nim> I'm not excessively optimistic, it's targetted at 1.13 only, and not sure if they will do all the things we need or just part of them
17:39:50 <nim> that's why I think it's unreasonable to rely on this and we need to get our own tooling to work
17:42:09 <nim> except for the $GOPROXY/mymodule/@v/list part I think we could implement all the rest with not too much effort
17:42:20 <nim> and then it's just mass packaging and testing
17:44:45 <nim> eclipseo, except for the BuildRequires part
17:45:04 <nim> I'm pretty sure a spec conforming to the template here: https://pagure.io/go-rpm-macros/blob/master/f/templates/rpm/spectemplate-go-0-source-minimal.spec
17:45:35 <nim> could generate indiferently devel and modules packages with no packager change in the spec
17:46:12 <nim> that's the nice part of abstarcting everything in macros
17:46:29 <eclipseo> yeah i was reading the template earlier, seems straightforward now
17:47:02 <nim> the only reason you would need manual change is that as long as https://github.com/rpm-software-management/rpm/issues/104 is not done
17:47:29 <nim> if modules generate golang-mod() provides and devel packages generate golang() packages
17:47:57 <nim> you need to switch manually BR from one to the other since they are not autogenerated
17:49:12 <nim> I don't hink repurposing golang() provides in module mode is doable unless we dump all existing devel packages in one go and restart everything from scratch
17:50:11 <eclipseo> it seems it's on the right path, generally ignatenko things go through
17:52:15 <nim> everything is moving in the right direction Go side and rpm side, it's the Go SIG in the middle which needs to adapt
17:52:20 <nim> that's painful :(
17:53:13 <nim> thanks to you and others the golang packageset is a lot better shape than it was a year ago
17:53:37 <nim> otherwise I'de have just said dump everything and restart from scratch for modules
17:54:17 <eclipseo> I'm rot sure I will put as much work upgrading the current packages, I'd like to wait for the new macros to settle in
17:55:49 <nim> the point is, that with your work, we don't need to upgrade the current packages
17:56:48 <nim> just put the new fixed golist/macro code in the distro (that's for Pharaoh_Atem, QuLogic and me), wheck your specs still build fine
17:57:03 <nim> and then just do new specs in new improved mode
17:57:29 <nim> and start working on modules so those new specs generate modules by default
17:58:00 <nim> and then once we get to the point that generating modules should just work for new-style specs
17:58:18 <nim> yes there will be a need for a mass cleanup of your specs
17:58:27 <eclipseo> ok
17:58:48 <nim> hopefully, they should be more regular than the previous ones
17:58:50 <eclipseo> I got to go now
17:59:01 <nim> so the mass cleanup could be done with scripting
17:59:24 <nim> eclipseo, that's the end of the hour anyway. Pharaoh_Atem didn't come back :(
17:59:38 <eclipseo> yeah :(
17:59:44 <eclipseo> maybe erdeeting then
17:59:45 <nim> eclipseo, do you want to add something to the meeting log?
17:59:52 <eclipseo> endmeeting i mean
18:00:21 <nim> eclipseo, otherwirse I'll just record he should work with jcajka to get the new packages in
18:00:22 <eclipseo> the oly action is for Pharaoh_Atem to push jcajka in the right direction
18:00:35 <eclipseo> yeah
18:01:11 <eclipseo> i'm going, see you later
18:01:54 <nim> #agreed Pharaoh_Atem should work with jcajka and redhat-rpm-config maintainers so the new golist and macro packages can be used in the distro
18:02:23 <nim> #chair nim
18:02:23 <zodbot> Current chairs: nim
18:02:31 <nim> #agreed Pharaoh_Atem should work with jcajka and redhat-rpm-config maintainers so the new golist and macro packages can be used in the distro
18:03:12 <nim> #info SIG members should think about where they want to go with Go modules for Go 1.13
18:03:25 <nim> eclipseo, thanks a lot for your time!
18:03:37 <nim> #endmeeting