14:00:26 <andreasn> #startmeeting Weekly Cockpit meeting 2017-02-13
14:00:26 <zodbot> Meeting started Mon Feb 13 14:00:26 2017 UTC.  The chair is andreasn. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:00:26 <zodbot> The meeting name has been set to 'weekly_cockpit_meeting_2017-02-13'
14:00:32 <andreasn> .hello andreasn
14:00:33 <zodbot> andreasn: andreasn 'Andreas Nilsson' <anilsson@redhat.com>
14:01:04 <garrett_> .hello garrett
14:01:05 <zodbot> garrett_: garrett 'Garrett LeSage' <garrett@lesage.us>
14:01:15 <dperpeet> .hello dperpeet
14:01:16 <zodbot> dperpeet: dperpeet 'None' <dperpeet@redhat.com>
14:01:45 <mvollmer> .hello mvo
14:01:46 <zodbot> mvollmer: mvo 'Marius Vollmer' <marius.vollmer@gmail.com>
14:02:17 <andreasn> #topic Agenda
14:02:22 <github> [cockpit] mvollmer opened pull request #5884: WIP - plugins playground (master...base1-plugin-api) https://git.io/vDozx
14:02:26 <mvollmer> * package plugins
14:02:42 <mvollmer> * translations
14:03:49 <andreasn> anything else?
14:04:58 <andreasn> #topic Package Plugins
14:05:53 <mvollmer> alright
14:06:09 <mvollmer> so I started working on the "resource globbing" feature
14:06:38 <mvollmer> where when you load "./foo.*.js" you magically get all files matching the glob concatenated
14:07:07 <mvollmer> but I decided to explore a different approach first
14:07:33 <pitti> I don't have context for that, but is that supposed to be an alternative to webpack for third-party plugins?
14:07:38 <mvollmer> stefw is not here, which is a bit bad timing since he will probably set me straight immediately
14:07:40 <mvollmer> anyway
14:08:04 <mvollmer> pitti, in a sense
14:08:19 <mvollmer> first user is the machines package
14:08:26 <mvollmer> which would like to have pluggable providers
14:08:39 <mvollmer> one for libvirtd, one for something else
14:09:02 <mvollmer> so I made this: https://github.com/cockpit-project/cockpit/pull/5884
14:09:04 <github> [cockpit] dperpeet opened pull request #5886: WIP kdump: Enable testing on continuous atomic (master...kdump-continuous) https://git.io/vDo2f
14:09:14 <mvollmer> which is a completely different approach to loading plugins
14:09:32 <mvollmer> a package can declare in its manifest that it has a plugin for some other package
14:09:42 <mvollmer> and that other package can find its plugin via the manifests
14:09:49 <mvollmer> i think this is pretty clean
14:09:55 <dperpeet> mvollmer, what is the load order?
14:10:13 <mvollmer> heh, too early for details like that! :-)
14:10:23 <dperpeet> I think there are use cases for bot
14:10:24 <dperpeet> both
14:10:25 <mvollmer> order shouldn't matter for plugins, no?
14:10:27 <dperpeet> that's why I'm asking
14:10:35 <dperpeet> well
14:10:44 <dperpeet> a provider might need to be available first
14:10:52 <mvollmer> can be solved
14:11:04 <mvollmer> loading should be doable in any order, but initialization can be ordered
14:11:07 <dperpeet> while in other cases the "main" plugin may choose only to load others, selectively
14:11:21 <pitti> "first" is not well-defined for these usually -- what if there are two plugins which want to be "first"
14:11:28 <dperpeet> it sounds interesting
14:11:30 <mvollmer> impossible is nothing
14:12:02 <dperpeet> and how does the plugin become aware of others that are available for it?
14:12:03 <mvollmer> so, the fundamental topic here is: globbing or manifests?
14:12:24 <dperpeet> I mean how does the package become aware of its plugins
14:12:31 <mvollmer> with globbing, the bridge does the hard work and caching csp, etc should just work
14:13:07 <mvollmer> with manifests, no changes required to the bridge, but more there might be issues with csp
14:13:48 <mvollmer> dperpeet, either by listing all files that match a glob pattern,
14:13:54 <mvollmer> or by scanning all manifests
14:14:14 <mvollmer> i like the manifests also because that's what the shell is using
14:14:42 <mvollmer> to let packages plug themselves into the navigation
14:14:46 <dperpeet> the manifests seem cleaner, if they cover all the use cases
14:14:57 <mvollmer> plugging a provider into "machines" is kinda the same, no?
14:14:59 <dperpeet> do we have a "stories" page for this?
14:15:11 <dperpeet> it might be valuable to design this as a cockpit feature
14:15:19 <mvollmer> yeah
14:15:21 <dperpeet> to make sure we cover all the use cases
14:15:33 <mvollmer> it's possible to implement it without any changes to cockpit.js or the bridge
14:15:41 <mvollmer> but a common 'framework' would be nice
14:16:13 <mvollmer> i need to talk with mlibra of course
14:16:34 <dperpeet> if we can make the manifest idea work properly with CSP and caching, then I'm for that
14:16:43 <dperpeet> less magical and more aligned with what we've been doing already
14:16:58 <mvollmer> right now I need "script-src 'unsafe-eval'"
14:17:06 <mvollmer> which sucks of course
14:17:12 <dperpeet> it also allows dropping in plugins without changing the package that benefits
14:17:30 <dperpeet> if we can't make it work with CSP, then we can't use it
14:17:32 <mvollmer> well, that is also true with globbing
14:17:51 <dperpeet> globbing needs to have a proper wildcard though, right?
14:17:52 <mvollmer> dperpeet, you mean, make it work without 'unsafe-eval'?
14:18:07 <dperpeet> mvollmer, without any CSP exceptions I'd say
14:18:31 <mvollmer> yeah
14:18:58 <mvollmer> I'll try to learn more about that
14:19:41 <dperpeet> but like I said, if we can make it work, manifests seem cleaner
14:20:37 <mvollmer> okay
14:21:03 <andreasn> next topic?
14:21:15 <mvollmer> yeah
14:21:31 <andreasn> #topic translations
14:21:43 <mvollmer> this was the other user for resource globbing
14:21:55 <mvollmer> but I need help with motivating this again
14:22:22 <mvollmer> the idea was that we load "po/po.*.js" in the shell
14:22:47 <mvollmer> and the bridge then combines all translations of all packages into one big response
14:23:07 <mvollmer> i have to admit I am not clear why we want to do that.
14:23:12 <dperpeet> combine the advantages of per package translations with the caching behavior of a single file
14:23:15 <dperpeet> I think
14:23:30 <mvollmer> makes sense
14:23:34 <pitti> you mean a glob over module_dir/*/po/po.<browserlang>.js ?
14:23:45 <dperpeet> we have a lot of packages, potentially
14:23:52 <mvollmer> no, just a single dir
14:23:53 <pitti> AFAICS we want to concat the translations of all packages, but certainly not of all languages
14:23:56 <dperpeet> which makes it very costly to pull in a lot of stuff
14:24:09 <mvollmer> /usr/share/cockpit/po/po.*.js
14:24:12 <dperpeet> right, one language I'd say
14:24:18 <pitti> po.*.js sounds like "all the langs"
14:24:25 <mvollmer> yes, for one language only
14:24:27 <pitti> but the request string should tell us which lang the browser wants?
14:24:42 <mvollmer> yes, it does, this is all working already
14:24:47 <pitti> ack
14:24:52 <mvollmer> if you ask for shell/po.js
14:25:01 <mvollmer> you get shell/po.de.js.min.gz
14:25:03 <mvollmer> say
14:25:45 <mvollmer> but we want to get the concatenation of po.base.de.min.js.gz and po.shell.de.min.js.gz, etc
14:26:12 <mvollmer> so there is the first problem: concatenating gzip files doesn't just work, unfortunately.
14:26:44 <mvollmer> but we can get around that by excellent caching, I guess
14:27:12 <mvollmer> where the bridge recompresses this only once
14:27:24 <mvollmer> after any change to the packages
14:27:45 <pitti> it doesn't? "cat ./dist/shell/po.de.js.gz ./dist/shell/po.fr.js.gz | gzip -cd|less"  looks reasonable here
14:28:02 <mvollmer> yeah, stefw says that browsers don't do the right thing
14:28:24 <mvollmer> they only take the first member
14:28:26 <pitti> oh, ok, yes; probably serves Content-Length: from the first .gz only or so
14:29:03 <mvollmer> no, the browser probably just stops after uncompressing the first member and ignored the rest
14:29:20 <mvollmer> anyway, this can be solved
14:30:41 <mvollmer> so, about the reason for doing this.
14:31:13 <mvollmer> an alternative would be to pack the translations into the webpack thing, no?
14:31:40 <mvollmer> no, it wouldn't
14:31:47 <pitti> I thought this was for 3rd party plugins?
14:31:51 <mvollmer> since we would need to include all languages
14:32:43 <mvollmer> for 3rd party packages, yes
14:33:15 <mvollmer> after you install the 3rd party package, all cockpit frames would start loading its translations
14:33:37 <mvollmer> the shell actually needs them for translating strings from the manifest
14:34:20 <mvollmer> but I have an idea for that also
14:36:26 <mvollmer> so, hmm, I don't really want to add resource globbing to the bridge, as you can tell
14:36:43 <dperpeet> is there a reason for that?
14:37:14 <mvollmer> merging files seems to be tricky
14:37:22 <mvollmer> with gzip etc
14:38:24 <mvollmer> but if performance is the motivation, then we absolutely need to concatenate and zip
14:38:29 <petervo> the problem we are trying to solve is when the package versions are mismatched
14:39:02 <petervo> and we need to make sure each package as it's proper translation
14:39:36 <petervo> so the idea was the split up the language files and use the glob to load whatever is there
14:39:43 <mvollmer> yeah, this is another problem: can translations be merged safely in the first place?
14:39:44 <petervo> with one request
14:40:14 <petervo> i think so
14:41:34 <pitti> mvollmer: there's msgmerge for that, which will deal with the formatting and weeding out dupes
14:41:52 <mvollmer> but we don't want to run that in the bridge at run-time, no?
14:42:15 <petervo> no we don't but i don't think that's necessary
14:42:25 <petervo> we've already done all the formatting
14:42:38 <petervo> and these are just json files
14:43:01 <petervo> or rather js/json
14:43:23 <mvollmer> yeah, but if a new package brings its own translation of "Dashboard", will that change the translation in the shell navigation also?
14:44:42 <petervo> it's not like each package has it's own translation, it's just that we split things up when we build/package
14:44:56 <petervo> and then reassemble at run time
14:45:21 <mvollmer> so this all happens in a single source tree, our own?
14:45:26 <petervo> but in theory yes it could, but that's we are only doing these for our packages
14:45:56 <mvollmer> ahh, i see
14:46:06 <mvollmer> how would we split the translations?
14:46:10 <petervo> yes, other packages would need a different way to load their translations if they have them
14:46:18 <mvollmer> aha!
14:46:38 <petervo> that was part of my initial concern with the glob stuff at brno
14:46:44 <mvollmer> so it's really only about version mismatch between our own packages?
14:46:47 <petervo> right
14:47:00 <mvollmer> what was your concern?
14:47:19 <mvollmer> that it doesn't extend cleanly to 3rd parties?
14:47:24 <petervo> yes
14:47:38 <petervo> but really we don't need it to
14:47:55 <petervo> chances are they'll have their own fancy i18n loader anyways
14:47:56 <mvollmer> alright
14:48:04 <petervo> and they don't need all our translations
14:48:30 <mvollmer> so the problem of how shell translates strings from the manifests is still open?
14:48:51 <mvollmer> petervo, right now all translations are in cockpit/po/, right?
14:49:01 <petervo> right
14:49:09 <mvollmer> and we want to split them out into cockpit/pkg/networkmanager, ...?
14:49:23 <mvollmer> how? manually?
14:50:06 <petervo> we should have documented this better, but my thought was we would still all be in there
14:50:09 <pitti> a *.pot and the *.po files know where strings came from, so you can sort this automatically
14:50:18 <petervo> just different files per package
14:50:23 <petervo> and lang
14:50:34 <petervo> yes, stef had a command for that
14:50:46 <petervo> not sure if he merged it yet or not
14:51:15 <mvollmer> right, the fog is clearing... :-)
14:51:17 <pitti> i. e. create a *.pot per plugin (which should be automatic), and run msgmerge on each of the *.pots with the current "everything" *.po, this will pick what it needs
14:51:48 <mvollmer> and the *.pot is the output from scanning the sources?
14:52:08 <pitti> right, from xgettext normally (or intltool-extract, or whatever the equivalent is in the js world)
14:52:18 <mvollmer> right
14:52:24 <pitti> i. e. teh thing that scans sources for _("") things
14:52:40 <mvollmer> yeah
14:53:02 <petervo> really all we care about being split up with is the json
14:53:10 <petervo> so the output of po2json
14:55:43 <andreasn> so there is only 5 minutes left of this meeting
14:55:49 <mvollmer> okay, I'll try to write this all down in the trello card
14:55:59 <andreasn> excellent
14:56:01 <andreasn> eot?
14:56:37 <mvollmer> current conclusion for me: we really want resource globbing, and I'll use it for the plugins also.
14:57:27 <andreasn> #topic Open Floor
14:57:43 <dperpeet> I just wanted to note that the current Outreachy round will end in roughly 3 weeks
14:58:12 <dperpeet> bhakti is making good progress and will do some usability testing soon
14:58:34 <dperpeet> or ask some questions about usability of the designs for firewall configuration, anyway
15:00:32 <andreasn> sounds good
15:00:35 <andreasn> #endmeeting