16:01:57 <stefw> #startmeeting
16:01:57 <zodbot> Meeting started Mon Jan  5 16:01:57 2015 UTC.  The chair is stefw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:57 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:02:05 <stefw> Any agenda?
16:02:28 <dominikp> work direction, focus
16:02:40 * stefw won't start discussing the multi-server features, until andreasn and others are here
16:02:46 <mvollmer> jQuery amend
16:02:47 <stefw> * Testing update
16:02:51 <mvollmer> metrics channel
16:03:41 <mvollmer> my two topics should be short, so we could get them done with first, maybe.
16:03:50 <stefw> #topic jQuery amend
16:04:23 <stefw> mvollmer, you're on :)
16:04:35 <mvollmer> so, yeah, I started having the same doubts as stefw...
16:04:45 <mvollmer> why do we do it?
16:05:03 <stefw> a little backgound might be in order?
16:05:10 <mvollmer> yes.
16:05:36 <dominikp> I have no idea what this is about
16:05:44 <mvollmer> in many places the DOM is reconstructed from scratch whenever something in the data changes.
16:06:00 <dominikp> ah, that jquery feature (got it)
16:06:15 <mvollmer> we will be using mustache more and more to create those DOM pieces
16:07:01 <mvollmer> now i was thinking that it is important to update the DOM incrementally, with small changes instead of replacing a large chunk with a mostly identical large chunk.
16:07:40 <mvollmer> but browser are doing a good job with replacing large chunks.
16:07:46 <stefw> there's a vast array of possibilities in between extremely fine grained updates and replacing large chunks
16:08:12 <stefw> the use case i was trying to solve with the amend work was simply keeping the interactive state of rows when nothing significant has changed
16:08:25 <mvollmer> and in any case, our code should still be correct even if large chunks get replaced.
16:08:30 <stefw> rows, or dom elements, or links, etc.
16:08:42 <stefw> the code is still correct, but the UI behavior has usability issues
16:08:43 <mvollmer> open menus
16:09:01 <stefw> for example selecting data, scroll location, tab index, etc...
16:09:21 <stefw> i'm fine with throwing away this UI state when something changes for real (like a significant UI element is added/removed)
16:09:33 <stefw> but for a simple class add/remove or a number change (eg: cpu usage)
16:09:39 <stefw> we shouldn't be throwing away such UI state
16:09:47 <mvollmer> correct
16:09:57 <stefw> so that's why the jquery amend branch was very simple, because it targetted fixing very specific use cases
16:10:09 <stefw> it could get more complex later (eg: optional key based correlation)
16:10:20 <stefw> if further use cases prove to be necessary to fix
16:10:31 <dominikp> it's easier to maintain usability if only data changes, not the layout
16:10:36 <stefw> the nice thing is, as you mentioned mvollmer, the code pretty much is correct without this
16:10:47 <stefw> and this can be treated as an optimization
16:10:56 <stefw> a very nice to have optimization, but still an opmitization
16:11:29 <mvollmer> would be good to find code for it.
16:11:50 <mvollmer> ok, so summary: I am unsure how to proceed with it.
16:11:55 <stefw> it's not a priority
16:11:59 <stefw> i'm fine with completing it at a later
16:12:00 <stefw> time
16:12:09 <stefw> that's why i haven't pushed to complete it, other stuff is more important
16:12:22 <dominikp> what's the current state? it is a branch and not yet included?
16:12:26 <stefw> it's on a branch
16:12:29 <stefw> it mostly works, needs testing
16:12:33 <mvollmer> we can have the current version in master so that we actually can start using it.
16:12:46 <mvollmer> i have added some tests
16:12:49 <stefw> i also wonder if it should be a real plugin in a separate repo
16:12:54 <stefw> and we pull it in via /lib
16:13:14 <stefw> could still be in https://github.com/cockpit-project/jquery-amend
16:13:18 <stefw> ie: a sibling repo
16:13:42 <stefw> i wrote this code because it didn't appear to exist
16:13:48 <mvollmer> yeah
16:13:51 <stefw> wouldn't have been against pulling in another self-contained plugin
16:13:59 <stefw> everyone seems to have this code deeply intertwined with other logic
16:14:23 <stefw> and nobody offers it as a separate plugin ... probably because they need to fix corner cases ... in which case the code quickly gets into the weeds
16:14:31 <mvollmer> if we want proper incremental insert/remove in a sorted list, we pretty much have to implement "diff".
16:14:32 <dominikp> separate repo seems cleaner to me
16:14:50 <stefw> mvollmer, yeah, i'm fine with adding optional key support later
16:14:55 <mvollmer> yep
16:14:56 <stefw> but again, it's not a blocker
16:15:08 <stefw> and not a priority
16:15:10 <mvollmer> very interesting, though... :-)
16:15:38 <stefw> mvollmer, can you put the react link into the notes?
16:15:41 <stefw> so we can find it later?
16:15:52 <stefw> the one that describes the key based diff optimization?
16:15:53 <mvollmer> of this meeting?
16:15:55 <stefw> sure
16:16:25 <mvollmer> #info http://facebook.github.io/react/docs/reconciliation.html
16:16:31 <stefw> mvollmer, so what do you think we should do, bring it into cockpit now as is (with your tests)?
16:16:39 <stefw> or shelve this until later, since it's not priority?
16:16:42 <mvollmer> I would say so
16:16:45 <stefw> ok
16:17:05 <stefw> i can make a repo based on your branch and tests
16:17:09 <mvollmer> ok
16:17:20 <mvollmer> skip the key stuff, of course.
16:17:30 <stefw> #action stefw will put jquery-amend in a cockpit-project repo and bring it into cockpit including mvollmer's tests
16:17:59 <stefw> #info https://github.com/cockpit-project/cockpit/pull/1567
16:18:13 <stefw> #topic Metric Channels
16:18:24 <mvollmer> right
16:19:02 <mvollmer> I think https://github.com/cockpit-project/cockpit/pull/1609 is ready for review
16:19:17 <mvollmer> I did some potentially controversial things
16:19:18 <stefw> cool
16:19:26 <stefw> #info https://github.com/cockpit-project/cockpit/pull/1609
16:19:39 <mvollmer> like, "counter" metrics are differentiated in the bridge
16:20:03 <stefw> is that a PCP concept?
16:20:11 <mvollmer> "counter" semantics, yes.
16:20:34 <stefw> what are the potential issues with that?
16:20:35 <mvollmer> and it means that only the deltas are meaningful, not the absoulte value
16:21:01 <mvollmer> the client will not know the absolute value, in case it _does_ have a meaning.
16:21:21 <stefw> can the client open another metric channel with different semantics?
16:21:30 <mvollmer> no
16:21:31 <stefw> for the same metric?
16:21:47 <mvollmer> semantics are fixed per metric, it's not like units that you can convert.
16:22:00 <mvollmer> but we can add an option to not do the diffinf.
16:22:01 <stefw> is the word "semantics" from PCP?
16:22:05 <mvollmer> yes
16:22:48 <mvollmer> the main reason to do it is to handle 64bit counters.
16:23:00 <stefw> interesting
16:23:03 <stefw> well i think it's a valid use case
16:23:09 <mvollmer> the action is in the low bits, but we might lose them when converting to double.
16:23:26 <mvollmer> (there is a test case to prove that, even. :-)
16:24:01 <stefw> so the issue here for me is that there's no backwards-compatible way to tell cockpit-bridge not to diff the counter values
16:24:15 <stefw> well, besides having a new metric channel type
16:25:03 <stefw> but we can look at that further during review, i guess.
16:25:14 <mvollmer> yeah
16:25:31 <stefw> any other issues? i'm interested in what made you add omit_instances to the channel options?
16:25:32 <fche> hi guys
16:25:37 <fche> can I help with anything ?
16:25:50 <stefw> welcome to the cockpit meeting :)
16:25:58 <stefw> we're discussing merging the metric channel work that mvollmer has been doing
16:25:59 <mvollmer> stefw, to filter out "lo" explicitly, for example.
16:26:16 <stefw> mvollmer, interesting
16:26:23 <mvollmer> i don't think we have a concrete use right now.
16:26:54 <stefw> #action stefw will review the metric channel code
16:26:55 <mvollmer> so, "for completeness"
16:27:10 <mvollmer> fche, hi!
16:27:12 <stefw> fche, -> https://github.com/mvollmer/cockpit/blob/metrics-channel/tools/pcp.supp
16:27:57 <mvollmer> next up would be pcp archives for me.
16:28:04 <mvollmer> fche, i will need help with that.
16:28:12 <stefw> right, that's good, and we'll need andreasn's work on the design there
16:28:13 <fche> please ping any time
16:28:38 <stefw> ok, next topic?
16:28:41 <dominikp> wait
16:28:52 <dominikp> I was just reading the source code while you discussed this
16:29:25 <dominikp> is lack of numeric precision in the browser frontend an issue? is there a specific case?
16:29:42 <mvollmer> dominikp, yes.
16:30:18 <mvollmer> dominikp, Peformance Copilot can report 64 bit counter metrics, like bytes received on a network interface
16:30:43 <mvollmer> but the normal way to parse JSON will give you a double.
16:30:45 <dominikp> yes, but are there cases where we need the numeric values in the frontend, not just the string value
16:30:56 <stefw> we do get numeric values
16:31:04 <stefw> as JSON numbers
16:31:31 <stefw> here's an example of what they end up looking like when transferred:
16:31:31 <stefw> https://github.com/cockpit-project/cockpit/pull/1609/files#diff-81551d4c3e08fb4033c1c32084ca4965R792
16:31:43 <mvollmer> to display network I/O we would need to compute the difference between two numbers.
16:32:12 <mvollmer> but the low bits are lost if the counter goes beyong 53 bits.
16:32:34 <stefw> mvollmer, is that on the browser side?
16:32:40 <mvollmer> yes
16:32:49 <stefw> interesting
16:32:50 <mvollmer> well, actually, not sure.
16:33:05 <stefw> well on the bridge side we're using gint64
16:33:08 <stefw> so it should be 63 bits
16:33:10 <mvollmer> i guess json-glib might output proper 64bit integers.
16:33:15 <stefw> yes it should
16:33:24 <dominikp> we can do calculations before we get to js
16:33:26 <stefw> although json-glib has been known to have bugs :D
16:33:40 <mvollmer> dominikp, yes, that's what we do.
16:33:41 <stefw> right, that's what he's doing with the diff code
16:33:53 <fche> I don't think json per se dictates bit-width of the numeric literals
16:34:07 <dominikp> ok then :)
16:34:09 <stefw> fche, right, but javascript does
16:34:49 <stefw> ok
16:34:58 <stefw> #topic Testing and CI
16:35:05 * stefw wishes jscotka was still around
16:35:24 <stefw> So today I was working on getting CI going for our integration test suite
16:35:36 <stefw> I've upgraded files.cockpit-project.org to RHEL 7 so it's easier to run VMs and containers on it
16:35:41 <stefw> and got this far: http://files.cockpit-project.org/jenkins/
16:36:04 <stefw> you can see that we have a cockpit git master build, which fails during testing
16:36:16 <stefw> Due to an invalid /tmp directory for the jenkins user
16:36:33 <stefw> but rather than fix the symptom, i think the issue is that the build/test environment is not contained well enough
16:37:11 <stefw> so i'm working on getting reproducable jenkins slaves that we instantiate from snapshots (either qcow2 and/or ostree) and run in a container
16:37:29 <stefw> i'd like to also run ./VERIFY in such a container, so that all its dependencies are isolated from the host physical system
16:37:54 <mvollmer> but it would still spin up qemu, right?
16:37:56 <dominikp> what about multiple containers to test inter-machine operation?
16:38:03 <stefw> mvollmer, yes, at least for now
16:38:30 <stefw> dominikp, we currently use qemu virtual machines to do that, but it is possible that some of those use cases can be replaced by containers
16:38:46 <stefw> i'm trying to scope this work a bit:
16:39:11 <stefw> * Get the current VERIFY test suite running as CI against two OS's (Fedora 21 and Rawhide)
16:39:34 <stefw> and i hope we can go from there with further changes incrementally
16:39:53 <stefw> I see the lack of testing against multiple OS's as something that is going to bite us
16:40:50 <stefw> #info http://files.cockpit-project.org/jenkins/ is up and running
16:40:56 <stefw> But I do have a question that i wanted to bring up
16:40:59 <stefw> and that's about security
16:41:14 <stefw> it seems that Travis will happily run/build whatever you want it to
16:41:14 <mvollmer> stefw, is jenkins in a VM?
16:41:20 <stefw> yes the jenkins master is in a VM
16:41:34 <mvollmer> but the build slaves are not?
16:41:42 <stefw> well the executor for the build slaves will not be
16:41:47 <stefw> but that's not such a big deal
16:41:50 <stefw> we can try and script those well
16:42:06 <mvollmer> ok
16:42:07 <stefw> so essentially each time a build slave is needed, it'll ask the host to spawn one, and it'll spawn a container
16:42:16 <stefw> although tbh, containers are not secure either
16:42:43 <stefw> with CI pulling from github.com it's really easy to execute someone's code from their pull request
16:43:10 <stefw> how do other people do this? do they only test once something has landed into master?
16:43:13 <stefw> or post review?
16:43:23 <stefw> ie: manually triggered?
16:44:15 * stefw still can't believe that Travis CI actually lets you run whatever shit you want for 10 minutes
16:44:48 <dominikp> they treat requests from somewhere else differently
16:45:03 <dominikp> http://docs.travis-ci.com/user/pull-requests/
16:45:49 <stefw> isn't that about secrets?
16:46:15 <stefw> in other words travis will still happily execute code from random pull requests
16:46:18 <dominikp> they don't really screen pull requests is what I gather from that
16:46:20 <stefw> it just won't share your projects secrets with them
16:47:03 <dominikp> how much effort would it be to create a vm for the test?
16:47:19 <stefw> yes, we could do that
16:47:28 <dominikp> it won't prevent malicious code
16:47:29 <stefw> and that would sorta insulate the host from security issues
16:47:32 <dominikp> but it would preserve the host
16:47:38 <stefw> but the code could still attack the rest of the internet
16:48:03 <dominikp> you can restrict the vm (traffic, firewall, ...)
16:48:13 <dominikp> and a timeout takes care of the rest
16:48:35 <stefw> yes but this is really getting complicated fast
16:48:40 <stefw> in addition jenkins has security issues
16:48:44 <stefw> where slaves can tell the master to do things
16:50:04 <dominikp> are there alternatives?
16:50:19 <stefw> i don't know. i guess we have to figure that out
16:50:27 <stefw> manually triggering the build after review is an alternative
16:50:35 <dominikp> does the master have to be on the host, or can it run in a vm?
16:51:11 <stefw> it's in a VM
16:51:23 <stefw> but all the same issues apply there
16:51:44 <dominikp> you can reset it regularly
16:51:51 <stefw> anyway, i can do more research around this, i guess
16:52:12 <stefw> #info No real nice answer to the CI security issues, stefw will continue to look into it
16:52:13 <dominikp> I don't think manual triggering is good, though
16:53:04 <dominikp> not until it's actually an issue
16:53:30 <stefw> well manual triggering would let us start using this right away
16:53:38 <stefw> while we wait to figure out the plethora of security issues
16:53:42 <stefw> i agree it's suboptimal
16:54:10 <dominikp> can you exempt a github group from manual approval?
16:54:32 <stefw> i can look into that
16:55:19 <stefw> #action stefw will try to get jenkins and github working together with manual approval and perhaps a whitelist
16:55:24 <dominikp> manual triggering wouldn't be that bad if access to the trigger isn't too limited
16:55:52 <stefw> #topic Roadmap
16:56:03 <stefw> Should we wait until at least andreasn and others are here for this?
16:56:35 <stefw> That is, in order to start updating our roadmap and planning out the steps?
16:56:45 <stefw> mvollmer, you still around?
16:56:49 <mvollmer> yep
16:56:54 <dominikp> it would be better, probably
16:57:14 <stefw> i think we have lots of warming up to do, to get back into the rhythm of things today and tomorrow
16:57:14 <dominikp> I can pick my way along until then
16:57:24 <stefw> i tihnk more folks will join in on Wednesday
16:58:48 <stefw> alright anything else?
16:58:52 <stefw> #topic Open Floor
17:00:53 <stefw> #endmeeting