14:02:52 <mvollmer> #startmeeting Cockpit
14:02:52 <zodbot> Meeting started Mon Nov 23 14:02:52 2015 UTC.  The chair is mvollmer. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:52 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:02:52 <zodbot> The meeting name has been set to 'cockpit'
14:02:53 <dperpeet> :)
14:03:00 <dperpeet> .hello dperpeet
14:03:01 <zodbot> dperpeet: dperpeet 'None' <dperpeet@redhat.com>
14:03:04 <mvollmer> .hello mvo
14:03:08 <zodbot> mvollmer: mvo 'Marius Vollmer' <marius.vollmer@gmail.com>
14:03:11 <andreasn> .hello andreasn
14:03:12 <zodbot> andreasn: andreasn 'Andreas Nilsson' <anilsson@redhat.com>
14:03:18 <stefw> .hello stefw
14:03:19 <zodbot> stefw: stefw 'Stef Walter' <stefw@redhat.com>
14:03:36 <mvollmer> #topic Agenda
14:03:49 <dperpeet> * avocado / selenium testing
14:04:03 <mvollmer> * Debian
14:05:50 <mvollmer> Ready?
14:05:57 <andreasn> * sos-report maybe
14:06:10 <andreasn> but yeah, ready
14:06:12 <mvollmer> yes
14:06:19 <mvollmer> #topic avocado / selenium testing
14:06:26 <dperpeet> jscotka has done some work to get browser testing via selenium into Cockpit https://github.com/cockpit-project/cockpit/pull/3190
14:06:50 <dperpeet> we can also use this opportunity to merge the avocado tree https://github.com/cockpit-project/cockpit/tree/master/test-avocado into test
14:07:05 <dperpeet> now, avocado tests are designed a bit differently
14:07:15 <dperpeet> since they ultimately should run on bare-metal also, they need cleanup
14:07:32 <andreasn> Selenium as in http://docs.seleniumhq.org/ ?
14:07:35 <dperpeet> that means, they could all run as one big test in the old style
14:07:45 <dperpeet> andreasn, yes, correct
14:07:51 <jscotka> I've added possibility to use selenium server and instruction how to use docker in selenium
14:08:06 <dperpeet> but - if we run all the avocado tests in one go, we get that as one single entry result in our test logs
14:08:16 <dperpeet> all tests passed - or they failed, no count
14:08:38 <dperpeet> if we want to stick to our clean-slate premise, we can autogenerate tests
14:08:41 <dperpeet> http://stackoverflow.com/questions/32899/how-to-generate-dynamic-parametrized-unit-tests-in-python/32939#32939
14:08:51 <dperpeet> to get all the avocado tests in their own instance
14:08:55 <dperpeet> the question is, is this necessary?
14:09:01 <dperpeet> or should we hack/parse the result somehow
14:09:11 <dperpeet> and just run it in a single vm instance
14:09:26 <dperpeet> so, avocado run test_a test_b test_c
14:09:42 <dperpeet> or separate vms with "avocado run test_a", "avocado run test_b" et cetera
14:10:03 <dperpeet> I think the cleanest solution is to start a vm for each test, as we do for the others
14:10:21 <dperpeet> and in the bare metal case, call them all together
14:10:29 <dperpeet> with a different starting mechanism
14:10:50 <dperpeet> what do you think?
14:11:06 <mvollmer> whatever is easier, I'd say.
14:11:12 <stefw> dperpeet, we could run them as one go for now
14:11:18 <mvollmer> yeah
14:11:19 <jscotka> perfect. I vote for that.
14:11:19 <stefw> but once avocado has TAP output, it would be easy to integrate it
14:11:44 <dperpeet> the advantage of that is less borderline black magic code (generating tests via source)
14:11:51 <dperpeet> and less divergence between bare metal and vms
14:12:04 <jscotka> stefw, I hope so, maybe I can simulate now it via xunit XSLT trasformation to TAP
14:12:24 <dperpeet> I have another note on this
14:12:41 <dperpeet> to reduce testing deps on the host and version issues, I run avocado locally inside the test vm
14:12:51 <dperpeet> and copy the test files across for that
14:12:59 <dperpeet> that way, avocado doesn't have to be on the host
14:13:22 <dperpeet> any objections to that?
14:13:36 <jscotka> it is clean solution in case you would like to avoid changing hosts and check versions of avocado
14:13:55 <dperpeet> yes, avocado is version sensitive (host vs guest)
14:14:15 <dperpeet> we won't be able to run any tests that way that tinker with connection speed
14:14:24 <stefw> i think that's fine
14:14:26 <dperpeet> but I don't think that's a priority right now
14:14:34 <stefw> the goal for jscotka is to run avocado on the machine
14:14:35 <dperpeet> and avocado tests aren't meant to replace the old ones anymore, I guess
14:14:41 <stefw> correct
14:14:47 <stefw> the goal is to have separate tests that do not require a virtual machine
14:14:52 <stefw> and test certain aspects of cockpit
14:14:57 <dperpeet> but run them in a vm for us anyway
14:15:02 <stefw> yes, we run them in a vm
14:15:09 <stefw> but the tests themselves don't require a virtual machine
14:15:17 <dperpeet> yes, then we're on the same page
14:15:21 <stefw> testing on certain architecturs requires this
14:15:45 <stefw> architectures where virtualization is either extremely slow, or non-existant
14:15:56 <stefw> but it's nice we can run the same tests
14:15:59 <stefw> in an x86_64 vm
14:16:04 <stefw> and make sure they don't regress, etc
14:16:08 <dperpeet> also helps to keep them updated
14:16:21 <dperpeet> ok, that's it on this topic from me
14:17:11 <mvollmer> next?
14:17:31 <mvollmer> #topic Debian
14:17:46 <mvollmer> I am reworing how we build packages a bit
14:17:55 <mvollmer> with an eye towards debian
14:18:16 <mvollmer> #info https://github.com/cockpit-project/cockpit/pull/3197
14:18:27 <mvollmer> #info https://github.com/cockpit-project/cockpit/pull/3202
14:18:51 <mvollmer> With the first PR, we just copy a tarball into the VM, and then make a srpm out of it inside the VM
14:19:10 <mvollmer> building and installing is done in a single vm instance
14:19:23 <mvollmer> but needs to be done in separate ones for atomic, so that is broken right now
14:19:29 <mvollmer> or, incomplete
14:19:33 <stefw> why?
14:19:51 <mvollmer> for atomic we build on a different os than we install into, no?
14:19:56 <stefw> ah right, makes sense
14:19:59 <mvollmer> so, we need two VMs
14:20:11 <dperpeet> mvollmer, can we install without rebuilding with the changes you made?
14:20:12 <stefw> i thought you meant srpm runs in a different vm from building the packages
14:20:14 <mvollmer> nothing dramatic, but it needs extra code and extra testing
14:20:47 <mvollmer> dperpeet, there will be options, but not right now
14:21:05 <mvollmer> I am thinking --build-only, --install-only, or something like that
14:21:19 <mvollmer> but right now, you need to rebuild
14:21:40 <dperpeet> ok
14:21:42 <mvollmer> dperpeet, is that important for you?
14:21:46 <dperpeet> that is important when working on tests
14:21:52 <mvollmer> okay
14:21:56 <dperpeet> the code that's being tested remains constant
14:21:59 <mvollmer> I'll make sure we get that feature
14:22:10 <dperpeet> thanks!
14:22:20 <dperpeet> the seconds spent on building add up after a while :)
14:22:32 <mvollmer> i usually only re-run testsuite-prepare when the code has changed
14:22:52 <mvollmer> the tests themselves don't make any changes to the images, so they stay clean.
14:23:27 <mvollmer> anyway, on to Debian
14:23:37 <mvollmer> the basics are in place
14:23:40 <mvollmer> image creation
14:23:45 <mvollmer> building with pbuilder
14:24:01 <mvollmer> the Debian people are given great hints on github
14:24:10 <mvollmer> "are giving"
14:24:12 <mvollmer> sorry
14:24:23 <mvollmer> one thing came up
14:24:57 <mvollmer> should we use "make dist" instead of "git archive" to create the source tarball?
14:25:04 <stefw> the latter is much faster
14:25:06 <stefw> and that's why we use it
14:25:12 <stefw> for work in progress stuff
14:25:16 <mvollmer> yeah
14:25:21 <stefw> 'make dist' and 'make distcheck' are used to produce official tarballs
14:25:24 <stefw> which we actually build releases from
14:25:30 <stefw> the test packages don't need that
14:25:58 <mvollmer> i have to talk more with mbiebl, but right now it sounds as if they expect us to test exactly what they'll ship.
14:26:13 <mvollmer> like, the binaries that fall out of our CI will be uploaded into Debian.
14:26:14 <stefw> that sounds great, fedora isn't set up for that
14:26:20 <stefw> so it would be a step forward
14:26:27 <stefw> but lets not get ahead of ourselves :)
14:26:30 <mvollmer> but then we need to use "make dist", no?
14:26:32 <stefw> but yes, doing promotion right would be awesome
14:26:38 <stefw> mvollmer, we probably would need to do a full make distcheck
14:26:40 <stefw> get versions right
14:26:41 <stefw> etc...
14:26:44 <mvollmer> yep
14:26:53 <mvollmer> but maybe I read to much into the comments
14:26:55 <stefw> so it would be an alternate mode of running tihs
14:26:58 <dperpeet> can we tie that to "thorough"?
14:27:10 <stefw> distros usually have requirements on *where* stuff is built=
14:27:11 <mvollmer> https://github.com/cockpit-project/cockpit/pull/3202#issuecomment-158940304
14:27:37 <mvollmer> he seems to apply the criteria for official Debian builds to our CI builds.
14:27:40 <stefw> mvollmer, the actual debian file needs to have two modes
14:27:42 <stefw> like the spec file
14:27:48 <stefw> one where we build from git
14:27:52 <stefw> and the other where we build from a tarball
14:27:57 <stefw> er, debian files
14:28:04 <stefw> look at the spec file
14:28:08 <mvollmer> yep
14:28:24 <dperpeet> I have a follow-up topic here
14:28:37 <mvollmer> but we can sort that out as we make progress
14:28:54 <mvollmer> in any case, cooperation is happening and is very close
14:29:25 <mvollmer> they are fixing the failing unit tests
14:29:43 <mvollmer> open issue for me: how to install build deps
14:29:50 <mvollmer> it's a pain on debian
14:29:53 <mvollmer> always was
14:30:18 <mvollmer> pbuilder can do it, maybe it can expose it somehow
14:30:22 <dperpeet> create a developer package with the package dependencies?
14:30:42 <mvollmer> yes, mk-build-deps
14:31:28 * mvollmer remembers when he had a tool that made a local apt repo for the source package so that one can use apt-get builddeps
14:31:44 <mvollmer> maybe I can find that code...
14:32:07 <mvollmer> so, good progress all around
14:32:22 <dperpeet> nice work!
14:32:57 <mvollmer> thanks!
14:33:09 <mvollmer> dperpeet, follow up topic now?
14:33:24 <dperpeet> yes, building twice in a row discussion
14:33:55 <mvollmer> #topic Building twice in a row
14:34:01 <dperpeet> since we're working on the building process right now, it might be a good time to discuss what to do regarding https://github.com/cockpit-project/cockpit/issues/3165 - question posed by stefw
14:35:01 <stefw> i was thinking about that a bit more, and perhaps we should have files that we conditionally clean, based on the presence of a .git directory
14:35:34 <stefw> But to summarize the basic issue is that we distribute generated sources (along with the originals) in the tarball
14:35:45 <stefw> so currently when doing a tarball build, 'make clean' removes those generated sources
14:35:59 <stefw> and then the build all of a sudden requires a lot more dependencies to regenerate those sources
14:37:23 <petervo> should we have a cleanall command for devs
14:37:44 <dperpeet> yeah, a separate mode might work
14:37:47 <petervo> and have clean be move conservative about what it removes
14:37:58 <stefw> yes, that's another idea, although i'm not super happy about plumbing a whole new command through
14:38:04 <stefw> or rather a whole new make target
14:38:16 <stefw> there is also --enable-maintainer-mode that we could use to change the behavior of 'make clean'
14:38:36 <petervo> that might work better
14:39:16 <dperpeet> our targets and build dependencies are convoluted enough as it is
14:39:24 <stefw> yup
14:39:48 <dperpeet> I think piggy-backing on maintainer-mode would work
14:39:59 <mvollmer> i think the standard is "make maintainer-clean"
14:40:15 <mvollmer> to clean generated and distributed files
14:40:31 <stefw> or dist-clean
14:40:32 <stefw> but yeah
14:40:36 <stefw> that's really annoying
14:40:39 <stefw> since you have to autogen again
14:43:28 <mvollmer> so the issue is that "make clean" cleans too much, right?
14:43:40 <stefw> in certain cases yes
14:43:45 <stefw> i think we can just make it clcean less in those cases
14:43:55 <mvollmer> yes
14:44:27 <mvollmer> how to clean those files is a less urgent issue, I'd say
14:44:37 <mvollmer> the ones that "make clean" doesn't clean anymore
14:44:43 <mvollmer> after we have fixed it
14:44:46 <stefw> ok
14:46:32 <andreasn> next topic?
14:46:37 <mvollmer> yep
14:46:45 <mvollmer> #topic sosreport
14:47:01 <mvollmer> it's waiting for the external channels
14:47:23 <cockpitbot> 1 tests failed - http://files.cockpit-project.org/logs/master-8ceec5d9-fedora-23/log.html
14:47:26 <andreasn> I was thinking of start making mockups for the additional things that people were asking for
14:47:44 <andreasn> that could then go in as a kind of phase 2 work
14:47:53 <andreasn> at some point
14:47:57 <mvollmer> what have people been asking for, just to make sure? :-)
14:48:06 <andreasn> plugins
14:48:37 <andreasn> that feels like it's the main one
14:48:51 <andreasn> but I might have forgotten something
14:48:55 <mvollmer> okay
14:49:05 <mvollmer> case number
14:49:29 <andreasn> and ticket number
14:49:36 <mvollmer> right
14:49:41 <andreasn> and name?
14:50:00 <mvollmer> yes
14:50:22 <andreasn> ok, I'll start sketching those out then
14:51:43 <mvollmer> cool
14:51:51 <andreasn> I think that was it on that
14:51:56 <mvollmer> the implementation for plugins might need a proper API in sosreport
14:52:06 <andreasn> ah, right
14:52:12 <andreasn> because now it's only flags?
14:52:50 <mvollmer> yeah, scraping stdout for the plugins list might be hairy
14:53:02 <andreasn> ugh, yes
14:53:20 <mvollmer> so, put those last and make them optional in the design
14:53:57 <andreasn> but how about the ticket number and stuff, is that the same?
14:54:05 <andreasn> or does that have an api?
14:54:18 <mvollmer> no, those are easy, just additional command line arguments
14:54:23 <andreasn> ah, ok
14:54:32 <mvollmer> the user will have to know them and type them in
14:55:07 <andreasn> plugins would be selected all by default I think, and then allow to uncheck under some expander (or something) I think
14:55:33 <mvollmer> ohh sorry, wait a second
14:55:40 <mvollmer> "profiles", not plugins
14:55:45 <mvollmer> let's do profiles
14:56:01 <mvollmer> almost the same thing, but there are less of them.
14:56:08 <andreasn> ah, I see
14:56:15 <andreasn> I'll have to read up a bit on that again
14:56:29 <andreasn> because I think I totally missed the concept of profiles last time I looked at it
14:56:48 <andreasn> but yeah, that sounds good
14:57:44 <mvollmer> hmm, I think F23 might not have profiles... is that possible?
14:57:50 <mvollmer> let's check off-line
14:58:06 <andreasn> yep
14:58:17 <andreasn> EOT I think
14:58:57 <mvollmer> yep
14:59:11 <mvollmer> (sosreport on my f2 is newer than on f23...)
14:59:23 <andreasn> odd
14:59:25 <mvollmer> #topic Any other business
15:00:48 <andreasn> talk deadlines for Devconf is coming up fairly soon I think
15:00:57 <andreasn> like early next week
15:01:48 <andreasn> nothing else apart from that
15:02:50 <mvollmer> can I help with the talks?
15:03:47 <dperpeet> I think that makes sense once we know what's accepted
15:03:58 <mvollmer> okay
15:04:06 <andreasn> yeah, the proposal doesn't have to be a full talk, right?
15:04:13 <andreasn> only a title and synopsis
15:04:49 <dperpeet> yes, pretty much
15:06:20 <mvollmer> alright!
15:06:25 <mvollmer> thanks everyone!
15:06:28 <andreasn> thanks!
15:06:29 <mvollmer> #endmeeting cockpit