19:02:16 <mattdm> #startmeeting Council (2018-05-16)
19:02:16 <zodbot> Meeting started Wed May 16 19:02:16 2018 UTC.
19:02:16 <zodbot> This meeting is logged and archived in a public location.
19:02:16 <zodbot> The chair is mattdm. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:16 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:02:16 <zodbot> The meeting name has been set to 'council_(2018-05-16)'
19:02:18 <mattdm> #meetingname council
19:02:18 <zodbot> The meeting name has been set to 'council'
19:02:20 <mattdm> #chair mattdm jkurik jwb langdon robyduck bexelbie dperpeet Amita nb dgilmore pbrobinson
19:02:20 <zodbot> Current chairs: Amita bexelbie dgilmore dperpeet jkurik jwb langdon mattdm nb pbrobinson robyduck
19:02:22 <mattdm> #topic Introductions, Welcomes
19:02:22 <dgilmore> hola
19:02:31 <mattdm> hello everyone!
19:02:32 * pbrobinson waves
19:02:41 <dperpeet> greetings :)
19:02:58 <langdon> shoot.. completely forgot about this.. i have an eye appt "now" .. ill pay attention as long as i can
19:03:00 <mattdm> cool -- two of the objectives leads. which is good because this is the objectives update meeting
19:03:01 <langdon> .hello2
19:03:02 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
19:03:11 <langdon> mattdm: double oops
19:03:21 <nb> .hello2
19:03:23 <zodbot> nb: nb 'Nick Bebout' <nb@nb.zone>
19:03:25 <mattdm> okay let's start and make you go first?
19:03:28 <langdon> sgallagh: you around?
19:03:39 <mattdm> or wait for sgallagh?
19:03:43 <sgallagh> .hello2
19:03:44 <langdon> it is literally started 3m ago and the doc is late
19:03:46 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
19:04:04 <mattdm> #info Today, we're doing Objectives Updates
19:04:16 <dperpeet> .hello2
19:04:17 <zodbot> dperpeet: dperpeet 'None' <dperpeet@redhat.com>
19:04:23 <mattdm> let's start with modularity until langdon's doctor appears :)
19:04:28 <mattdm> #topic Fedora Modularity
19:04:28 <langdon> ok.. appt now.. sorry :(
19:04:33 <mattdm> #undo
19:04:33 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x7fa32c667b50>
19:04:35 * sgallagh has a hard stop in 30 min
19:04:46 <mattdm> oh, redo again. got time for a quick update?
19:04:50 <mattdm> #topic Fedora Modularity
19:05:10 <mattdm> #info update from sgallagh covering for langdon since langdon is having his eyeballs poked at with needles or something
19:05:15 <sgallagh> #info Fedora 28 Server Edition was released with modules available by default
19:05:20 <mattdm> \o/
19:05:40 <mattdm> Feedback on that has been good. Lots of press interest
19:05:52 <mattdm> And positive comments on e.g. Hacker News.
19:05:55 <sgallagh> I wasn't at Summit, but I'm told many questions were asked at the booth
19:05:56 <bexelbie> is this meeting on video?
19:05:58 <bexelbie> .hello bex
19:06:01 <mattdm> bexelbie: nope!
19:06:02 <zodbot> bexelbie: bex 'Brian (bex) Exelbierd' <bexelbie@redhat.com>
19:06:05 <mattdm> it's on typing.
19:06:23 <bexelbie> +1
19:06:49 <x3mboy> !
19:06:59 <mattdm> x3mboy: if it's about modularity, go for it :)
19:07:15 <sgallagh> x3mboy: I have a couple more things to say, then I'll open the floor if htat's okay
19:07:16 <x3mboy> Sure. I was being asked about more info on how to deal with modules and install profiles
19:07:28 <x3mboy> Sorry, my fingers were fast
19:07:31 <sgallagh> No problem
19:08:02 <x3mboy> eom
19:08:05 <sgallagh> Going forward, our next milestone is to have the modular repositories available for ALL Fedora installs, not just Server
19:08:37 <sgallagh> At present, there is one serious blocker and a handful of important bug-fixes that need to be addressed before that can happen.
19:08:54 <mattdm> Is that including support in Software, or just "Software won't mess things up"?
19:08:55 <sgallagh> We expect this to be resolved for F29 Beta
19:09:48 * jwboyer is here.  apologies for being late
19:09:50 <sgallagh> For F29, we're committing to "Software won't mess things up". I'm personally hoping to work with hughsie and kalev to get to "Software can install modules"
19:10:02 <mattdm> sgallagh++
19:10:05 <zodbot> mattdm: Karma for sgallagh changed to 6 (for the f28 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
19:10:07 <sgallagh> But the former is a blocker.
19:10:13 <mattdm> *nod*
19:10:34 <sgallagh> .bug 1575626
19:10:36 <zodbot> sgallagh: Bug 1575626 – libdnf does not handle module stream updates properly - https://bugzilla.redhat.com/1575626
19:10:36 <sgallagh> specifically
19:11:09 <mattdm> #info Module repositories coexisting with packagekit/microdnf (even if module install is not supported) should be in place by F29 beta (September)
19:11:17 <sgallagh> For another serious issue, we discovered today that DNF doesn't behave entirely as expected with regards to the difference between "default" modules and "enabled" modules.
19:11:28 <sgallagh> (The difference being subtle but important)
19:11:42 <sgallagh> s/should/must/
19:12:05 <mattdm> #undo
19:12:05 <zodbot> Removing item from minutes: INFO by mattdm at 19:11:09 : Module repositories coexisting with packagekit/microdnf (even if module install is not supported) should be in place by F29 beta (September)
19:12:13 <mattdm> #info Module repositories coexisting with packagekit/microdnf (even if module install is not supported) will be in place by F29 beta (September)
19:12:17 <sgallagh> ack
19:12:22 <mattdm> #info (it's a release blocker)
19:12:23 <jwboyer> sgallagh, link to a bug?
19:12:41 <sgallagh> jwboyer: I was just filing it when I got pinged, so it doesn't exist yet :)
19:12:57 <jwboyer> sgallagh, ok.  can you throw me on the CC list?
19:13:01 <sgallagh> Will do
19:13:05 <jwboyer> thx
19:13:25 <mattdm> How are things going with convincing people to *make* modules?
19:13:30 <sgallagh> In the interest of not wasting time, I'll avoid going through the details.
19:13:37 <jwboyer> sure
19:13:56 <sgallagh> mattdm: Slowly, but I haven't been hearing nearly as many vitriolic complaints as I did a few months ago
19:14:06 <mattdm> I guess I'll take it :)
19:14:29 <sgallagh> I'd like to see involvement grow faster, and I hope to get some response to a blog series we're preparing on the subject.
19:14:48 <sgallagh> I'm aiming to get those pitched to CommBlog in the next week or two
19:14:58 <mattdm> sgallagh: Is there any progress on making the hoops to make a module less ... hoopy?
19:15:16 <mattdm> (Like, needing to have a module.md to review before I can even get git branches created?)
19:15:24 <sgallagh> mattdm: Can you be any less specific? You almost provided some context there :-P
19:15:43 <sgallagh> mattdm: That's no longer the case, IIRC.
19:15:43 <mattdm> https://pagure.io/modularity/issue/97
19:16:02 <sgallagh> Let me check to make sure that got approved by FPC, but I know we had a streamlined process out.
19:16:27 * mattdm is not sure why fpc involvement needed because it's not a packaging thing
19:16:51 <sgallagh> #link https://fedoraproject.org/wiki/Modularity/Adding_New_Modules_and_Managing_Defaults
19:17:09 <sgallagh> This is considerably more streamlined than the earlier version.
19:17:37 <sgallagh> It requires no re-review for modules
19:17:57 <mattdm> sgallagh: okay, cool. I'll look later
19:18:09 <mattdm> and we should publicize that. if it was annoucned, I missed it
19:18:22 <sgallagh> OK, I think that's my status. Let me scroll back to x3mboy's question...
19:18:23 <mattdm> and that might bring back some people who looked long ago and gave up.
19:18:36 <mattdm> ok cool. and dperpeet or pbrobinson, which of you wants to go next? :)
19:18:43 <dperpeet> I can go :)
19:18:53 * pbrobinson doesn't mind
19:18:58 <sgallagh> x3mboy: Actually, that was a statement. What was thee question?
19:19:02 <sgallagh> *the
19:19:08 <mattdm> *thine
19:19:13 <sgallagh> What specifically do you want to know?
19:19:24 <dgilmore> mattdm: you are getting old :)
19:19:41 <mattdm> just hang out with Quakers
19:20:07 <x3mboy> sgallagh, where people can fin documentation on "how to install a module"; "How to update a module"; "How to install a module with an specific Install Profile"; "What is an Install Profile"
19:20:15 <pbrobinson> traditional english....
19:20:39 <sgallagh> x3mboy: Those are topics I intend to answer with a Fedora Magazine article soon.
19:20:53 <bexelbie> sgallagh, can we get some docs as well in quickdocs or a modularity docs repo?
19:20:57 <dgilmore> you're up dperpeet
19:21:12 <x3mboy> sgallagh, Cool. The only answer I have for that kind of questions is https://docs.fedoraproject.org/fedora-project/subprojects/fesco/en-US/Using_Modules.html
19:21:16 <sgallagh> bexelbie: Yeah, that would be a great idea. Can you point me (offline) to the right contacts?
19:21:30 <bexelbie> sgallagh, talk with asamalik or me
19:21:35 <dperpeet> so should I start with CI update?
19:21:41 <mattdm> #topic Fedora CI Update
19:21:45 <mattdm> dperpeet: yes please :)
19:21:49 <dperpeet> thank you :)
19:21:51 <sgallagh> bexelbie: OK, if asamalik is the right person, I'll just ask him to do it, since he's on my team :)
19:21:55 <sgallagh> OK< I think I'm done.
19:22:08 <dperpeet> the short of it: CI is currently disabled
19:22:46 <dperpeet> gating was killed completely last week, with a dramatic irc fesco vote (I gleaned that from a ticket)
19:22:47 <mattdm> sadface
19:22:51 <dperpeet> #link https://pagure.io/fesco/issue/1872
19:23:11 <mattdm> As I understand it, tests wree failing to run at all
19:23:22 <mattdm> and a missing test is (rightly) interpretted as failure
19:23:35 <dperpeet> there were some issues with waiving test results, and not well enough documented waiving mechanisms
19:23:37 <mattdm> overall resulting in something like 20% of packages failing
19:23:56 <Evolution> 9%, from the numbers bowlofeggs gave me
19:24:09 <Evolution> but that's still way too high, as on that day that was ~=100 packages
19:24:11 <dperpeet> the good news is that greenwave is now closer to implementing the real opt-in mechanism
19:24:20 <dperpeet> where each package can define its policy
19:24:33 <dperpeet> in addition to the global ones
19:25:13 <dperpeet> so while it's down for now (which makes me very sad), we will soon be in a place of higher control for individual packagers
19:25:32 <dperpeet> I truly hope that gating is enabled again soon, even if we keep the permissive policy in place
19:25:54 <dperpeet> which I had hoped would be chosen instead of disabling gating
19:26:08 <mattdm> What is being done to address the failing test runs?
19:26:24 <dperpeet> for one we're working on improved CI monitoring
19:27:16 <dperpeet> #link https://fedoraproject.org/wiki/CI/Tests/recent_builds
19:27:21 <dperpeet> that's one such place to take a look
19:27:45 <jwboyer> to be clear, these were tests failing to start.  not failures of the tests catching actual bugs, correct?
19:28:11 <dperpeet> I believe so, I think there were some issues last week with the infrastructure
19:28:22 <mattdm> yeah, Evolution said that there were only something like 10 actual issues
19:28:32 <jwboyer> ok good.  because if we disabled gating because 100 packages had actual test failures, i would be pretty annoyed.
19:28:44 <mattdm> (which was soemthing like 0.8%, whcih seems right)
19:29:03 <dperpeet> also, it turns out that due to a bug in bodhi, gating wasn't actually working before
19:29:21 <dperpeet> that was discovered after everyone thought it worked and had turned to other things :)
19:29:22 <mattdm> Was the hundred failures just a fluke that's unlikely to repeat, or was there something fundamental?
19:29:50 <Evolution> I don't think it's a fluke.
19:30:12 <mattdm> dperpeet: so in that link, light blue is "test failed to run"?
19:30:13 <jwboyer> do we have actual data that says it isn't a fluke?
19:30:41 <dperpeet> light blue means tests weren't run, nothing was done
19:30:50 <dperpeet> most likely because there aren't tests in the dist-git repo
19:31:03 <dperpeet> yellow are actual test failures
19:31:15 <dperpeet> red is infrastructure, those are the ones we don't want :)
19:31:19 <mattdm> so it's the red "infra failures" that were previously endemic?
19:31:33 <dperpeet> I'm not sure which numbers were the problem
19:31:40 <dperpeet> I believe the turning point was one CVE update
19:31:53 <dperpeet> which couldn't be waived through on the same day
19:32:01 <mattdm> well, it seems like *every* infra failure is a problem
19:32:12 <mattdm> unless it's one in a hundred thousand or something
19:32:21 <dperpeet> yes, and we're working on ironing those out and retriggering
19:32:48 <dperpeet> we're still running the tests now, even with gating disabled
19:32:59 <dperpeet> so results are still available
19:33:12 <mattdm> Right now, there's 9 builds with tests at all on that page, and 4 of them are infra fail
19:34:13 <dperpeet> right, but mostly we need to address the UX issues
19:34:21 <dperpeet> there will always be failures
19:34:28 <Evolution> I disagree.
19:34:36 <dperpeet> retriggering tests and waiving results always need to be possible
19:34:42 <Evolution> we do not want to train packagers to waive tests
19:35:16 <dperpeet> I wouldn't say train
19:35:47 <dperpeet> the tests are in because the packagers want to have them
19:35:53 <mattdm> If my tests fail for no reason more often than they fail for reason, that trains me to not use them
19:36:19 <Evolution> the UI needs to be fixed. no question. *BUT* it is more important to make sure that the tests run reliably
19:36:52 <dperpeet> but if they are disabled and have no effect at all, there is not as much reason to work on the tests at all
19:36:58 <dperpeet> that said
19:37:09 <dperpeet> these efforts are worked on by different people
19:37:16 <dperpeet> so luckily we don't need to weigh them against each other :)
19:37:23 <mattdm> yes, good. :)
19:37:37 <dperpeet> we're definitely working on improving the test results
19:37:40 <dperpeet> hence the CI monitoring
19:37:44 <dperpeet> with proper end to end monitoring
19:37:51 <dperpeet> so we can be alerted if something goes wrong
19:37:56 <mattdm> Is it the infra team working on the failure problem? do we need to focus more people on that?
19:38:15 <dperpeet> my team and the pipeline folks are working on those issues
19:38:19 <mattdm> or to put it another way, who owns each failure?
19:38:23 <dperpeet> we need the infra team to make sure retriggering tests works
19:38:33 <mattdm> ok
19:38:52 <Evolution> dperpeet and I (along with a few others) have a meeting about this tomorrow morning
19:38:56 <dperpeet> I'll make a note to prepare some statistics for next time
19:39:13 <mattdm> okay, thanks both of you :)
19:39:34 <mattdm> dperpeet: how long does it take for that recent builds page to refresh?
19:39:44 <dperpeet> mattdm, I believe every 3 hours or so
19:39:49 <dperpeet> sliding window of 24 hours
19:39:57 <mattdm> okay. I just built something, so I'll check back this evening :)
19:40:13 <dperpeet> we didn't want to clobber the infrastructure with polling :)
19:40:32 <mattdm> fair. especially if it's already having some issues. Hopefully those can be worked out soon!
19:40:41 <mattdm> Is there anything else going on we shoudl know about?
19:41:25 <dperpeet> I think that's it - I seriously hope gating is enabled again soon, with opt-in policy
19:41:29 <mattdm> how are we doing on the Key Results from https://fedoraproject.org/wiki/Objectives/Continuous_Integration_and_Delivery_of_Fedora_Atomic_Host#Key_Results?
19:41:45 <mattdm> yeah, I hope so too
19:42:55 <dperpeet> I can't seem to find the atomic statistics page now
19:43:03 <dperpeet> but for the Basic Operating System we have this
19:43:05 <dperpeet> #link https://fedoraproject.org/wiki/CI/Tests/stat
19:43:22 <dperpeet> and for Fedora Server
19:43:24 <dperpeet> #link https://fedoraproject.org/wiki/CI/Tests/stat_fedoraserver
19:43:38 <dperpeet> with roughly 75 repos that have tests merged
19:43:56 <mattdm> that first link has an "atomic" column, with 42 packages
19:44:08 <dperpeet> well, those are the tests that actually cover the atomic host
19:44:32 <mattdm> oh, not the packages in it?
19:44:38 <dperpeet> there might be packages in Atomic that have tests, even if they don't cover Atomic Host
19:44:45 <dperpeet> but that's nitpicking
19:44:45 <mattdm> ah ok.
19:45:10 <dperpeet> with gating enabled overall, I have to admit we've shifted the focus slightly away from the Atomic only tests
19:45:25 <dperpeet> that said, we have made some progress on tests that cover the Atomic Host compose once it's done
19:45:35 <dperpeet> to enable those to be run in the CI pipeline
19:45:53 <mattdm> oh, that's good too. so, like, actually *integration* testing :)
19:45:57 <dperpeet> yes :)
19:46:06 <mattdm> Anyway, thanks for the update. Let's move on to IoT :)
19:46:11 <mattdm> #topic Fedora IoT
19:46:12 <dperpeet> thanks for your time!
19:46:15 <mattdm> thank you!
19:46:21 <mattdm> pbrobinson: wake up :)
19:46:38 * pbrobinson yawns
19:47:22 <mattdm> I hear something about a nightly Fedora IoT image for testing?
19:47:35 <pbrobinson> so nightly compose is running
19:47:44 <pbrobinson> almost consumable
19:48:09 <pbrobinson> only an initial-setup bug 1578930 to fix
19:48:22 <pbrobinson> and then I'll be sending an announcement out
19:48:37 <pbrobinson> with basic documentation on getting started
19:48:55 <Evolution> \o/
19:48:58 <bexelbie> will there be news for Raspberry PI here as well? It's been asked about to me directly recently
19:49:21 <pbrobinson> bexelbie: raspberry pi in which context?
19:49:44 <mattdm> well, for example, will this thing that is almost consumable be something to run on the Pi?
19:49:44 <bexelbie> pbrobinson, better support .. there were things said that I didn't fully process as i am not a Pi person so I don't know the current state
19:49:50 <jwboyer> bexelbie, no
19:49:50 <mattdm> Or is it targetting something else?
19:49:54 <mattdm> Wait, do my question first :)
19:50:04 <jwboyer> bexelbie, send general pi questions to the weekly arm meeting
19:50:17 <pbrobinson> bexelbie: define "better support"
19:50:25 <bexelbie> jwboyer, I don't know that hte majority of that audience is going to attend our IRC meetings
19:50:35 <bexelbie> they may not be a priority for us, but we shoul dhave a Maker-person answer
19:50:49 <jwboyer> bexelbie, great.  not now, not in the context of IoT
19:50:50 <jwboyer> come on
19:50:54 <bexelbie> pbrobinson, /me has nothing more here as I am not a Pi person
19:50:54 <jwboyer> let's stay on topci
19:51:09 <pbrobinson> so the plan is that the first nightly composes I'll be testing on virt for x86_64 and aarch64, plus the RPi and 96boards 410c
19:51:19 <bexelbie> jwboyer, my understanding was that Pi was a "poor mans IoT endpoint"
19:51:21 <mattdm> i'm going to #info that :)
19:51:41 <pbrobinson> mattdm: my bit?
19:51:56 <mattdm> #info The plan is that the first IoT nightly composes will be tested on virt for x86_64 and aarch64, plus Raspberry Pi and 96boards 410c
19:52:16 <mattdm> yes :)
19:52:34 <mattdm> so, does that mean that people looking to contribute should look to one of those devices?
19:52:58 <mattdm> and is that https://www.96boards.org/product/dragonboard410c/?
19:53:18 <pbrobinson> so we'll support everything we already support for aarch64/x86_64 but I need to focus on small testing
19:53:27 <pbrobinson> mattdm: yes
19:54:16 <mattdm> #link https://www.96boards.org/product/dragonboard410c/
19:54:25 <mattdm> yeah, focusing small makes sense
19:54:35 <mattdm> hence me emphasizing this
19:54:53 <pbrobinson> so it's not excluding anything and that'll be mentioned in the ticket but we can't work on everything on the outset and that's what community is for ;-)
19:55:06 <pbrobinson> s/ticket/email
19:55:25 <mattdm> yeah, so let's talk about community building in the remaining four minutes :)
19:55:37 <pbrobinson> right
19:55:57 <pbrobinson> so I'm going to set up a weekly meeting once the nightly is available
19:56:03 <mattdm> I'd like to see a WG formed, and "now there's something to see" seems like a good starting point.
19:56:14 <pbrobinson> and then move to testing and docs and WG stuff
19:56:30 <pbrobinson> yep, that's my exact plan
19:56:34 <mattdm> And I'd love to make sure that we have people other than just you doing testing and docs and stufff
19:56:44 <mattdm> and ideally even someone else to run the meeting, manage tickets, etc.
19:57:10 <pbrobinson> mattdm: yes, that's the plan
19:57:33 <mattdm> cool. do you want to do a general call for help, or do you have particular people in mind to tap?
19:57:48 <pbrobinson> once I have bits that people can consume it's then all about getting all the bits out of my head and to the people that are offering help/support etc
19:58:14 <mattdm> okay. as always, let us know how we can help.
19:58:22 <mattdm> And, with that, we're basically out of time....
19:58:24 <pbrobinson> I have some people already offering help, I'd prefer not do drown myself more than I already am
19:58:42 <pbrobinson> and once things start to get into a groove do a wider call for all sorts of things
19:58:48 <mattdm> makes sense.
19:59:04 <mattdm> We need to figure out how to make it scale so more people = more help, not more people = more work for you.
19:59:29 <mattdm> anyway, that's time for the meeting.
19:59:39 <pbrobinson> yep, agreed
19:59:43 <mattdm> thanks everyone!
19:59:45 <mattdm> dperpeet++
19:59:46 <zodbot> mattdm: Karma for dperpeet changed to 1 (for the f28 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
19:59:48 <mattdm> pbrobinson++
19:59:51 <mattdm> sgallagh++
19:59:52 <zodbot> mattdm: Karma for pbrobinson changed to 2 (for the f28 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
19:59:52 <dperpeet> thanks everyone!
20:00:01 <mattdm> #endmeeting