13:30:53 #startmeeting Testing bleeding edge kernels 13:30:53 Meeting started Wed Aug 3 13:30:53 2016 UTC. The chair is bkm. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:30:53 Useful Commands: #action #agreed #halp #info #idea #link #topic. 13:30:53 The meeting name has been set to 'testing_bleeding_edge_kernels' 13:31:02 speaker Paul Moore 13:31:10 #meetingname flock2016 13:31:10 The meeting name has been set to 'flock2016' 13:31:20 #topic introduction 13:31:41 pmoore@redhat.com or paul@paul-moore.com 13:32:00 SELinux, audit, labeled networking kernel maintainer 13:32:13 Created and maintain libseccomp project 13:32:43 Talk about kernel testing problems and improvements 13:33:30 How do we make leading edge changes accessible? 13:33:42 #topic kernel development cycle 13:34:17 * Create patch 13:34:32 * get it reviewed and changes made 13:34:47 * Patches merged into a subsystem next branch 13:35:09 this varies by subsystem 13:35:53 * Linus pulls the next branches into tree 13:36:19 2 week merge window after each kernel is released 13:37:07 Approximately 8 week of testing before release 13:37:42 #topic Kernel testing today 13:38:32 * Kernel releases packaged for Fedora Rawhide 13:39:38 * linux-next repository merges 'next' branches daily 13:39:59 * Primarily focused on exposing merge conflicts 13:40:43 * Coverage unclear 13:41:01 * General user testing rare 13:41:27 * As such does not find subsystem interaction problems 13:41:41 #topic Problems with current approach 13:42:22 Less than 8 weeks to find and solve problems 13:44:00 'Next' branches based on stable/old code 13:44:50 Software development problem 13:45:25 No way to easily test subsystem interaction 13:46:03 #topic Kernel testing aspirations 13:46:36 Regular testing of the latest upstream developments 13:47:40 Make test kernels more accessible to users - not all users are comfortable building kernels, but can test out rpms 13:48:06 Needs to be cheap, easy and portable 13:49:05 * Automated build and test processes 13:49:50 #topic continuous integration as a solution 13:50:25 Connect development with test and deployment 13:50:56 Constant stream of new, verified releases 13:51:12 #topic Continuous integration problems 13:51:38 Many CI frameworks require infrastructure 13:52:51 Does not appear well suited to kernel testing 13:53:07 #topic Simplified CI for kernel developers 13:54:27 Create/find a set of easily run regression tests: self contained, can isolate individual tests 13:54:50 Automate patching of Fedora kernel source rpm 13:54:57 Automate building 13:55:12 Automate ROM distribution to public repository 13:55:50 Automate testing via virtual machines 13:56:13 #topic Progress so far 13:56:38 Useable test suite for SELinux and audit 13:57:01 * updated selinux-testsuite project 13:57:16 * created audit-testsuite since no suitable existing one 13:57:27 Codes on github 13:57:51 Fedora kernel patching automation 13:58:04 Copr repository for kernel build and distribution 13:58:26 Highly recommend Copr 13:58:41 * Weekly builds and test runs for the past year 13:58:55 #topic Lessons learned 13:59:15 Regular regression testing works - if you do it! 13:59:42 * Catches problems early 13:59:55 * Minimizes time looking through git log 14:00:33 * Less stress during merge window and RC cycle 14:01:15 Automated patching works better than expected 14:01:29 * Looking at automating further 14:01:59 Copr great tool, but has reliability issues 14:02:12 * Expect this to improve 14:02:19 #topic Future improvements 14:02:39 Automate more 14:02:47 * VM test execution 14:03:02 * Automatic build and test triggers 14:03:26 Increase test coverage 14:04:03 * Tests need to be easily and quickly run 14:04:28 * Not comprehensive tests, aim to catch simple mistakes early 14:04:52 Additional security subsystems beyond SELinux and audit 14:05:12 #topic PRoject links 14:05:41 #link https://github.com/SELinuxProject/selinux-testsuite 14:06:03 #link https://github.com/linux-audit/audit-testsuite 14:06:13 Bash scripts 14:06:44 #link https://github.com/pcmoore/copr-pkg_scripts 14:07:21 #link https://copr.fedoraincloud.org/coprs/pcmoore/kernel-secnext 14:07:59 #topic Questions 14:08:05 Blog post coming 14:08:40 #topic Example installing and running 14:09:08 see readme at pcmore 14:09:37 Install and configure Copr client - makes hosting repository 14:10:02 Clone github repository pcmoore/copr-pkg 14:11:19 Create and configure new Copr project directory 14:11:31 Clone the upstream project and add any remotes 14:11:43 Clone the fedora package repository 14:12:33 Welcomes contributions - email, pull requests etc 14:12:55 Then run scripts 14:13:06 * Generate patches from upstream 14:13:34 * Create a patched SRPM and submit to Copr 14:14:15 Read the documentation, more configuration options there 14:15:28 #topic Questions 14:16:08 Want to extend to architectures other than x86_64 14:17:20 Power and ARM also of interest 14:17:41 Could use Koji as well 14:17:53 #endmeeting