15:00:03 <asamalik> #startmeeting Minimization Team Meeting
15:00:03 <zodbot> Meeting started Wed Aug 14 15:00:03 2019 UTC.
15:00:03 <zodbot> This meeting is logged and archived in a public location.
15:00:03 <zodbot> The chair is asamalik. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:03 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:03 <zodbot> The meeting name has been set to 'minimization_team_meeting'
15:00:03 <asamalik> #meetingname minimization
15:00:03 <asamalik> #chair asamalik pbrobinson zbyszek feborges Eighth_Doctor lorbus salimma tdawson ignatenkobrain jaruga
15:00:03 <asamalik> #topic Roll call
15:00:04 <zodbot> The meeting name has been set to 'minimization'
15:00:04 <zodbot> Current chairs: Eighth_Doctor asamalik feborges ignatenkobrain jaruga lorbus pbrobinson salimma tdawson zbyszek
15:00:09 <asamalik> .hello2
15:00:10 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
15:00:14 <ignatenkobrain> .hello2
15:00:15 <zodbot> ignatenkobrain: ignatenkobrain 'Igor Gnatenko' <i.gnatenko.brain@gmail.com>
15:00:24 <jaruga> .hello2
15:00:24 <zodbot> jaruga: jaruga 'Jun Aruga' <jaruga@redhat.com>
15:00:28 <asamalik> Hi everyone! Welcome to the very first team meeting!
15:00:34 * pbrobinson is sort of here
15:00:36 <ignatenkobrain> I'll have to disappear soon, but I'll continue reading from the phone
15:00:57 <tdawson> .hello2
15:00:58 <zodbot> tdawson: tdawson 'None' <tdawson@redhat.com>
15:01:13 <lorbus> Hi everyone! Huge thanks to asamalik++ for kicking this off!
15:01:14 <zodbot> lorbus: Karma for asamalik changed to 13 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:01:15 <lorbus> .hello2
15:01:16 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
15:01:47 <asamalik> cookies! \o/
15:03:13 <asamalik> #topic === Agenda ===
15:03:41 <asamalik> so this is the firs one we have and we should decide how it's going to work the best for us
15:03:44 <asamalik> this is my proposal for today:
15:03:48 <asamalik> 1/ Introductions — let's do all do quick introductions, what our goals are, skills and interests, how we want to help
15:03:48 <asamalik> 2/ This meeting — how we want to use it, etc.
15:03:48 <asamalik> 3/ Team operations — let's discuss tracking work, discussions, etc.
15:03:48 <asamalik> 4/ What's next? — let's plan our short-term goals
15:03:48 <asamalik> 5/ Open Floor — Anything else?
15:03:54 <Son_Goku> .hello ngompa
15:03:55 <zodbot> Son_Goku: ngompa 'Neal Gompa' <ngompa13@gmail.com>
15:04:10 <asamalik> #chair Son_Goku
15:04:10 <zodbot> Current chairs: Eighth_Doctor Son_Goku asamalik feborges ignatenkobrain jaruga lorbus pbrobinson salimma tdawson zbyszek
15:04:20 <asamalik> Son_Goku I need to find a system in your IRC nicks! :P welcome
15:04:44 <Son_Goku> asamalik: they're all listed in LP: https://launchpad.net/~ngompa13
15:04:49 <Son_Goku> one day, FAS might let me do the same :)
15:04:59 <jaruga> Agree for the today's agenda.
15:05:11 <tdawson> agenda sounds good.
15:05:25 <ignatenkobrain> Son_Goku: you mean when we will migrate off FAS…? :)
15:05:35 * Son_Goku shrugs
15:05:58 <asamalik> #topic === Introductions ===
15:06:10 <asamalik> So let's just give everyone a minute or so to introduce themselves — what are your goals, interests, skills, how you want to help, maybe how much time you plan to give, etc.
15:06:13 <asamalik> let me start first
15:06:23 <jaruga> Sure.
15:06:34 <asamalik> so I'm the objective lead, and my goal is to build an environment for our maintainers where it's easy for them to keep things minimal
15:06:46 <asamalik> ... and easy to make decisions about size vs. features, seeing the potential impact.
15:07:01 <asamalik> I want to focus on good user experience of whatever tool/service we develop
15:07:14 <asamalik> ... and help you all with as much of the boring tracking/reporting/coordinating stuff as possible!
15:07:16 <asamalik> EOF
15:07:31 <ignatenkobrain> I'm Igor and I'm member of FPC and FESCo. Provenpackager and Sponsor. I'm here to help with whatever help you need related to the dependency solving (since I'm one of upstream libsolv maintainers). I'm also interested to get stuff more automated and less error-prone.
15:07:53 <pbrobinson> I'm pbrobinson, I'm the IoT lead and the smaller the better in that context, playing whack-a-mole on dep bloat is one of the first things I ever did as an active contributor in Fedora and a game I've played ever since
15:09:10 <Son_Goku> I'm Neal and I'm just a guy who happens to be everywhere. :) I'm here to help with being mindful about how we manage the dependency tree, including through automation. I want to help people get a better context of how dependencies work and their impact throughout the system, so that people make better decisions about how to package and how dependencies are made and handled.
15:09:49 <Son_Goku> I'm usually on IRC as Pharaoh_Atem, King_InuYasha, Son_Goku, Eighth_Doctor, etc.
15:09:55 <Son_Goku> depending on my machine ;)
15:10:12 <jaruga> I am Jun Aruga. I am interested in small sized Fedora container image and I am working for a multi architecture project.
15:10:18 <lorbus> I'm Christian, and I on the machine-config-operator team in the OpenShift org at Red Hat nowadays, and I've been involved with Fedora IoT WG and Container SIG, and Atomic/CoreOS WG for quite some time now. Reducing the base OS size is important for me in all of those contexts
15:10:35 <tdawson> I'm Troy Dawson.  Provenpackager, like to build OS's.  I also like making my servers as small as possible, while keeping security and features.  I'm not the best at automation or making good web pages, but I'm pretty good with rpms and getting things small.  Objective.  Get parts of Fedora as small as possible and try to translate that to RHEL9/10.
15:11:54 <asamalik> Thanks all for the intros! (I think that was everyone who's here, we can continue with others next time)
15:13:00 <asamalik> #topic === This meeting ===
15:13:21 <asamalik> so I think this meeting will be a useful place to sync in real time when there's everyone. I don't want it to be a status report meeting, but rather a forward-looking discussion.
15:13:32 <asamalik> We can talk about ideas, interesting discoveries, plans, etc. What other people think?
15:14:42 <lorbus> sgtm!
15:15:57 <lorbus> I have a few (naive?) ideas and like to gather thoughts and feedback on them :)
15:16:35 <asamalik> lorbus: cool! so let's get to that
15:16:37 <jaruga> A discovery: I noticed a person used Alpine Linux container as their container's demo. That means small sized container image is good for demo.
15:17:01 <jaruga> At flock 2019 event.
15:17:52 <asamalik> #topic === Team operations ===
15:18:25 <asamalik> so I think everyone has seen the Team page, right? https://docs.fedoraproject.org/en-US/minimization/team/
15:18:39 <jaruga> Yes, I saw it.
15:18:52 <asamalik> #info we have a bug tracker to track the actual package changes: https://bugzilla.redhat.com/show_bug.cgi?id=1734342
15:19:06 <asamalik> and also
15:19:21 <asamalik> #info we have a team issue tracker for discussion and tracking work: https://pagure.io/minimization/issues
15:20:12 <asamalik> so my idea was that if someone is working on something bigger, they could create an issue here so we all know what's going
15:21:06 * asamalik should probably create a ticket there to track to graph tool development
15:21:22 <jaruga> Agree.
15:21:46 <asamalik> although we'll figure tracking out as we go I guess... but the BZ might be handy for specific stuff
15:22:09 <asamalik> ok, let's get to the real things!
15:22:11 <asamalik> #topic === What's next? ===
15:22:32 <tdawson> Do we have a place that we can put results?  Such as "Adding glibc-minimal-langpack to your list of packages to install, can drop your container images size by 220M"
15:22:40 <asamalik> lorbus: time for your ideas!
15:23:26 <lorbus> both ideas are mostly focused on the container usecase (one more than the other). I know this isn't the focus, but as it's got to do with minimization, I'll just go ahead
15:23:28 <asamalik> tdawson: ha! that's a good question... I'd like to collect these "random tips" probably in the docs, maybe building some structure over time when there's a lot of other stuff
15:24:28 <lorbus> First: I'd like to remove the dependency on systemd from a couple of RPMs (I think there is an effort for this going on right now, too)
15:24:39 <asamalik> tdawson: let me create a Discoveries section in the docs, so we can start putting those there and I'll think about the next steps
15:24:49 <tdawson> asamalik: OK, sounds good.
15:24:52 <lorbus> This would enable building containers without systemd
15:25:36 <lorbus> thinking mostly server programs like Postfix, Dovecot, Coturn, Prosody etc
15:25:37 <tdawson> lorbus: Are you talking about this issue? - https://pagure.io/minimization/issue/2
15:25:51 <asamalik> lorbus: I like that! There is a ticket from ignatenkobrain about that I think, and zbyszek mentioned a Packaging Guidlines update about that as well
15:26:34 <asamalik> tdawson: that one, yes!
15:26:41 <lorbus> yes, it's essentially that. Zbigniew and Igor put it better than me :)
15:27:31 <lorbus> The second idea is somewhat similar: I would like for Fedora to enable building containers that do not ship a package manager
15:28:20 <dftxbs3e> lorbus, sorry I'm not a member, but I think it's important to comment this with my findings: python3.7 which is a dependency to dnf, takes up 50 megabytes in the current fedora:latest container image.
15:28:26 <asamalik> #idea remove systemd dependency from various server programs like Postfix, Dovecot, Coturn, Prosody etc — enables building smaller container images
15:28:33 <lorbus> (this is obviously doable with dnf and buildah, but I'd at least like to create documentation around that)
15:28:40 <asamalik> dftxbs3e: you don't need to be a formal member to contribute! you're very welcome here :)
15:28:51 <asamalik> dftxbs3e: you're also welcome to join btw :)
15:29:27 <jaruga> lorbus, why do you like shipping a container without a package manager? What is the use case?
15:29:38 <lorbus> dftxbs3e: exactly, dnf pulls in a lot, and I dont need it when running the container, nor the smaller microdnf
15:29:39 <asamalik> lorbus: yes that would be useful, too! about that, there might be some minimization / dependency optimization work to be done because when I was playing with those, I got some hardware-related stuff pulled in
15:29:48 <asamalik> lorbus: https://asamalik.fedorapeople.org/container-randomness/report.html
15:29:53 <asamalik> although that report is quite old...
15:30:10 <dftxbs3e> I've been fiddling with the official container image as we speak and what takes most the size is: gconv and python3.7. (/usr/lib64/python3.7 and /usr/lib64/gconv), gconv contains shared objects for locales support I believe, and python3.7 contains all the packages that are python, so actually not just python.
15:30:22 <asamalik> I should think about a way of making that report at an early stage than the actual published images
15:30:23 <lorbus> jaruga: e.g for running the container on openshift. I'd like a pre-built container, mount my config in it, and let it run
15:30:56 <tdawson> If you a) don't want to layer containers (ie.  what the container has is what you need) or b) just want to add configs ... then not having a package manager in your container is good.
15:31:00 <jaruga> lorbus, I see. Makes sense.
15:31:17 <lorbus> also less attack surface as a neat side effect
15:32:01 <asamalik> jaruga: you can build them like that for example: https://github.com/asamalik/container-randomness/blob/master/f30/scratch-fedora-httpd/Dockerfile
15:32:16 <tdawson> But for a general purpose container, some type of package manager is needed ... but I thought that was why we have microdnf.
15:32:33 <jaruga> asamalik, Yes I can imagine that. Thanks for the info.
15:32:54 <lorbus> asamalik: I saw that, I was wondering why all the scratch builds more than doubled in size in F31?
15:33:20 <asamalik> lorbus: I have no idea! we could look into that
15:34:00 <lorbus> tdawson: you summarized that perfectly :) of course there should always be a multi-prupose container that does have a pacman
15:34:00 <jaruga> lorbus, asamalik we can check the size by "du" command can't we?
15:34:07 <asamalik> #idea making sure that installing server apps in an empty root (like scratch container images) doesn't result in unneeded things being pulled in
15:34:21 <dftxbs3e> Another suggestion: one could decide to build container images with all of Fedora's packages rebuilt with -Os for overall smaller machine code.
15:34:43 <jaruga> lorbus, asamalik I mean to check which RPM package or files affect the file size.
15:34:47 <asamalik> jaruga: so I can list all the packages and their sizes... it will be probably caused by some dependencies being pulled in
15:35:01 <asamalik> so I shouldn't have said "no idea" but rather "not sure what exactly got pulled in and why"
15:35:21 <asamalik> jaruga: right
15:35:45 <jaruga> Regarding dftxbs3e 's comment python3.7 takes 50 megabytes.
15:36:11 <asamalik> dftxbs3e: building alternative variants of packages could be also an option, yes, if there is a demand for it
15:36:53 <asamalik> so I have an idea on how to approach this
15:37:30 <jaruga> asamalik, we need "I can list all the packages and their sizes" to analyze current state.
15:38:05 <asamalik> 1/ Identify apps / use cases we want to optimize. That could be "httpd" or "the virt stack".
15:38:37 <asamalik> 2/ Look at those installed on top of the Fedora container base image, the minimal install, and/or an empty directory and try to understand what's there and why
15:39:47 <asamalik> 3/ Find specific things to cut off... there might be Perl pulled in because of a single script (pbrobinson mentioned something doing this at Flock), there might be dependencies that are there by error, or DNF might be choosing a suboptimal RPM when there are choices in the repo, etc...
15:40:25 <asamalik> jaruga: for initial view you can use the "showme" tool: https://pagure.io/minimization/dependency-visualiser
15:40:51 <jaruga> asamalik, yeah. You created the great app.
15:41:20 <asamalik> it could be definitely improved a lot — different outputs, looking at repos to suggest alternatives, etc.
15:43:10 <asamalik> jaruga: thanks! glad you like it
15:43:42 <tdawson> Just did a quick test.  Using microdnf (which doesn't pull in python or systemd) created a 100M smaller container than plan dnf.
15:43:59 <jaruga> I think we also want a kind of CI to measure the total file size. To measure our activity. That was pbrobinson 's suggestion.
15:44:23 <asamalik> tdawson: that's nice, yeah!
15:44:55 <jaruga> tdawson, good finding!
15:45:01 <asamalik> since we're in the discovery phase, looking at things and doing random discoveries is what we should do — but we need to think about capturing those in a useful way
15:45:04 <lorbus> tdawson: I heard microdnf is not as stable as dnf and has some general shortcomings
15:45:08 <asamalik> I'll take an action to figure it out
15:45:37 <lorbus> and I don't think there is anyone dedicated to developing it
15:45:44 <tdawson> lorbus: possible ... I don't use it much.
15:45:44 <lorbus> (might be wrong here)
15:46:06 <asamalik> #action asamalik figure out how can we capture (random) discoveries in a useful way
15:46:15 <lorbus> I think nobody does/very few do, and that's why :/
15:46:48 <asamalik> lorbus: I think it's the DNF team maintaining it
15:47:37 <asamalik> is the "fedora-minimal" container image basically that? just including microdnf?
15:47:43 <lorbus> yes
15:47:52 <dftxbs3e> If we're looking into minimizing for the containers use case, then what helps it the most is efficient usage of layers more so than individual package size minimization.
15:48:03 <jaruga> I tried to use microdnf in past. But I had no idea about how to use. I like to see a blog article about it.
15:48:29 <jaruga> asamalik, yes fedora-minimal includes microdnf without dnf.
15:48:38 <asamalik> dftxbs3e: both are important... layers for reusing, but optimized packages that don't pull things you don't really need are also important
15:49:18 <asamalik> dftxbs3e: but it's the overall size that matters, definitely, not just a base image or a single container image
15:49:26 <asamalik> OK, we have 10 minutes!
15:50:18 <dftxbs3e> asamalik, yes, creating intermediate images for public usage would help the eco system waste less disk size instead of every of them importing fedora:latest and building upon it.
15:50:27 <asamalik> #action asamalik extend the Action Plan with specific use cases / stacks to focus on
15:50:35 <asamalik> anything specific people want to work on over the next week?
15:51:11 <asamalik> dftxbs3e: there is a Container Team in Fedora focusing exactly on this! I think cverna would tell you more
15:51:57 <jaruga> asamalik, I am interested in checking how to use microdnf. Because fedora-minimal container is more useful than fedora container when it is used on CI.
15:51:58 <dftxbs3e> asamalik, Thanks, noted.
15:52:04 <asamalik> dftxbs3e: they focus on the actual images and what goes into what layer.. and here we can help by optimizing the packages themselves so it's possible to build small layers/images
15:52:15 <jaruga> asamalik, For example, I am trying to run fedora containers on Travis CI. https://github.com/junaruga/fedora-workshop-multiarch/blob/master/.travis.yml
15:52:32 <lorbus> asamalik: I can't dedicate much time in the short term unfortunately :( happy to test anything you come up with, tough!
15:52:34 <jaruga> asamalik, But next week, I am in vacation :p
15:53:32 <asamalik> lorbus: no worries, and I might be sending you stuff to test, then :)
15:53:43 <lorbus> please do! :)
15:53:43 <cverna> yes layered container in Fedora did not really picked up, people are not really interested in maintaining them or consuming them
15:53:51 <dftxbs3e> I think what would get things started would be to write a bot that comments pull requests on src.fedoraproject.org with a report on package size, to spot the obvious cases.
15:54:15 <cverna> it is difficult to sell why someone should use postgresql from fedora registry rather than docker hub
15:54:30 <asamalik> dftxbs3e: yes something like this would be quite useful and it's my long-term goal
15:54:55 <dftxbs3e> asamalik, do you have access to Fedora's CI infrastructure? I'm guessing this is pretty easy to do.
15:54:57 <asamalik> dftxbs3e: not sure how easy that would be
15:55:16 <asamalik> dftxbs3e: haven't got time to play with the CI infra, yet
15:55:17 <jaruga> cverna, docker hub is faster to download a image than fedora official repository.
15:55:31 <dftxbs3e> Starting from: https://pagure.io/minimization/dependency-visualiser -- it is very easy.
15:55:32 <asamalik> dftxbs3e: do you want to look into that?
15:56:00 <lorbus> jaruga, cverna: we should really move the fedora registry over to quay IMO
15:56:04 <dftxbs3e> asamalik, Yes, I can look into it, though, I cannot guarantee anything.
15:56:13 <jaruga> lorbus, right.
15:56:33 <dftxbs3e> cverna, I havent heard about intermediate images anywhere, maybe Fedora Magazine should advertise them?
15:56:39 <lorbus> let's hope we open source quay some day :P
15:56:45 <cverna> lorbus: yes we are still waiting for OCI support for multi arch and flatpak
15:57:02 <lorbus> ack. thanks for the update on that!
15:57:18 <cverna> dftxbs3e: I would prefer not to advertise them since they are not maintained
15:57:26 <asamalik> dftxbs3e: cool!
15:57:46 <asamalik> all right!
15:57:46 <dftxbs3e> anyone knows if pagure has an API for posting comments?
15:57:52 <cverna> yes
15:57:54 <dftxbs3e> even better, a library for interacting with that API?
15:58:38 <cverna> dftxbs3e: https://github.com/packit-service/ogr
15:58:55 <dftxbs3e> Awesome. Thanks.
15:59:01 <cverna> dftxbs3e: but remember that very little package updates happen via pull requests
15:59:14 <asamalik> I think that's all what we have time for today, so let's move to #fedora-devel if there's more to discuss :)
15:59:16 <cverna> most packager directly push their changes
15:59:27 <asamalik> cverna dftxbs3e yeah reacting on builds might be easier to do
15:59:46 <jaruga> asamalik, sure. thanks for organizing the meeting.
16:00:01 <dftxbs3e> Thanks!
16:00:09 <cverna> asamalik: or enforcing PR workflow but that's a different can of worms :P
16:00:26 <asamalik> Thanks all for coming, I'll update the docs, and let's talk on #fedora-devel or in the tickets :)
16:00:35 <asamalik> #endmeeting