16:01:22 <bookwar> #startmeeting Fedora ELN SIG
16:01:22 <zodbot> Meeting started Fri Apr 23 16:01:22 2021 UTC.
16:01:22 <zodbot> This meeting is logged and archived in a public location.
16:01:22 <zodbot> The chair is bookwar. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:22 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:01:22 <zodbot> The meeting name has been set to 'fedora_eln_sig'
16:01:32 <cyberpear> .hi
16:01:33 <zodbot> cyberpear: cyberpear 'James Cassell' <fedoraproject@cyberpear.com>
16:01:35 <bookwar> #topic hello
16:01:41 <michel_slm> .hello salimma
16:01:42 <zodbot> michel_slm: salimma 'Michel Alexandre Salim' <michel@michel-slm.name>
16:01:54 <tdawson> .hi
16:01:55 <zodbot> tdawson: tdawson 'None' <tdawson@redhat.com>
16:02:02 <tstellar> .hello tstellar
16:02:03 <zodbot> tstellar: tstellar 'Tom Stellard' <tstellar@redhat.com>
16:02:14 <bookwar> sgallagh is not joining today, so i am going to try to chair the meeting, and if i am doing smth wrong - let me know :)
16:02:21 <bookwar> .hello2
16:02:22 <zodbot> bookwar: bookwar 'Aleksandra Fedorova' <alpha@bookwar.info>
16:03:06 <bookwar> let's see the topics for today...
16:03:33 <bookwar> If you have something - please name it, and i am looking into github issues
16:03:46 <tdawson> Composes
16:04:03 <tdawson> We're still doing composes every three hours
16:04:14 <tstellar> It's already tagged with meeting, but I would like to discuss: https://github.com/fedora-eln/eln/issues/40
16:04:17 <michel_slm> dcavalca asked me to bring up https://pagure.io/fedora-infrastructure/issue/9730 (rsync)
16:04:22 <tdawson> And, they are currently broken, but the broken'ness doesn't have to be in the meeting.
16:04:52 <bookwar> #topic https://github.com/fedora-eln/eln/issues/37
16:05:02 <bookwar> the first one is simple
16:05:07 <bookwar> #info  SIG Members: Make your org membership public
16:05:32 <bookwar> just a reminder, moving on
16:06:13 <bookwar> #topic Create infrastucture for building alternative composes based on ELN
16:06:14 * michel_slm just realized he's not an official member yet, woops
16:06:21 <bookwar> #link  https://github.com/fedora-eln/eln/issues/40
16:06:42 <bookwar> michel_slm: you can ping sgallagher on the ticket above and he will add you
16:06:54 <bookwar> so let's discuss the idea by tstellar
16:07:16 <bookwar> tstellar: can you maybe introduce it in short?
16:07:23 <tstellar> bookwar: Sure.
16:08:21 <tstellar> tstellar: I would like to make it possible to use ELN as a testing ground for system wide changes.
16:09:29 <bookwar> why do you consider additional koji and not the one we already have?
16:09:41 <tstellar> So I would like to propose finding a way to acquire and setup the infrastructure to do this.
16:10:50 <tstellar> bookwar: It seems less intrusive that way.  We already run into resource constraints on the s390x builders in the Fedora koji, so I don't want to add to that problem.
16:11:18 <tstellar> bookwar: But I'm not really opposed to using the Fedora koji for this.
16:12:13 <tdawson> I'm a little confused about creating new composes on system wide changes.  Will the changes be in a side tag?  Or are you thinking of only compose system wide changes?  Or are you thinking of rebuilding all of ELN with the change and looking at the compose of that?
16:12:34 <bookwar> tstellar: do you have a specific use case in mind? and do you expect resource usage be higher there than it is now in eln?
16:13:02 <tstellar> tdawson: My thought was that it would be like a permantent side-tag that syncs from rawhide like ELN does.
16:13:52 <tstellar> bookwar: The main use case I have in mind is rebuilding ELN with clang.  But I could also using this to test new compiler flags or gcc snapshots.
16:14:01 <bookwar> so you want to setup it next to ELN not apply changes to eln?
16:14:04 <michel_slm> is this different from the proposal to have a way to add more packages to ELN beyond what's slated to be in EL? (e.g. EPELN or ELN extras)
16:14:13 <tstellar> Sorry I mean it would sync from ELN not rawhide.
16:14:57 <tstellar> michel_slm: Yes, this is different.
16:15:11 <bookwar> michel_slm: i think the question here is whether we can experiment on clang directly in the ELN, or we would like to have a separate eln-clang buildroot
16:15:25 <bookwar> setup the same way and with the same content but in parallel
16:16:41 <bookwar> tstellar: i think to move forward with this task we need to gather some data. Using current ELN as example, to estimate how much of koji resources it "eats". And then we can ask Infra about can we double the ELN load in the existing koji
16:16:42 <tdawson> Sortof like a throw away ELN.  This month we are testing clang stuff, next week we are testing a gcc tag, etc..
16:17:50 <bookwar> afaik infra is looking into scalink koji builders in clouds, including multiarch. And scaling koji build farm would be more productive in long-term then copying it.
16:18:31 <tstellar> bookwar: That makes sense.  Any suggestions for how to measure current resource usage?  Should I just count builds per minute or look at build time too?
16:19:11 <bookwar> tstellar: how about we work on collecting the data about ELN load in a separate task? I would start with counting koji messages, and they have the duration too
16:19:22 <bookwar> so we can come up with some aggregated metrics
16:19:30 <tstellar> bookwar: OK
16:19:45 <bookwar> #action bookwar to create a task to measure ELN load on koji
16:20:17 <tstellar> bookwar: I don't mind trying to do that, unless someone else wants to.
16:20:47 <bookwar> tstellar: cool, i'll assign it to you then :)
16:21:01 <tstellar> bookwar: Ok, sounds good.
16:21:07 <bookwar> Let's move to the next topic
16:21:20 <bookwar> #topic rsync and mirroring
16:21:32 <bookwar> #link https://pagure.io/fedora-infrastructure/issue/9730
16:21:54 <bookwar> I am not sure i understand what's going on there
16:22:04 <bookwar> michel_slm: do you have an update?
16:22:23 <michel_slm> dcavalca asked me to bring it up precisely because there's no update :)
16:22:59 <michel_slm> we'd like to be able to mirror the ELN composes, so we can use it internally but also reduce the load on the main server
16:23:34 <tdawson> I thought the first step was for us to change the frequency of compose generation from every 3 hours to once a day.  Then they would think about putting the compose someplace.
16:23:47 <michel_slm> I think the problem is that what's exported is incomplete (dangling symlinks)
16:24:16 <michel_slm> ah. so it's blocked on the other topic that composes are still happening too frequent? yeah, I see some concern about disk usage on the ticket
16:24:55 <smooge> there are several items
16:25:13 <bookwar> #action bookwar ask jkaluza about access to the cron job which triggers ELN composes
16:26:34 <bookwar> smooge: ?
16:26:45 <bookwar> which ones?
16:27:25 <smooge> the key issue is that when I ask if a project if they plan to ever need to make their items available to the general public and they say no.. don't believe them
16:27:52 <tdawson> :)
16:28:14 <smooge> the exports and disk layouts were all planned on 'this will never be something we deliver to people to use'
16:28:51 <michel_slm> yeah, just assume it will be public at some point :) . I remember being asked "but why would you ever want to use this?" :p
16:29:21 <bookwar> well, not available for general public. Doesn't mean that certain specific public can not download it :)
16:29:55 <smooge> well they layouts were really set up for a couple of engineers to do so internally
16:30:10 <michel_slm> ah, that could be an important distinction. once this is resolved, if it's not meant to be public we might not put it on mirror.facebook.net
16:30:11 <smooge> anyway.. spilt milk and me crying
16:31:18 <bookwar> ok, next steps: 1) change the schedule to 1 compose per day 2) discuss if the job which builds composes can also push them out to a dedicated storage
16:31:30 <michel_slm> should we update the ticket with this info?
16:32:10 <bookwar> it seems both are eln tasks rather than infra. So we need to resolve this one https://github.com/fedora-eln/eln/issues/39
16:32:44 <bookwar> #action sgallagher to check https://github.com/fedora-eln/eln/issues/39
16:33:09 <bookwar> the issue here is that odcs automation is not done on the jenkins, so we have very limited access to the actual configuration
16:33:43 <bookwar> we should probably find a way to move to jenkins for everyone to contribute
16:34:07 <bookwar> Anyway, I will discuss it with Jan Kaluza next week, and try to get access to it
16:34:47 <michel_slm> there's this that sounds like 'how to consume ELN', so maybe the issue with missing files can be tracked there? or in a separate ticket
16:34:47 <michel_slm> https://github.com/fedora-eln/eln/issues/33
16:35:49 <bookwar> michel_slm: i think it is the good enough
16:37:11 <bookwar> ok, so we have tickets, we have things to do, we just really need to go and do them :)
16:37:23 <bookwar> let's see if we can manage it by next meeting
16:37:27 <michel_slm> ok, I'll link them both from the infra ticket so people who look at it can see where the action is happening
16:37:33 <bookwar> michel_slm: thanks
16:38:08 <bookwar> let's move to the probably last topic for today
16:38:18 <bookwar> #topic ELN compose status
16:38:52 <bookwar> So compose is failing with lorax templates something something
16:38:57 <bookwar> #link  https://github.com/fedora-eln/eln/issues/43
16:40:01 <bookwar> i tried to untag current lorax package -5, and use the older version -3. But that older version is not signed, so that i can not build compose with it. And with -4 version it still fails
16:40:15 <michel_slm> looks like runtime-install.tmpl is missing from the package?
16:40:50 <tdawson> I can get -3 in if you think that will work.  But I saw that  runtime-install.tmpl was in the package ... at least from the info.
16:40:56 <bookwar> michel_slm: yeah, probably an update in the rawhide, but i haven't dig into the sources yet
16:41:02 <tdawson> I didn't try pulling it out of the package to see if maybe it was corrupt.
16:41:54 <bookwar> tdawson: if you can get -3 version signed, i will trigger compose build with it. And we at least will know whether it helps or not.
16:42:30 <bookwar> we don't need to do it right now
16:43:08 <bookwar> but if you manage to do that, let me know or comment in the ticket, and i will trigger the compose
16:43:48 <bookwar> And for everyone else - feel free to join the debugging. Triggering composes is easy, and i think it is available for everyone in eln-sig
16:44:08 <bookwar> https://docs.fedoraproject.org/en-US/eln/compose/
16:44:23 <bookwar> #topic Open floor
16:44:36 <bookwar> I don't have anything more for today
16:44:38 <michel_slm> so .. tangential to the previous topic of alternate composes
16:45:03 <bookwar> yes, go on
16:45:25 <michel_slm> if we need something like "ELN extras" (e.g. we want to test if packages that are not meant to be in EL, but likely will need to be in EPEL or elsewhere), what's the best way to go about doing it?
16:45:39 <michel_slm> not sure if this is tracked in an issue yet, but it's been discussed in some previous meetings
16:45:40 <tdawson> bookwar: It's signed now, though not in the buildroot yet.
16:46:40 <michel_slm> idea being: 1) same as ELN, make sure packages in Rawhide are EL compatible; 2) for Facebook, we actually want to have ELN + "faux EPEL" consumable for internal testing
16:47:16 <bookwar> michel_slm: i would start with adding it as a separate workload to https://github.com/minimization/content-resolver
16:47:46 <bookwar> we need initial list of packages, which would be small
16:48:07 <bookwar> and then we need to request a tag in koji for building them
16:48:17 <michel_slm> ah, ok. and anyone who needs more added can then just submit it as a PR to this?
16:48:23 <michel_slm> (once we have a new workload)
16:48:52 <bookwar> yes, and it will resolve dependencies and show the full tree to add
16:49:40 <michel_slm> I can create an ELN issue to track this, so people can chime in with suggestions, then work on adding this workload
16:49:53 <michel_slm> oh, we just need the leaf packages we want added? nice
16:50:34 <michel_slm> that might actually solve the "figure out which dependencies we need to branch to make the packages we care about work in EPEL N+1" asa well
16:50:42 <bookwar> michel_slm: yes, let's create a task, and then define some small starting set. And we will be able to manage the list in the content resolver
16:50:44 <tdawson> But would we add the 'eln' label to the workload?
16:51:06 <bookwar> tdawson: no, we don't want it to be eln workload
16:51:55 <tdawson> OK, I agree
16:52:28 <bookwar> afaik you can define additional workloads in the content resolver. It may need couple of adjustments, and we will need to talk with Adam about it, but I would start from the task, and initial PR and then start conversation in it
16:53:22 <michel_slm> so, looks like since my invite expired (oops) I can't file a task yet. filed this to rerequest: https://github.com/fedora-eln/eln/issues/44
16:53:52 <michel_slm> actually, no, that's just a template offered, i can start a new issue
16:54:35 <bookwar> i retried you invite
16:54:45 <bookwar> you should get a new link
16:55:32 <bookwar> ok, let's stop for today. We have some stuff to do for everyone :)
16:55:38 <bookwar> thanks for participation
16:55:54 <tdawson> bookwar: Thanks for stepping up with sgallagh not here.
16:56:06 <michel_slm> bookwar++ thanks!
16:56:06 <zodbot> michel_slm: Karma for bookwar changed to 6 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:56:16 <bookwar> #endmeeting