18:00:45 <phsmoura> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting
18:00:45 <zodbot> Meeting started Tue Aug  9 18:00:45 2022 UTC.
18:00:45 <zodbot> This meeting is logged and archived in a public location.
18:00:45 <zodbot> The chair is phsmoura. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
18:00:45 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:45 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:00:45 <phsmoura> #chair nirik mobrien aheath1992 smooge
18:00:45 <phsmoura> #meetingname fedora_infrastructure_ops_daily_standup_meeting
18:00:45 <phsmoura> #info meeting is 30 minutes MAX. At the end of 30, its stops
18:00:45 <zodbot> Current chairs: aheath1992 mobrien nirik phsmoura smooge
18:00:45 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:00:46 <phsmoura> #info agenda is at https://board.net/p/fedora-infra-daily
18:00:54 <nirik> morning everyone.
18:00:59 <phsmoura> #info reminder: speak up if you want to work on a ticket!
18:01:00 <phsmoura> #topic Tickets needing review
18:01:02 <aheath1992> morning
18:01:06 <phsmoura> morning
18:01:16 <phsmoura> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1
18:01:27 <phsmoura> we dont have infra tickets today
18:01:41 <phsmoura> #info https://pagure.io/releng/issues?status=Open
18:01:55 <smooge> yay
18:01:58 <phsmoura> .releng 10952
18:01:59 <zodbot> phsmoura: Issue #10952: systemd changelog entries for mass rebuild are lacking version numbers - releng - Pagure.io - https://pagure.io/releng/issue/10952
18:02:17 <phsmoura> low,low,ops while waiting for feedback?
18:02:48 <nirik> sure. I suspect it might be a rpmdev-bumpspec bug...
18:03:33 * phsmoura waiting pagure response
18:03:57 <phsmoura> done. those were all the issues
18:04:08 <phsmoura> #topic Planning, Upcoming work and Open floor
18:04:27 <phsmoura> #info Recharge day is on Friday
18:04:42 <smooge> I am looking at an issue https://pagure.io/releng/issue/10909
18:05:13 <smooge> mostly just collecting info and adding it to the ticket. just noticed that the s390x is currently using 90GB of swap
18:05:29 <nirik> yeah... I might have oversubscribed it.. not sure.
18:06:15 <nirik> but not by that much I wouldn't have thought
18:07:19 <nirik> anyhow, I think their change to allow more memory per cpu will help. so we wait and see.
18:07:33 <smooge> it looks like the qemu process is using more memory than it sees internally. so each process is using 19GB but sees 16Gb internal. 19*16 is more than 256
18:08:05 <smooge> but I am not sure that is all the memory usage
18:08:39 <nirik> reminder: I'm gonna be out later today to go to the lovely airport... in about 3 hours.
18:08:47 <smooge> portland airport?
18:09:04 <nirik> yeah, portlandia
18:09:18 <smooge> good luck with the trip. that is what 2 to 3 hours there and 2 to 3 hours back?
18:09:41 <nirik> 2ish each way, longer if there's traffic/accidents. ;(
18:10:21 <smooge> well good luck. I have a copy of the current s390x build log going which will hopefully say where this build is dieing now
18:10:44 <nirik> yeah, it's saying it's 16gb overcommited, not 90...
18:10:47 <nirik> buildvmhost-s390x-01.s390.fedoraproject.org:    -16724.0/261804 mem(unused/total)       0/48 cpus(unused/total)
18:11:10 * nirik shuts down one vm...
18:15:03 <smooge> sorry my 90GB was going over how much the system was in swap while 100% real memory used and looking at the top memory users
18:15:52 <smooge> nirik I wonder if this is doing anything `8873 root      20   0   86.7g   7.9g   5700 S   1.7   3.1 188:42.01 collectd`
18:16:09 <smooge> thats 90GB of collectd ???
18:16:37 <smooge> anyway I think we are out of meeting time
18:17:05 <phsmoura> it lasts 30 min dont worry smooge
18:17:22 <smooge> ah ok.
18:18:08 <smooge> nirik, I see you are on the box so I will not touch any service
18:19:16 <nirik> yeah, collectd can likely be killed. it can't reach log01 anyhow for that I don't think
18:21:05 <phsmoura> is there anything else? Im back working on fedora planet, trying out some things before answer the thread in fedora infra
18:21:48 <nirik> I don't have much else...
18:22:27 <smooge> and that freed up 80GB of ram
18:22:36 <smooge> I have done a systemctl disable collectd
18:22:51 <nirik> hey, I can start that guest back then
18:23:08 <smooge> sorry 80GB of swapspace
18:23:15 <smooge> not 80GB of ram
18:23:36 <smooge> so no.. we were still oversubscribed
18:24:20 <smooge> phsmoura, I don't have anything elser
18:25:41 <phsmoura> ok, lets go back to the battle then :)
18:25:49 <phsmoura> thank you all for attending
18:25:50 <phsmoura> #endmeeting