#meeting-3:fedoraproject.org log

<@james:fedora.im>

19:04:18

!startmeeting Fedora Infrastructure Ops Daily Standup Meeting

<@meetbot:fedora.im>

19:04:19

Meeting started at 2024-10-22 19:04:18 UTC

<@meetbot:fedora.im>

19:04:19

The Meeting name is 'Fedora Infrastructure Ops Daily Standup Meeting'

<@james:fedora.im>

19:04:22

!chair nirik smooge phsmoura

<@james:fedora.im>

19:04:25

!meetingname fedora_infrastructure_ops_daily_standup_meeting

<@meetbot:fedora.im>

19:04:26

The Meeting Name is now fedora_infrastructure_ops_daily_standup_meeting

<@phsmoura:fedora.im>

19:04:27

hello

<@james:fedora.im>

19:04:28

!info meeting is 30 minutes MAX. At the end of 30, its stops

<@james:fedora.im>

19:04:32

!info agenda is at https://board.net/p/fedora-infra-daily

<@james:fedora.im>

19:04:36

!info reminder: speak up if you want to work on a ticket!

<@james:fedora.im>

19:04:42

!hi

<@zodbot:fedora.im>

19:04:43

James Antill (james)

<@james:fedora.im>

19:04:47

!topic Tickets needing review

<@james:fedora.im>

19:04:54

!info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1

<@james:fedora.im>

19:05:15

!info https://pagure.io/releng/issues?status=Open

<@nirik:matrix.scrye.com>

19:05:28

morning

<@james:fedora.im>

19:05:41

.releng 12412

<@james:fedora.im>

19:05:46

!releng #12412

<@james:fedora.im>

19:05:54

!releng 12412

<@zodbot:fedora.im>

19:05:54

● **Assignee:** patrikp

<@zodbot:fedora.im>

19:05:54

**releng #12412** (https://pagure.io/releng/issue/12412):** Unretirement request: rust-leb128**

<@zodbot:fedora.im>

19:05:54

<@zodbot:fedora.im>

19:05:54

● **Opened:** 6 hours ago by decathorpe

<@zodbot:fedora.im>

19:05:54

● **Last Updated:** 6 hours ago

<@nirik:matrix.scrye.com>

19:06:03

low low...

<@james:fedora.im>

19:06:20

!releng 12413

<@zodbot:fedora.im>

19:06:21

● **Last Updated:** Never

<@zodbot:fedora.im>

19:06:21

**releng #12413** (https://pagure.io/releng/issue/12413):**Stalled EPEL package: adobe-source-code-pro-fonts**

<@zodbot:fedora.im>

19:06:21

<@zodbot:fedora.im>

19:06:21

● **Opened:** 2 hours ago by dcavalca

<@zodbot:fedora.im>

19:06:21

● **Assignee:** Not Assigned

<@nirik:matrix.scrye.com>

19:06:47

low low also

<@james:fedora.im>

19:07:16

!releng 12398

<@zodbot:fedora.im>

19:07:20

● **Assignee:** Not Assigned

<@zodbot:fedora.im>

19:07:20

**releng #12398** (https://pagure.io/releng/issue/12398):**Make ppc64le Server_KVM image build not take two hours**

<@zodbot:fedora.im>

19:07:20

<@zodbot:fedora.im>

19:07:20

● **Opened:** 5 days ago by adamwill

<@zodbot:fedora.im>

19:07:20

● **Last Updated:** 20 hours ago

<@nirik:matrix.scrye.com>

19:07:37

med med I suppose? there's some attempts

<@james:fedora.im>

19:08:53

!topic Planning, Upcoming work and Open floor

<@james:fedora.im>

19:09:36

I was wondering ... when we (or more accurately, you ;) reinstall a machine ... do we delete or archive the old logs on log01?

<@nirik:matrix.scrye.com>

19:10:20

nope... we do archive off all the hosts logs yearly... but I havent' been deleting or doing anything with old logs.

<@nirik:matrix.scrye.com>

19:10:33

I'm a bit of a packrat sadly... I like keeping things. ;)

<@nirik:matrix.scrye.com>

19:11:01

but yeah, log01 has been alerting for space the last few days... I looked at it and found some builders with fs errors, but I am not sure thats the main reason.

<@james:fedora.im>

19:11:10

Yeh, but we could keep some of the older ones somewhere else ... like on a part of the netapp or something?

<@nirik:matrix.scrye.com>

19:11:23

so if others would like to look and see if they can find other problem applications/hosts that would be great

<@nirik:matrix.scrye.com>

19:11:30

sure, we could

<@james:fedora.im>

19:12:07

Had a quick look, and "proxy*" seemed like something near the top of the list ... but that's not new, or a bug, AFAICS.

<@nirik:matrix.scrye.com>

19:12:31

There's a lot of messages from fedocal and fmn... but not sure it's more than normal.

<@nirik:matrix.scrye.com>

19:12:39

it's kinda hard to sort through

<@james:fedora.im>

19:13:04

```

<@james:fedora.im>

19:13:04

``` 85G ┣━ proxy101.iad2.fedoraproject.org/2024

<@james:fedora.im>

19:13:04

┃ ┖┄ [165,787 file(s)]

<@james:fedora.im>

19:13:04

85G ┣━ proxy04.vpn.fedoraproject.org/2024

<@james:fedora.im>

19:13:04

┃ ┖┄ [153,719 file(s)]

<@james:fedora.im>

19:13:04

88G ┣━ proxy110.iad2.fedoraproject.org/2024

<@james:fedora.im>

19:13:04

┃ ┖┄ [165,647 file(s)]

<@james:fedora.im>

19:13:04

164G ┣━ proxy10.iad2.fedoraproject.org/2024

<@james:fedora.im>

19:13:04

┃ ┖┄ [169,637 file(s)]

<@james:fedora.im>

19:13:04

167G ┗━ proxy01.iad2.fedoraproject.org/2024

<@james:fedora.im>

19:13:27

Those are hosts > 75GB.

<@nirik:matrix.scrye.com>

19:13:59

yeah... I was looking at /var/log/merged/ as it seems like the daily messages.log gets really big until it's compressed, so it alerts and then recovers.

<@nirik:matrix.scrye.com>

19:14:16

It could be particular hosts tho perhaps yeah

<@james:fedora.im>

19:15:30

Yeh, merged is like 12GB ... and hosts is 1.5TB :)

<@nirik:matrix.scrye.com>

19:17:05

so yeah, probibly we should just look at archiving the last 6 months or something?

<@nirik:matrix.scrye.com>

19:17:13

the oldest 6 months

<@james:fedora.im>

19:17:54

Yeh, I think if we just archived Jan-Jul for proxies we'd be fine. But maybe easier to do it for everyone.

<@nirik:matrix.scrye.com>

19:18:27

can you open a ticket on it? we can figure out the best plan and do it... I don't know it's super urgent, but the nightly alerts are anoying. ;)

<@james:fedora.im>

19:18:45

Sure, med/med ?

<@nirik:matrix.scrye.com>

19:18:52

sure

<@nirik:matrix.scrye.com>

19:20:18

Anything else ?

<@nirik:matrix.scrye.com>

19:20:58

Hoping that the current RC will last and we will have release next week. ;)

<@nirik:matrix.scrye.com>

19:21:00

we will see.

<@phsmoura:fedora.im>

19:22:48

I have some questions but will sent in internal channel since the doc is internal

<@nirik:matrix.scrye.com>

19:23:04

ah yeah, I plan to put that somewhere today...

<@nirik:matrix.scrye.com>

19:23:41

will do that after lunch. ;)

<@james:fedora.im>

19:24:14

Also ... can you send me the iptables script in private

<@nirik:matrix.scrye.com>

19:24:28

yeah

<@nirik:matrix.scrye.com>

19:25:44

I don't think I have anything else...

<@james:fedora.im>

19:27:20

Me either.

<@james:fedora.im>

19:28:02

!endmeeting