16:00:15 <eddiejennings> #startmeeting Infrastructure (2022-06-09)
16:00:15 <zodbot> Meeting started Thu Jun  9 16:00:15 2022 UTC.
16:00:15 <zodbot> This meeting is logged and archived in a public location.
16:00:15 <zodbot> The chair is eddiejennings. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:00:15 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:15 <zodbot> The meeting name has been set to 'infrastructure_(2022-06-09)'
16:00:38 <eddiejennings> #meetingname infrastructure
16:00:38 <zodbot> The meeting name has been set to 'infrastructure'
16:00:38 <eddiejennings> #chair nirik siddharthvipul mobrien zlopez bodanel dtometzki jnsamyak computerkid
16:00:38 <eddiejennings> #info Agenda is at: https://board.net/p/fedora-infra
16:00:38 <eddiejennings> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:00:38 <zodbot> Current chairs: bodanel computerkid dtometzki eddiejennings jnsamyak mobrien nirik siddharthvipul zlopez
16:00:40 <eddiejennings> #info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
16:00:42 <eddiejennings> #topic greetings!
16:01:05 <eddiejennings> Hey folks!  My apologies for not doing the necessary reminders :(
16:01:07 <eddiejennings> .hi
16:01:08 <zodbot> eddiejennings: eddiejennings 'Eddie Jennings' <eddie@eddiejennings.net>
16:01:21 <nikolay__> Hi
16:01:28 <eddiejennings> How is everyone today! :D
16:01:31 <eddiejennings> ? rather :)
16:02:01 <nikolay__> ok:)
16:02:02 <nirik> morning
16:03:25 <eddiejennings> #topic New folks introductions
16:03:26 <eddiejennings> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:03:26 <eddiejennings> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:03:35 <eddiejennings> Do we have any new folks with us today?
16:03:47 <eddiejennings> If so, let yourself be known, and introduce yourself :)
16:04:28 <darknao> .hi
16:04:29 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:05:04 <phsmoura> morning
16:05:59 <eddiejennings> Seems like we don't have any new folks this morning.  So let's keep moving :)
16:06:02 <dtometzki> hi together
16:06:14 <mkonecny> .hello zlopez
16:06:15 <zodbot> mkonecny: zlopez 'Michal Konecny' <michal.konecny@psmail.xyz>
16:06:28 <eddiejennings> #topic Next chair
16:06:28 <eddiejennings> #info magic eight ball says:
16:06:28 <eddiejennings> #info chair 2022-06-09 - eddiejennings
16:06:28 <eddiejennings> #info chair 2022-06-16 - mkonecny
16:06:42 <eddiejennings> So we are covered for next week thanks to mkonecny :)
16:06:57 <eddiejennings> Any volunteers for 2022-06-23?
16:07:01 <zlopez> It seems that the matrix bridge doesn't work
16:07:10 <dtometzki> i can do 06-23
16:07:27 <eddiejennings> If you're new to the project, this is a fun way to get involved.  You follow the script from our board.net agenda :)
16:07:41 <nirik> Yeah, the bridge is really behind again. ;(
16:07:45 <eddiejennings> 6-23 sold to dtometzki
16:07:51 <dtometzki> thanks
16:07:54 <eddiejennings> bridge = sad panda :)
16:08:37 <eddiejennings> Any takers for 6-30 to round out June?
16:10:01 <eddiejennings> If no takes for 6-30 right now, that's fine.  We're covered for two weeks :)
16:10:22 <eddiejennings> Let's move along.
16:10:39 <eddiejennings> #topic announcements and information
16:10:53 <eddiejennings> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:10:54 <eddiejennings> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:10:54 <eddiejennings> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request
16:10:54 <eddiejennings> #info please help us with improving contribution to fedora infra https://discussion.fedoraproject.org/t/improving-contribution-to-fedora-infrastructure/38294/8
16:10:56 <eddiejennings> #info oncall should also handling #fedora-releng pings if possible
16:10:58 <eddiejennings> #info the-new-hotness 1.2.0 is released. See changelog at https://github.com/fedora-infra/the-new-hotness/releases/tag/1.2.0
16:11:01 <eddiejennings> #info Map of the critical services in Fedora Infra is now available at https://docs.fedoraproject.org/en-US/infra/map_critical_services/
16:11:04 <eddiejennings> #info Fedora annual contributor survey is now available https://fedoraproject.limequery.com/2022
16:11:14 <nirik> #info tomorrow is a Red Hat "day of learning" so many RH employee's will be away learning something...
16:11:19 <petebuffon[m]> .hello petebuffon
16:11:20 <zodbot> petebuffon[m]: petebuffon 'Peter Buffon' <pabuffon@gmail.com>
16:11:25 <zlopez> I forgot to remove my entries from last week
16:11:45 <eddiejennings> I've got the survey opened now in another tab, and will take it after the meeting.  I encourage everyone to take the survey.
16:11:56 <nirik> oh two more things:
16:12:20 <nirik> #info elections are open for voting! Please vote today: https://elections.fedoraproject.org/
16:12:44 <eddiejennings> added
16:12:52 <eddiejennings> zlopez, which one needs to be removed?
16:12:52 <nirik> #info Call for proposals for Nest open: https://communityblog.fedoraproject.org/nest-with-fedora-fedora-hatch-announcing-dates-call-for-volunteers/
16:13:16 <eddiejennings> added
16:13:37 <zlopez> eddiejennings: the-new-hotness and map of critical apps
16:13:54 <eddiejennings> done :)
16:14:14 <nirik> oh another one:
16:14:41 <eddiejennings> I thought the line was "but wait, there's more!"  Of if you're of the Apple-ilk, "oh, and one more thing" :P
16:14:53 <nirik> #info bodhi updated
16:15:11 <mobrien> .hi
16:15:12 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com>
16:15:32 <eddiejennings> Lots of things going on :D
16:15:47 <eddiejennings> Any other announcements?
16:17:15 <eddiejennings> Hearing none, let's move on!
16:17:36 <eddiejennings> #topic Oncall
16:17:37 <eddiejennings> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:17:37 <eddiejennings> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:17:37 <eddiejennings> ## .oncalltakeeu .oncalltakeus
16:17:37 <eddiejennings> #info nirik on call from 2022-06-03 to 2022-06-08
16:17:39 <eddiejennings> #info petebuffon on call from 2022-06-09 to 2022-06-15
16:17:41 <eddiejennings> #info eddiejennings on call from 2022-06-16 to 2022-06-22
16:17:53 <mobrien> Is zodbot dead?
16:18:06 <nirik> no?
16:18:30 <eddiejennings> Do we have any takers for rounding out the month of June?  2022-06-23 to 2022-06-30 (fixed my info, so my on-call ends on 6-23)
16:18:46 <nirik[m]> the bridge is really really slow
16:18:46 <eddiejennings> zodbot seems healthy to me.
16:19:24 <eddiejennings> How far behind is the bridge?  I can slow down to help it catch up
16:19:41 <nirik> eddiejennings: its still at the start of the meeting
16:19:57 <eddiejennings> Wow :(
16:20:02 <nirik> so, no, just go on.
16:20:47 <eddiejennings> Looks like no takers right now for the last week of June.  That's ok for now, since we're covered for two weeks.
16:21:09 <eddiejennings> #info Summary of last week: (from current oncall )
16:21:19 <eddiejennings> nirik, take it away!
16:21:27 <nirik> so it's been really quiet... I don't think there were any oncall pings.
16:21:40 <nirik> (well, quiet for pings, otherwise it's been super busy)
16:21:47 <eddiejennings> Ha!
16:21:55 <eddiejennings> Well, that's good in some sense :)
16:22:04 <eddiejennings> Let's move to part two of nirik :)
16:22:06 <eddiejennings> #topic Monitoring discussion [nirik]
16:22:06 <eddiejennings> #info https://nagios.fedoraproject.org/nagios
16:22:06 <eddiejennings> #info Go over existing out items and fix
16:22:27 <nirik> lets see.
16:23:02 <nirik> 1 mgmt interface showing down (but it's not really, it's just not reachable from outside it's network)
16:23:46 <nirik> just some random small alerts we never seem to get around to cleaning up.
16:24:00 <nirik> thats all, can move on.
16:24:15 <eddiejennings> Noted.  Thank you for keeping us functioning :D
16:24:38 <eddiejennings> And now part 3 of nirik :)
16:24:41 <eddiejennings> #topic Learning topic
16:24:59 <eddiejennings> You were on the schedule to talk about Zabbix today.  Is that still a go?
16:25:02 <nirik> well, except...
16:25:16 <nirik> I have had 0 time to work on this. ;( So I'd like to postpone...
16:25:28 <eddiejennings> I can empathize with the 0 time.
16:26:07 <eddiejennings> Shall we look at older tickets, or have a general open floor discussion and maybe end a bit early?
16:26:18 <nirik> perhaps move it out to the 23rd? I can try again then...
16:26:46 <eddiejennings> Sure.  That gives me more time to prepare btrfs content :D
16:26:54 <nirik> general open floor and ending early sounds good to me.
16:27:36 <eddiejennings> I concur. :)
16:27:37 <eddiejennings> #topic Open Floor
16:28:23 <eddiejennings> So, I've been in the new job for about a month, and thankfully most of the things I touch are now RHEL.
16:28:39 <eddiejennings> One thing that has changed my life, which is probably well known to many of you is: systemctl cat name_of_your_service
16:28:47 <eddiejennings> If you've never used that, try it sometime. :D
16:30:01 <zlopez> I know systemctl, but didn't knew cat command
16:30:50 <nirik> fun tip: systemctl can figure out what service a process belongs to for you... so 'systemctl status <pid>' will show you the status of that processes service. ;)
16:30:55 <eddiejennings> Neither did I.  I can't remember where I saw it, but it was found when I was troubleshooting something else, and I don't know how I lived without it.  So easy to do a quick check of how a unit is configured.
16:31:00 <zlopez> And systemctl edit is useful as well
16:31:21 <zlopez> systemctl status is what I'm using most of the time
16:31:38 <eddiejennings> ooo.  I didn't know you could feed a process id to systemctl status
16:31:56 <zlopez> Really useful is journalctl -f -u <service_name> as well, which will just follow logs of specific unit
16:32:44 <nirik> one I use all the time: journalctl -b -1 ...thats show the journal from the _previous_ boot... so you can see what happened to cause a reboot or lockup easily.
16:33:13 <eddiejennings> This last month has made me appreciate how Fedora Infra uses Ansible.  Most of our Ansible uses are either ad-hoc commands for shell, or playbooks that run the shell command.
16:33:20 <darknao> systemctl edit <service> : if you need to create override very quickly
16:33:23 <eddiejennings> Rather than using Ansible to enforce a configuration standard :(
16:33:53 <eddiejennings> On journalctl, I've found combining --since and --until to be very useful.
16:33:55 <zlopez> nirik: That one is awesome, helped me so many times
16:34:30 <eddiejennings> I'll have to give -b -1 a try with journalctl
16:34:47 <nirik> systemctl enable service --now is nice to save typing....
16:35:18 <eddiejennings> That's a good one as well
16:36:08 <zlopez> nirik: Doesn't enable starts the service as well?
16:36:29 <nirik> zlopez: nope, just enable for next boot
16:36:48 <nirik> but --now makes it also start it
16:38:16 <eddiejennings> Training question:  I know there are many 3rd party resources for RHCSA and RHCE exam prep (as I used them for my exams).  Does anyone know of resources outside of Red Hat Leaning for the various exams that eventually lead to RHCA?
16:39:42 <nirik> no idea off hand
16:40:00 <eddiejennings> Example would be this exam  Red Hat Certified Specialist in Linux Diagnostics and Troubleshooting exam (EX342)
16:40:35 <eddiejennings> Yeah.  Everything I've seen is focused on RHCSA and RHCE, which makes sense.  I'm not opposed to one day paying for Red Hat Learning, but it's a non-trivial expense :(
16:41:27 <phsmoura> I dont know resources outside RH, but for RHCA I think the person should have 5 certs
16:41:41 * nirik nods
16:41:42 <phsmoura> at least
16:41:51 <eddiejennings> phsmoura, correct, and there's a plethora of certs from which to choose
16:42:29 <eddiejennings> I'm of the mind of chasing the skills rather than the cert.  The certs provide a structured curriculum to follow to learn the skills.
16:42:37 <phsmoura> eddiejennings: yes, ah that reminds
16:42:41 <eddiejennings> At least that's how I look at it
16:42:49 <phsmoura> theres a book about ansible
16:42:56 <phsmoura> called ansible for devops
16:43:14 <phsmoura> by jeff gerling
16:43:26 <eddiejennings> I'll look into that.
16:43:32 <phsmoura> he is not from RH, but in my opinion is a great book
16:43:38 <nirik> he's great...
16:43:50 <nikolay__> btw, here is example how to open tcpdump on remote and open it on local wireshark:)
16:43:50 <nikolay__> ssh banana -p 2222 sudo tcpdump -i eth0 -U -s0 -w - 'host 1.1.1.1' | wireshark -k -i -
16:43:56 <phsmoura> he has a lot of contributions in ansible too
16:44:08 <eddiejennings> I recognize his name from stuff in Ansible Galaxy
16:44:23 <phsmoura> so, if you decide to pursue an ansible cert that might help?
16:44:23 <eddiejennings> nice nikolay__
16:44:42 <zlopez> wireshark is helpful for troubleshooting
16:44:52 <nirik> nikolay__: cool. wireshark is fun... its own large topic
16:45:58 <eddiejennings> Another little thing I found last week for a quick "can traffic make it to this remote server"  echo > /dev/tcp/targetip/targetport && echo "Port is responding"
16:46:18 <eddiejennings> If you don't have any real tools at your disposal, I found that useful
16:46:48 <nikolay__> wow
16:47:31 <eddiejennings> I can't recall the article I found that mentioned that.  It's rationale was everything on Linux is a file of somesort, so the above should work if a connection can be established, and so far I've found it to be true
16:49:22 <eddiejennings> This was a good discussion :).  Any other things to mention before we wrap up?
16:50:54 <phsmoura> nothing here
16:50:59 <nirik> nope
16:51:10 <zlopez> eddiejennings: Really nice way to test tcp port
16:51:29 <eddiejennings> :D
16:51:56 <zlopez> eddiejennings++
16:52:04 <eddiejennings> Thanks for coming everyone!  Thus endeth. . .
16:52:05 <eddiejennings> #endmeeting