16:00:01 <nirik> #startmeeting Infrastructure (2020-12-03)
16:00:01 <zodbot> Meeting started Thu Dec  3 16:00:01 2020 UTC.
16:00:01 <zodbot> This meeting is logged and archived in a public location.
16:00:01 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:01 <zodbot> The meeting name has been set to 'infrastructure_(2020-12-03)'
16:00:02 <nirik> #meetingname infrastructure
16:00:02 <zodbot> The meeting name has been set to 'infrastructure'
16:00:02 <nirik> #chair nirik pingou smooge cverna mizdebsk mkonecny abompard siddharthvipul mobrien
16:00:02 <zodbot> Current chairs: abompard cverna mizdebsk mkonecny mobrien nirik pingou siddharthvipul smooge
16:00:02 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
16:00:02 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:00:19 * pingou in a meeting so ping me is needed :)
16:00:24 <mkonecny> .hello zlopez
16:00:25 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@psmail.xyz>
16:00:29 <darknao> .hello2
16:00:30 <zodbot> darknao: darknao 'Francois Andrieu' <naolwen@gmail.com>
16:00:39 <TomAPC> .hello tomapc
16:00:40 <zodbot> TomAPC: tomapc 'Tom Collins' <tcollins@teratux.com>
16:00:54 <mizdebsk> hello
16:01:11 <nirik> #topic New folks introductions
16:01:11 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:01:11 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:01:26 <nirik> Ahoy new folks. Anyone new like to introduce themselves in a line or two?
16:01:40 <mobrien> .hi
16:01:42 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com>
16:02:52 <overturerob> hello!  irc/fas overturerob.  i've been in the tech industry for a long time, in the past 10 years mostly doing monitoring, building out nagios, icinga and vendor frameworks with built in and custom monitoring tools and generally manage my own CentOS and RH resources.  happy Fedora 33 user, along with other distros.  hoping to contribute and of course nothing "beneath me".
16:02:52 * nirik will wait a min more for new people.
16:03:09 <nirik> welcome overturerob!
16:03:22 <mobrien> welcome overturerob!
16:03:36 <darknao> welcome overturerob :)
16:03:44 <nirik> we have a bunch of nagios... which we keep thinking to replace sometime, but then nagios does work...
16:04:20 <overturerob> I understand :)
16:04:57 <smooge> we are needing to make it better
16:04:59 <overturerob> its ubiquitous but a bit long in the tooth.  working is first of course.  though at some point...
16:05:09 <nirik> in any case, welcome, do dive on in with the getting started page... let us know in #fedora-admin if you want to be added to the apprentice group.
16:05:56 <overturerob> thanks for warm reception!  already grouped.  cheers
16:06:23 <nirik> ok, next meetings:
16:06:26 <nirik> #info 2020-12-03 - nirik
16:06:26 <nirik> #info 2020-12-10 - mobrien
16:06:26 <nirik> #info 2020-12-17 - ?
16:06:26 <nirik> #info 2020-12-24 - canceled
16:06:27 <nirik> #info 2020-12-31 - canceled
16:06:36 <nirik> I went and marked 24th and 31st canceled
16:06:46 <nirik> do we want to cancel the 17th also? or someone want to take it?
16:06:55 <mkonecny> I can take it
16:07:03 <nirik> cool. thanks mkonecny!
16:07:42 <nirik> #topic announcements and information
16:07:42 <nirik> #info CPE Sustaining EU-hours team has a Monday through Friday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:07:42 <nirik> #info CPE Sustaining NA-hours team has a Monday through Friday 30 minute meeting going through tickets at 1800 UTC in #fedora-admin
16:07:42 <nirik> #info Datacenter move is over, but some items still need to be done: see https://fedoraproject.org/wiki/Infrastructure/2020-post-datacenter-move-known-issues
16:07:44 <nirik> #info no meetings on December 24th and 31st
16:07:51 <nirik> any other announcements or info from anyone?
16:08:58 <nirik> ok, guess not. ;)
16:09:00 <nirik> #topic Oncall
16:09:00 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:09:00 <nirik> #info siddharthvipul  is oncall for 2020-11-26 to 2020-12-03
16:09:00 <nirik> #info ? is oncall for 2020-12-03 to 2020-12-10
16:09:01 <nirik> #info ? is oncall for 2020-12-17 to 2020-12-24
16:09:14 <nirik> I can take the next oncall? unless someone else wants it?
16:09:32 <nirik> I think for the holidays we should just set it to 'please file a ticket'
16:09:58 <mkonecny> No need to have someone on oncall for holidays
16:09:58 <nirik> .oncalltakeus
16:09:58 <zodbot> nirik: Kneel before zod!
16:10:13 <nirik> mkonecny: yeah. Should we start that on the 17th?
16:10:27 <mobrien> I'll take if you want nirik
16:10:39 <mobrien> you're busy enough
16:10:44 <mkonecny> nirik: We can
16:11:06 <smooge> I agree .. just file a ticket
16:11:25 <nirik> mobrien: sure, feel free to take it. ;)
16:11:42 <nirik> ok, so 2020-12-17 to 2021-01-07 is no one
16:11:47 <mobrien> ok cool. I'll just register to zodbot znd take it in a min
16:12:36 <nirik> sounds good.
16:12:40 <mobrien> .oncalltakeeu
16:12:40 <zodbot> mobrien: Kneel before zod!
16:12:42 <nirik> #topic Monitoring discussion [nirik]
16:12:42 <nirik> #info https://nagios.fedoraproject.org/nagios
16:12:42 <nirik> #info Go over existing out items and fix
16:12:46 <nirik> lets see
16:13:26 <nirik> smooge: so, should we just take those 3 mustangs out of inventory? I don't know that we can revive them? or is there still hope?
16:13:30 <nirik> (The 3 down hosts)
16:13:36 <smooge> take them out
16:13:53 <smooge> I need to go over which ones they are and put in derack tickets
16:14:15 <smooge> no I need to go over which ones, document how to make derack tickets and have mark or david file them
16:14:27 <mobrien> +1
16:14:28 <nirik> +1
16:14:52 <nirik> The rest is low swap thing, or random staging things...
16:15:30 <nirik> we can probibly fix a few more of those before the holidays.
16:15:59 <nirik> mobrien: do the osbs ones need to be disabled still?
16:16:01 <mobrien> It wouldn't be right not to have a few nagios alerts :P
16:16:26 <mobrien> nirik: no I'll enable them now
16:16:40 <nirik> cool. yeah, I am sure we will not get to 100% clear. ;(
16:17:16 <nirik> ok, that takes us on to:
16:17:29 <nirik> #topic Learning topic
16:17:29 <nirik> #info packager workflow [nirik]
16:17:43 <nirik> I was gonna talk about packager workflow some. :)
16:18:08 <nirik> Did anyone have specific areas of that they wanted to know? Or just in general updating a package?
16:18:37 <nirik> https://fedoraproject.org/wiki/Package_update_HOWTO should be a pretty good guide
16:19:18 <nirik> its kind of a wall of text. ;) I'll go over what I do normally...
16:19:41 <nirik> So, say there's an update upstream of a package. I'll use the koji package (which I maintain) as an example.
16:20:24 <nirik> I try and enable anitya / release-monitoring for my packages, so I see the upstream release (because I watch the upstream project), but in case I miss it, I get a bugzilla bug about the update from release mornitoring
16:21:21 <nirik> Then I take a look at the update and see the changes. It varies with upstreams widely. Some upstreams have releases that make massive changes, some are just small bugfixes, some are a mix (sadly)
16:22:09 <nirik> the amount of changes tells you which releases you should look at updating. rawhide is always good to update. Stable branches are only normally updated if there's bugfixes and not too many bug changes.
16:23:08 <nirik> I then usually checkout the package with fedpkg (the existing fedora package): fedpkg -a koji (or whatever package) the -a is https instead of ssh, I tend to just use that these days
16:23:53 <nirik> then adjust the release/version in the spec file, and add a changelog entry (in vim you can do cntrol-\ to add a changelog)
16:24:23 <nirik> then use 'spectool -g *spec' to download the source from upstream. You can then check the checksum of it against upstream if they have any way to do that.
16:24:44 <nirik> then 'fedpkg prep' to unpack that source and apply any patches.
16:25:04 <nirik> I do this to make sure all/any patches still apply, and the source unpacks right, etc.
16:25:14 <nirik> then I usually make a src.rpm: fedpkg srpm
16:25:38 <nirik> and either locally mock build it or koji build --scratch it... make sure it builds fine.
16:26:04 <nirik> You can then install and test the scratch build or mock build and make sure it generally works. You can do that in a container or vm or whatever.
16:26:30 <nirik> then it's time to upload things. 'fedpkg new-sources newsource.tar.gz' to upload the new source to the lookaside cache.
16:27:10 <nirik> then I usually do 'fedpkg commit -c -p' as a short cut. That does a commit using your changelog entry as log and pushes it.
16:27:34 <nirik> then 'fedpkg build' to fire off an official build.
16:27:55 <nirik> Make sure the build finishes ok.
16:28:10 <nirik> for rawhide then bodhi makes an update for you and runs gating tests and then pushes it stable.
16:28:36 <nirik> for rawhide you need to close the release monitoring bug once you have done the build.
16:28:52 <nirik> If you are doing updates to stable releases too, you can attach that bug in bodhi and bodhi will close it.
16:29:07 <nirik> Any questions on that ? :)
16:29:46 <mobrien> I'm a little unclear on what bodhi does in the process
16:30:11 <mobrien> testing is done in koji right?
16:30:17 <nirik> bodhi takes a build in koji and pushes it out as an update...
16:30:24 <nirik> nope. koji just does the build.
16:30:38 <mobrien> oh ok
16:30:42 <nirik> well, you can run tests in a check section
16:31:13 <mkonecny> The tests itself are done in greenwave, if I'm not mistaken
16:31:23 <nirik> for stable releases, bodhi also is the thing that composes all the updates repos... in rawhide it's just a place to report test results and moving packages around so it can be composed in the next nightly
16:31:48 <nirik> mkonecny: greenwave keeps track of results, but it's not doing the testing...
16:32:02 <nirik> there's fedora-ci and openqa doing tests.
16:32:23 <mkonecny> You are right
16:32:26 <nirik> they report things back to greenwave and bodhi asks greenwave if its ok to push or not
16:32:48 <mkonecny> The greenwave is just the mediator
16:32:58 <nirik> it's all very complicated. ;( but it was more accumulated over time than designed from the top down. :)
16:32:59 <mobrien> Its really coll how it all comes together
16:33:29 <mkonecny> But rather complex
16:33:58 <nirik> Folks think bodhi would be pretty simple... just get an update and report tests and handle bugs and feedback attached and composes for updates... but all those things are very complex. ;(
16:34:52 <nirik> anyhow, any other questions?
16:35:34 <nirik> ok, that takes us to...
16:35:41 <nirik> #topic open floor
16:35:46 <nirik> Anything for open floor?
16:36:33 * nirik will close out in a min if nothing comes up.
16:36:40 <mkonecny> Maybe some wood to cover it :-)
16:37:02 <overturerob> thanks.  I'm checking out Fedora nagios now.  can't believe I haven't seen that til now.. have a good one all
16:37:47 <mobrien> thanks for the talk nirik. It was really good
16:37:50 <mobrien> nirik++
16:38:24 <mkonecny> Yeah, It was really good
16:38:32 <mkonecny> nirik++
16:38:32 <zodbot> mkonecny: Karma for kevin changed to 13 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:39:10 <mobrien> I have one semi related question
16:39:16 <nirik> thanks.
16:39:20 <nirik> sure, shoot
16:39:25 <mobrien> How do we build in staging?
16:39:43 <mobrien> is it just a flag?
16:40:03 <nirik> yeah... there's a convience command...
16:40:05 <nirik> stg-koji
16:40:13 <nirik> thats just like 'koji' but for staging
16:40:19 <mobrien> ah ok handy
16:40:20 <nirik> and I think fedpkg-stg
16:40:30 <nirik> fedpkg-stage
16:40:38 <nirik> against the stg pkgs
16:41:01 <nirik> (I wish they were consistent, but oh well)
16:41:08 <nirik> Thanks for coming everyone!
16:41:11 <nirik> #endmeeting