19:00:05 <smooge> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting
19:00:05 <zodbot> Meeting started Thu Dec  5 19:00:05 2019 UTC.
19:00:05 <zodbot> This meeting is logged and archived in a public location.
19:00:05 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:05 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:00:05 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
19:00:05 <smooge> #chair smooge nirik relrod
19:00:05 <zodbot> Current chairs: nirik relrod smooge
19:00:05 <smooge> #info meeting is 30 minutes MAX. At the end of 30, its stops
19:00:05 <smooge> #info agenda is at https://board.net/p/fedora-infra-daily
19:00:11 <nirik> ha.
19:00:25 <smooge> even set up a board
19:00:32 <cverna> o/
19:00:48 <smooge> #topic Tickets needing review
19:00:59 * nirik is having browser problems today.
19:01:30 <smooge> .ticket 8445
19:01:31 <zodbot> smooge: Issue #8445: Provide newer versions of python-kerberos and python-requests-kerberos in epel7-infra-stg - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8445
19:01:39 <smooge> that is the current one needing review
19:02:01 <nirik> yeah.
19:02:15 <nirik> so, I guess we can do this. I don't like it, but we can do it.
19:02:38 <smooge> i guess the other would be to move it to RHEL8?
19:02:50 <nirik> we could ask, but I bet they aren't really ready for that.
19:02:51 <smooge> which I expect would be 'fun'
19:03:03 <smooge> ok who is getting the ticket?
19:03:51 <nirik> well, I don't know we need to assign it now do we?
19:04:06 <nirik> just move it to waiting for assignee and whoever can get to it can get to it
19:04:07 <smooge> what do I need to move it to wait on assignee ?
19:04:25 <cverna> what it needed to be done here ? Build the new version and tag them in epel7-infra ?
19:04:35 <cverna> or it is already built ?
19:04:48 <nirik> cverna: build them and they will be in epel7-infra-stg for them to test in stg.
19:04:51 <fm-admin> pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#8445: mbs https://pagure.io/fedora-infrastructure/issue/8445
19:04:53 <fm-admin> pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#8445 https://pagure.io/fedora-infrastructure/issue/8445
19:04:53 <nirik> then later move to prod
19:04:54 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#8445: "Provide newer versions of python-kerberos and python-requests-kerberos in epel7-infra-stg" https://pagure.io/fedora-infrastructure/issue/8445#comment-615049
19:05:32 <smooge> I asked for the exact versions to be given so we aren't pulling some version which looks close enough but isn't
19:05:53 <cverna> thanks ok
19:05:57 <nirik> well, they mentioned them... but unclear if they just got them from fedora or what
19:06:39 <smooge> exactly. if they had an internal version or ones from RHEL-8 or ...
19:06:57 <nirik> hum... thats in epel? python-requests-kerberos...
19:07:04 <nirik> .whoowns python-requests-kerberos
19:07:05 <zodbot> nirik: owner: dcallagh
19:07:56 <smooge> ok that is the only need review. We get to move to the next section
19:08:20 <smooge> #topic Current tickets being worked on (1 ticket per ops)
19:08:40 <nirik> we all have more than 1 assigned I am sure. ;)
19:08:51 <smooge> well we can only work on 1 at a time
19:09:21 <smooge> I am working on https://pagure.io/fedora-infrastructure/issue/8442 and have run into a very weird error when trying to build a guest
19:09:26 <smooge> https://paste.centos.org/view/8a3dc7fd
19:09:40 <smooge> there is no sdc but everything from dracut to lvm thinks there should be
19:10:01 <nirik> thats normal on those... can be ignored.
19:10:27 <smooge> ok the install crashed with that output so I thought it might be that
19:10:40 <nirik> I'm working on moving builders to f31 (I'm not sure if we have a ticket)
19:10:50 <smooge> noo I missed the real error. Thanks nirik.
19:11:02 <smooge> cool how is that going?
19:11:11 <smooge> do you need help on anything?
19:11:39 <nirik> I need to do buildvmhost-03 today... then arm and ppc6le needs finishing up. If I can avoid fires and meetings I might get it finished this week.
19:12:09 <nirik> there's still bugs with x86_64 installs... but I am just ignoring them for now.
19:12:38 <relrod> ok, I'm back from my appointment
19:12:43 <nirik> 1. sometimes (randomly) the virt-install just hangs. and 2) sometimes, randomly, eth0 has dhcp instead of the ip I passed to dracut
19:12:44 * relrod catches up
19:12:59 <cverna> what are the steps needed when we move the builders to a new version of Fedora ?
19:12:59 <nirik> so I have just been nuking and redoing until they work.
19:13:21 * cverna is curious :)
19:13:26 <nirik> cverna: delete all builders, redeploy from ansible. But also in this case we: updated firmware on servers, and moved them to rhel8
19:14:03 <cverna> so there is a lot of baby sitting and looking at what went wrong ?
19:14:05 <nirik> and of course disable/make sure no builds are happening on those, reinstall and bring back up
19:14:20 <nirik> in this cycle yes. In the past it has been much more direct
19:14:38 <cverna> ok thanks :)
19:15:11 <nirik> but we usually don't need firmware and rhel major updates. ;)
19:15:27 <relrod> we should figure that out though because it's going to make the proxy reinstalls painful too...
19:15:51 <nirik> relrod: it would be good to know if it affects those or is related to the buildvmhost hw.
19:17:20 <relrod> nirik: I can try doing one or two and seeing... But I'm going to feel weird if I do a few and don't get them done before I go on PTO
19:17:28 * relrod doesn't like leaving things half finished
19:17:40 <nirik> no worries. I was gonna try stg sometime soon.
19:17:41 <nirik> after builders
19:17:51 <nirik> smooge: ok, anything else we should cover here today?
19:18:14 <smooge> not from me. I am working through problems with osbs-aarch64 one step at a time and should have hte master up soonish
19:18:17 <relrod> nirik: well, I can try stg
19:18:55 <cverna> nirik: if you want to focus and finish the builders, let us know if a fire comes your way
19:19:02 <smooge> #topic Ending meeting
19:19:02 <smooge> #info Thank you all for coming. See you next business day.
19:19:03 <smooge> #endmeeting