18:00:07 <nirik> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting
18:00:08 <zodbot> Meeting started Tue Oct 13 18:00:07 2020 UTC.
18:00:08 <zodbot> This meeting is logged and archived in a public location.
18:00:08 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:08 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:08 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:00:08 <nirik> #chair mboddu nirik smooge pingou mobrien
18:00:08 <nirik> #meetingname fedora_infrastructure_ops_daily_standup_meeting
18:00:08 <nirik> #info meeting is 30 minutes MAX. At the end of 30, its stops
18:00:08 <nirik> #info agenda is at https://board.net/p/fedora-infra-daily
18:00:08 <zodbot> Current chairs: mboddu mobrien nirik pingou smooge
18:00:08 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:00:09 <nirik> #topic Tickets needing review
18:00:10 <nirik> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1
18:00:13 <smooge> here
18:00:18 <nirik> anyone around for standup today ?
18:00:30 <nirik> hey smooge
18:00:57 <nirik> smooge: you want to mod tickets today?
18:00:57 <smooge> i can mind tickets
18:01:08 <nirik> .ticket 9385
18:01:09 <zodbot> nirik: Issue #9385: Please turn on: server-named queues for fedora-messaging - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9385
18:01:53 <nirik> so, I am not sure where we are here... if astepano got it all working enough, or we need to fix docs, or we still need to enable something.
18:01:58 <mboddu> I am here
18:02:08 <nirik> I guess med/med/ops and we can work it from there.
18:02:11 <nirik> hey mboddu
18:02:24 <fm-admin> pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9385: medium-gain, medium-trouble, and ops https://pagure.io/fedora-infrastructure/issue/9385
18:02:25 <fm-admin> pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9385 https://pagure.io/fedora-infrastructure/issue/9385
18:02:56 <nirik> .ticket 9387
18:02:57 <zodbot> nirik: Issue #9387: Install ansible-freeipa on Batcave - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9387
18:03:23 <nirik> This will need a freeze break...
18:03:27 <nirik> but I'm fine with it.
18:03:41 <nirik> so I guess how about:
18:04:03 <nirik> close that "ok, sounds fine, please submit a PR and we can get Freeze break +1s there" ?
18:04:15 <smooge> ok will do that
18:04:48 <fm-admin> pagure.issue.edit -- smooge edited the close_status and status fields of ticket fedora-infrastructure#9387 https://pagure.io/fedora-infrastructure/issue/9387
18:04:49 <nirik> thats it for infra... anything on releng side mboddu ?
18:04:49 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#9387: "Install ansible-freeipa on Batcave" https://pagure.io/fedora-infrastructure/issue/9387#comment-695946
18:05:10 <mboddu> nirik: Yup, but we already discussed about it, just needs moderating
18:05:30 <mboddu> .releng 9804
18:05:31 <zodbot> mboddu: Issue #9804: Orphan kyotocabinet package only on Rawhide - releng - Pagure.io - https://pagure.io/releng/issue/9804
18:05:36 <mboddu> And there is one more
18:05:59 <mboddu> Anyway, this is low,low,ops?
18:06:49 <nirik> ack
18:07:37 <mboddu> .releng 9805
18:07:38 <zodbot> mboddu: Issue #9805: sidetag cleanup policy - releng - Pagure.io - https://pagure.io/releng/issue/9805
18:07:54 <nirik> this looks like a load of .... oh wait, I filed this one. :)
18:08:17 <mboddu> :)
18:08:24 <mboddu> So, med,med,ops?
18:08:27 <nirik> med/med/ops?
18:08:31 <nirik> yeah, +1
18:08:50 <mboddu> But first question is do we wanna implement --days-old before asking koji folks from days old since last build?
18:09:25 * mboddu thinks waiting is the best option
18:09:38 <mboddu> We can implement the --empty-delay though
18:10:21 <nirik> I'm fine waiting a bit...
18:11:04 <smooge> I have one ticket to ask for checks/ideas. ticket 5290
18:11:20 <nirik> .ticket 5290
18:11:21 <zodbot> nirik: Issue #5290: Generate infrastructure map - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/5290
18:12:11 <smooge> I am at 'what is the actual problem we are trying to solve? why are we trying to solve it? who are we solving it for'
18:12:43 <nirik> well, this was a ticket some apprentices made a number of years back... they were eager to make a map to show how things are connected better for new people.
18:12:56 <nirik> but it seems to have kind of languished.
18:13:20 <nirik> I don't think that nagviz thing is packaged up...
18:14:23 <mobrien> I think even if the nagios thing worked it would show all the infra that we have but probably not how it connects
18:15:14 <smooge> The navgiz would need packaging and it looks like a ball of PhP hell. It also needs a lot of information and I am not sure who would have the time to put it in place.
18:15:39 <smooge> To get to show how things connect you have to map it out clearly in how you distribute checks and other items
18:15:51 <smooge> which basically would require a rewrite of how we do nagios
18:16:09 <smooge> which we are looking at anyway.. but I don't see us doing it anytime soon
18:16:10 <nirik> nagios has a number of maps in it... but not sure how useful they are.
18:17:02 <mobrien> It would be really nice to have but it would need to be done in a fully automated way or it wouldn't be kept up to date
18:17:21 * nirik nods.
18:17:41 <nirik> also, this might be something to add to any stats gathering / monitoring requirements...
18:17:42 <smooge> in the end, I am not seeing this as work apprentices are doing as much as work we are doing as we would need to figure out what data is needed, where we store/keep-up that data, how we do that.. etc
18:18:44 <nirik> I'm fine with closing it in favor of the later iniatives...
18:21:01 <nirik> or keeping it if apprentices can do something with it.
18:21:03 <nirik> Either way
18:22:28 <nirik> thoughts? opinions? rotten fruit?
18:23:08 <mobrien> I would say close it, but when an initiative is being scoped for a new monitroing service add it in there?
18:24:50 <nirik> sounds ok to me, smooge ?
18:24:56 <nirik> mboddu: ?
18:25:16 <smooge> writing up items
18:25:58 <nirik> ok, any other tickets or issues to discuss?
18:26:35 <mobrien> I have a quick question. With our current setup in dns
18:27:14 <mobrien> When we define a region in named.conf and assign country codes to it the geoIP is rerouted there
18:27:35 <mobrien> is there a possibility to have a backup second region? In case there is some issue?
18:27:56 <nirik> no, I don't think so...
18:28:15 <mboddu> nirik: Sounds good to me
18:28:19 <mobrien> Ok, just an idea that I had, probably overkill anyway
18:28:19 <nirik> but we can put some out of region proxies in all regions?
18:28:41 <nirik> ie, proxy01 and proxy10 in iad2... could just always be in in case something goes wrong with the ones in that region?
18:29:03 <fm-admin> pagure.issue.edit -- smooge edited the close_status and status fields of ticket fedora-infrastructure#5290 https://pagure.io/fedora-infrastructure/issue/5290
18:29:04 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#5290: "Generate infrastructure map" https://pagure.io/fedora-infrastructure/issue/5290#comment-695953
18:29:13 <mobrien> Ya, I figure the iad ones should be the fall back as most of the traffic will end up there anyway
18:29:19 <nirik> I did hit that this weekend. aws had problems getting to sa-east-1... I tried to disable the 2 proxies in it, but it wouldn't let me because that would leave too few
18:30:07 <smooge> oh yes.. sorry I was going to bring that up next
18:30:23 <smooge> I was thinking that proxy01/proxy10 should be in each zone
18:30:50 <mobrien> So we could add 01 10 and potentially 14 have all the regions, I think it needs 3 minimum
18:31:13 <nirik> but the problem there is that we don't normally want to use those...
18:31:25 <nirik> that means anything going to them is gonna be slow. ;(
18:31:37 <nirik> so, I dunno.
18:31:51 <mobrien> ya, that is why I was asking about the 'fallback' option
18:32:29 <smooge> nirik, the alternative would be proxy14/proxy03
18:32:36 <nirik> I guess ideally, if you remove the proxies for a region and it gets too low, it would automagically add proxy01/10 to keep 3 up... but that sounds pretty complex for a pretty corner case, so perhaps we should just ignore it.
18:32:39 <smooge> that site has 2 network pipes also
18:33:05 <nirik> yeah, but 01/10/03/14 are all far from say... aus, or apac.
18:33:57 <smooge> ok for aus/apac just keep those in there. for africa put the eu ones, but for sa 03/14
18:34:07 <mobrien> I think EU and APAC are probably a lot more resilient than SA or AFR because they have multiple regions
18:35:08 <mobrien> SA and AFR have a single region each in AWS and they are both relatively new as well, they would be the ones to default to 03/14 I would say
18:36:47 <nirik> well, we are over time... lets end meeting and keep discussing?
18:36:52 <nirik> #endmeeting