18:00:07 #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 18:00:08 Meeting started Tue Oct 13 18:00:07 2020 UTC. 18:00:08 This meeting is logged and archived in a public location. 18:00:08 The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:08 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:08 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:08 #chair mboddu nirik smooge pingou mobrien 18:00:08 #meetingname fedora_infrastructure_ops_daily_standup_meeting 18:00:08 #info meeting is 30 minutes MAX. At the end of 30, its stops 18:00:08 #info agenda is at https://board.net/p/fedora-infra-daily 18:00:08 Current chairs: mboddu mobrien nirik pingou smooge 18:00:08 The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:09 #topic Tickets needing review 18:00:10 #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:00:13 here 18:00:18 anyone around for standup today ? 18:00:30 hey smooge 18:00:57 smooge: you want to mod tickets today? 18:00:57 i can mind tickets 18:01:08 .ticket 9385 18:01:09 nirik: Issue #9385: Please turn on: server-named queues for fedora-messaging - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9385 18:01:53 so, I am not sure where we are here... if astepano got it all working enough, or we need to fix docs, or we still need to enable something. 18:01:58 I am here 18:02:08 I guess med/med/ops and we can work it from there. 18:02:11 hey mboddu 18:02:24 pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9385: medium-gain, medium-trouble, and ops https://pagure.io/fedora-infrastructure/issue/9385 18:02:25 pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9385 https://pagure.io/fedora-infrastructure/issue/9385 18:02:56 .ticket 9387 18:02:57 nirik: Issue #9387: Install ansible-freeipa on Batcave - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9387 18:03:23 This will need a freeze break... 18:03:27 but I'm fine with it. 18:03:41 so I guess how about: 18:04:03 close that "ok, sounds fine, please submit a PR and we can get Freeze break +1s there" ? 18:04:15 ok will do that 18:04:48 pagure.issue.edit -- smooge edited the close_status and status fields of ticket fedora-infrastructure#9387 https://pagure.io/fedora-infrastructure/issue/9387 18:04:49 thats it for infra... anything on releng side mboddu ? 18:04:49 pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#9387: "Install ansible-freeipa on Batcave" https://pagure.io/fedora-infrastructure/issue/9387#comment-695946 18:05:10 nirik: Yup, but we already discussed about it, just needs moderating 18:05:30 .releng 9804 18:05:31 mboddu: Issue #9804: Orphan kyotocabinet package only on Rawhide - releng - Pagure.io - https://pagure.io/releng/issue/9804 18:05:36 And there is one more 18:05:59 Anyway, this is low,low,ops? 18:06:49 ack 18:07:37 .releng 9805 18:07:38 mboddu: Issue #9805: sidetag cleanup policy - releng - Pagure.io - https://pagure.io/releng/issue/9805 18:07:54 this looks like a load of .... oh wait, I filed this one. :) 18:08:17 :) 18:08:24 So, med,med,ops? 18:08:27 med/med/ops? 18:08:31 yeah, +1 18:08:50 But first question is do we wanna implement --days-old before asking koji folks from days old since last build? 18:09:25 * mboddu thinks waiting is the best option 18:09:38 We can implement the --empty-delay though 18:10:21 I'm fine waiting a bit... 18:11:04 I have one ticket to ask for checks/ideas. ticket 5290 18:11:20 .ticket 5290 18:11:21 nirik: Issue #5290: Generate infrastructure map - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/5290 18:12:11 I am at 'what is the actual problem we are trying to solve? why are we trying to solve it? who are we solving it for' 18:12:43 well, this was a ticket some apprentices made a number of years back... they were eager to make a map to show how things are connected better for new people. 18:12:56 but it seems to have kind of languished. 18:13:20 I don't think that nagviz thing is packaged up... 18:14:23 I think even if the nagios thing worked it would show all the infra that we have but probably not how it connects 18:15:14 The navgiz would need packaging and it looks like a ball of PhP hell. It also needs a lot of information and I am not sure who would have the time to put it in place. 18:15:39 To get to show how things connect you have to map it out clearly in how you distribute checks and other items 18:15:51 which basically would require a rewrite of how we do nagios 18:16:09 which we are looking at anyway.. but I don't see us doing it anytime soon 18:16:10 nagios has a number of maps in it... but not sure how useful they are. 18:17:02 It would be really nice to have but it would need to be done in a fully automated way or it wouldn't be kept up to date 18:17:21 * nirik nods. 18:17:41 also, this might be something to add to any stats gathering / monitoring requirements... 18:17:42 in the end, I am not seeing this as work apprentices are doing as much as work we are doing as we would need to figure out what data is needed, where we store/keep-up that data, how we do that.. etc 18:18:44 I'm fine with closing it in favor of the later iniatives... 18:21:01 or keeping it if apprentices can do something with it. 18:21:03 Either way 18:22:28 thoughts? opinions? rotten fruit? 18:23:08 I would say close it, but when an initiative is being scoped for a new monitroing service add it in there? 18:24:50 sounds ok to me, smooge ? 18:24:56 mboddu: ? 18:25:16 writing up items 18:25:58 ok, any other tickets or issues to discuss? 18:26:35 I have a quick question. With our current setup in dns 18:27:14 When we define a region in named.conf and assign country codes to it the geoIP is rerouted there 18:27:35 is there a possibility to have a backup second region? In case there is some issue? 18:27:56 no, I don't think so... 18:28:15 nirik: Sounds good to me 18:28:19 Ok, just an idea that I had, probably overkill anyway 18:28:19 but we can put some out of region proxies in all regions? 18:28:41 ie, proxy01 and proxy10 in iad2... could just always be in in case something goes wrong with the ones in that region? 18:29:03 pagure.issue.edit -- smooge edited the close_status and status fields of ticket fedora-infrastructure#5290 https://pagure.io/fedora-infrastructure/issue/5290 18:29:04 pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#5290: "Generate infrastructure map" https://pagure.io/fedora-infrastructure/issue/5290#comment-695953 18:29:13 Ya, I figure the iad ones should be the fall back as most of the traffic will end up there anyway 18:29:19 I did hit that this weekend. aws had problems getting to sa-east-1... I tried to disable the 2 proxies in it, but it wouldn't let me because that would leave too few 18:30:07 oh yes.. sorry I was going to bring that up next 18:30:23 I was thinking that proxy01/proxy10 should be in each zone 18:30:50 So we could add 01 10 and potentially 14 have all the regions, I think it needs 3 minimum 18:31:13 but the problem there is that we don't normally want to use those... 18:31:25 that means anything going to them is gonna be slow. ;( 18:31:37 so, I dunno. 18:31:51 ya, that is why I was asking about the 'fallback' option 18:32:29 nirik, the alternative would be proxy14/proxy03 18:32:36 I guess ideally, if you remove the proxies for a region and it gets too low, it would automagically add proxy01/10 to keep 3 up... but that sounds pretty complex for a pretty corner case, so perhaps we should just ignore it. 18:32:39 that site has 2 network pipes also 18:33:05 yeah, but 01/10/03/14 are all far from say... aus, or apac. 18:33:57 ok for aus/apac just keep those in there. for africa put the eu ones, but for sa 03/14 18:34:07 I think EU and APAC are probably a lot more resilient than SA or AFR because they have multiple regions 18:35:08 SA and AFR have a single region each in AWS and they are both relatively new as well, they would be the ones to default to 03/14 I would say 18:36:47 well, we are over time... lets end meeting and keep discussing? 18:36:52 #endmeeting