17:59:34 <nirik> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 17:59:34 <zodbot> Meeting started Wed Sep 2 17:59:34 2020 UTC. 17:59:34 <zodbot> This meeting is logged and archived in a public location. 17:59:34 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:59:34 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:59:34 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 17:59:34 <nirik> #chair mboddu nirik smooge pingou mobrien 17:59:34 <nirik> #meetingname fedora_infrastructure_ops_daily_standup_meeting 17:59:34 <nirik> #info meeting is 30 minutes MAX. At the end of 30, its stops 17:59:34 <zodbot> Current chairs: mboddu mobrien nirik pingou smooge 17:59:34 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 17:59:34 <nirik> #info agenda is at https://board.net/p/fedora-infra-daily 17:59:35 <nirik> #topic Tickets needing review 17:59:37 <nirik> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 17:59:41 <nirik> oops... I am early 17:59:43 <mboddu> Hello 17:59:59 <mboddu> Well, we can close early :D 18:00:05 <smooge> hello 18:01:00 <nirik> hey. sure... someone want to mod tickets? 18:01:49 <nirik> or I can and someone can list them? whatever works. :) 18:02:06 <mboddu> I can do it 18:02:17 <mboddu> ^ Updating tickets 18:02:20 <nirik> .ticket 9292 18:02:21 <zodbot> nirik: Issue #9292: Slow transfer speeds from fedoraproject servers - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9292 18:02:32 <nirik> I think smooge was working on this... not sure how much we can really do here. 18:02:47 <mboddu> Put everything in cloud :D 18:03:08 <smooge> nirik, at this point I think it is close can't fix but I was wanting to see if anyone had any ideas 18:03:10 <nirik> no, no, then you still have to get to them... instead, it will be the next new thing: 18:03:16 <nirik> _networkless!_ 18:03:25 <mboddu> lol 18:03:46 <fm-admin> pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9292: high-trouble, medium-gain, and ops https://pagure.io/fedora-infrastructure/issue/9292 18:03:47 <fm-admin> pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#9292 to smooge https://pagure.io/fedora-infrastructure/issue/9292 18:03:48 <fm-admin> pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9292 https://pagure.io/fedora-infrastructure/issue/9292 18:04:07 <mboddu> Thanks smooge 18:04:09 <smooge> nirik, mboddu do you guys have any ideas? 18:04:45 <mboddu> No idea, but I am trying whats are the speeds on my end 18:04:47 <nirik> I don't really. I guess asking networking folks to see if theres any known upstream issues, or if they can advertise us differently to go over another provider 18:05:17 <smooge> nirik, we have 2 network links. one is via cogentco and the other is via the one which is at 3/4 use 18:06:06 <smooge> I asked upstream and they looked at the network link and the cogentco is pretty much full. The other one is close. 18:06:13 <nirik> ugh. 18:06:29 <nirik> so we don't have the same BW as we did in phx2? 18:06:34 <smooge> we have more 18:06:41 * nirik is going over the cogentco link. 18:06:48 <smooge> as its dedicated to us 18:07:17 <smooge> or at least that is what I think we have.. 18:07:22 <nirik> huh, so we must be using more? I wonder where.... 18:07:55 <smooge> or maybe we have less 18:08:06 <smooge> I don't know at this point.. and should just ask 18:08:25 <nirik> yeah, I'd say lets ask... can't hurt. 18:08:31 <nirik> and would be good to know exactly 18:09:22 <nirik> so, high trouble / high gain / ops/ 18:09:24 <nirik> ? 18:09:33 <mboddu> +1 18:09:41 <mboddu> FWIW, I am hitting 20+ MBPS 18:09:53 <nirik> you must not be using the cogentco link 18:10:13 <nirik> my downloads start at 3-5MB/sec, then drop down to about 500k 18:11:03 <nirik> anyhow... 18:11:16 <nirik> .ticket 9293 18:11:17 <zodbot> nirik: Issue #9293: Permission for Noggin/AAA team members to auth services in staging - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9293 18:11:36 <nirik> I'd say close this with pingou willing to get them access/run playbooks... 18:11:57 <nirik> otherwise we can add people to groups to allow various playbooks as we know which ones they are 18:12:25 <fm-admin> pagure.issue.edit -- smooge edited the close_status and status fields of ticket fedora-infrastructure#9293 https://pagure.io/fedora-infrastructure/issue/9293 18:12:26 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#9293: "Permission for Noggin/AAA team members to auth services in staging" https://pagure.io/fedora-infrastructure/issue/9293#comment-675368 18:12:33 <mboddu> +1 18:12:48 <smooge> done.. [sorry was writing an email to ask about bandwidth] 18:12:51 <mboddu> I am hitting cognentco 18:12:53 <nirik> there's one private ticket, but we will just handle that in ticket 18:13:05 <mboddu> 8.|-- be3472.ccr42.jfk02.atlas.cogentco.com 0.0% 50 17.0 17.9 14.9 27.9 2.4 18:13:05 <mboddu> 9.|-- be2807.ccr42.dca01.atlas.cogentco.com 0.0% 50 29.9 25.0 22.1 32.0 2.3 18:13:05 <mboddu> 10.|-- be3084.ccr41.iad02.atlas.cogentco.com 0.0% 50 25.0 23.9 21.8 35.1 2.1 18:13:18 <mboddu> Anyway, we passed ahead of it 18:13:36 <nirik> huh, ok. 18:13:44 <nirik> any releng tickets? 18:14:02 <smooge> well in that case I am expecting it is a link firther out thant be2807.ccr42.dca01.atlas.cogentco.com which is the bottleneck 18:14:17 <mboddu> There is an unretirement ticket, low,low,ops. 18:14:37 <nirik> +1 18:14:43 <mboddu> .fasinfo andreamanzi 18:14:44 <zodbot> mboddu: User: andreamanzi, Name: Andrea Manzi, email: andrea.manzi@gmail.com, Creation: 2012-12-02, IRC Nick: None, Timezone: UTC, Locale: en, GPG key ID: None, Status: active 18:14:47 <zodbot> mboddu: Approved Groups: cla_done cla_fpca packager fedorabugs 18:15:20 <mboddu> I thought it might not be even low :D 18:15:43 <mboddu> Thats all I have 18:16:37 <nirik> cool. Any other PR's, freeze breaks, issues, tickets or poems? 18:16:50 <smooge> .ticket 9250 18:16:53 <zodbot> smooge: Issue #9250: Mails from copr-fe-dev.aws.fedoraproject.org being marked as spam - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9250 18:17:27 <smooge> ok so it looks like the way spf is set up needs to be updated 18:17:33 <nirik> yeah, I do not want to add all of amazon to our stuff. 18:17:50 <smooge> or all of google 18:18:19 <nirik> I wonder how it changes (if any) if they just sent it direct instead of via us? 18:18:51 <smooge> if they send it direct they are not neutral anymore in the spf because we do not authorize that zone 18:19:09 <smooge> at the moment they are 'neutral' and anything not sending via bastion is 'bad' 18:19:14 <smooge> or whatever the term is 18:19:45 <smooge> I am thinking that what might be needed is putting an spf record in the aws.fedoraproject.org dns zone 18:20:05 <smooge> but I am going to need to read up on it and figure it out.. 18:20:14 <nirik> or for that host specifically. 18:20:15 <smooge> and in 48 hours I turn into a pumpkin 18:20:18 <nirik> not the entire thing. 18:20:44 <nirik> I don't see how we need google in there at all. 18:21:03 <smooge> me either since we don't do anything with google 18:22:14 <nirik> ok, yeah, I guess it does need to be for the domain... aws.fedoraproject.org... adding that host as a permitted sender 18:22:44 <smooge> anyway I am going to put some changes in today.. and then I am going to drop the ticket so someone else can work on it while I am on PTO 18:23:40 <nirik> we could also I suppose masquerade that host... ie, they send to bastion and we re-write it to be from 'copr@fedoraproject.org' or something... which might have a higher reputation (or not?) 18:23:46 <nirik> sounds good 18:24:19 <smooge> ok spf does not inherit.. so I need to duplicate that record in each of our subzones 18:24:35 <smooge> so that may help part 18:25:01 <smooge> aka aws.fedoraproject.org needs its own spf record even if the mail goes through bastion which is ok for fedoraproject.org 18:25:02 <nirik> I'd say just add it to aws for now 18:25:06 <smooge> yep 18:25:12 <nirik> because SPF is horrible. ;) 18:25:44 <nirik> ok, anything else before we close out? 18:27:08 <smooge> not from me 18:27:13 <nirik> #endmeeting