19:00:09 <smooge> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 19:00:09 <zodbot> Meeting started Tue Feb 4 19:00:09 2020 UTC. 19:00:09 <zodbot> This meeting is logged and archived in a public location. 19:00:09 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:09 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:09 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 19:00:09 <smooge> #chair smooge nirik relrod cverna 19:00:09 <smooge> #info meeting is 30 minutes MAX. At the end of 30, its stops 19:00:09 <smooge> #info agenda is at https://board.net/p/fedora-infra-daily 19:00:09 <zodbot> Current chairs: cverna nirik relrod smooge 19:00:09 <smooge> #topic Tickets needing review 19:00:10 <smooge> #info https://pagure.io/fedora-infrastructure/issues 19:00:19 <nirik> hello party people. 19:00:21 <smooge> good morning every one 19:00:37 <nirik> nice... thanks freenode 19:00:49 <nirik> ok, lets just start back and work our way up again. 19:00:58 <nirik> .ticket 8568 19:00:59 <zodbot> nirik: Issue #8568: New pkgdb group request - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8568 19:01:16 <nirik> pingou: did you have a chance to look or find a SOP for this process? 19:01:19 <cverna> hello 19:01:30 <cverna> ha i forgot to ask him 19:01:42 <nirik> .ticket 8587 19:01:43 <zodbot> nirik: Issue #8587: start.fpo is not updating - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8587 19:01:44 <nirik> no worries. 19:01:52 <nirik> did we want to close this one? it seems working? 19:02:38 <cverna> yes +1, it can be reopen worst case 19:03:00 <nirik> yep. +1 to close. 19:03:09 <cverna> on it 19:03:10 <nirik> .ticket 8595 19:03:11 <zodbot> nirik: Issue #8595: Jenkins seems to be out of space - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8595 19:03:21 <nirik> bstinson was looking at this, so leave it I guess 19:03:28 <smooge> brian got back from travel and seems to have taken it 19:04:01 <fm-admin> pagure.issue.edit -- cverna edited the close_status and status fields of ticket fedora-infrastructure#8587 https://pagure.io/fedora-infrastructure/issue/8587 19:04:02 <fm-admin> pagure.issue.comment.added -- cverna commented on ticket fedora-infrastructure#8587: "start.fpo is not updating" https://pagure.io/fedora-infrastructure/issue/8587#comment-624321 19:04:03 <nirik> .ticket 8606 19:04:04 <zodbot> nirik: Issue #8606: Renew or Retire Certificate for *.fedorahosted.org - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8606 19:04:15 <nirik> so, we do need lists.fedorahosted.org for sure. 19:04:29 <nirik> do we want to keep the * still? or let it go? 19:05:10 <smooge> I would say let it go and just do a lists.fedorahosted, www.fedorahosted 19:05:24 <nirik> we are coming up on 3 years since the retirement. 19:05:39 <cverna> +1 to let go 19:05:45 <nirik> if we do let it go, we need to drop the A record and remove it from proxies 19:06:01 <nirik> can someone update the ticket with that info then? 19:06:13 <smooge> nirik, yep 19:06:15 <smooge> will do so 19:06:26 <nirik> .ticket 8607 19:06:27 <zodbot> nirik: Issue #8607: Renew Certificate for *.fedoraproject.org - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8607 19:06:43 <nirik> I think we should perhaps get a new digicert wildcard for this. 19:06:59 <nirik> we use it in a number of places and its a pain to update them all, IMHO 19:07:42 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#8606: "Renew or Retire Certificate for *.fedorahosted.org" https://pagure.io/fedora-infrastructure/issue/8606#comment-624323 19:07:54 <smooge> yep will do so 19:07:57 <cverna> I don't have much opinion on that, but whatever the easiest seems like a good deal to me :) 19:08:02 <nirik> and we should get it well in advance so we don't have to deal with it durin the move 19:08:11 <nirik> when does it expire? 19:08:12 <smooge> I will put it on my todo list today 19:08:17 <smooge> 85 days from now 19:08:29 <nirik> oh... huh 19:08:35 <nirik> december 2020 my browser says 19:08:42 <cverna> that's plenty of time to forget about it :P 19:08:51 <nirik> oh, thats getfedora. 19:09:21 <nirik> .ticket 8608 19:09:22 <zodbot> nirik: Issue #8608: Renew Certificate for fedoracommunity.org, www.fedoracommunity.org - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8608 19:09:49 <nirik> this one I would think would be fine to move to letsencrypt 19:10:05 <nirik> .ticket 8610 19:10:06 <zodbot> nirik: Issue #8610: Please link the "Bug reports" link on src.fp.o directly to RHBZ - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8610 19:10:23 <nirik> so, unclear on this one. Is it solved? or is there something still to be done? 19:10:47 <smooge> i don't know 19:10:55 <cverna> I think it is solved 19:10:55 <smooge> i would say its done 19:11:07 <cverna> I can close it 19:11:10 <nirik> ok. close then and ask them to reopen if further is needed. 19:11:29 <nirik> .ticket 8612 19:11:29 <fm-admin> pagure.issue.edit -- cverna edited the close_status and status fields of ticket fedora-infrastructure#8610 https://pagure.io/fedora-infrastructure/issue/8610 19:11:30 <fm-admin> pagure.issue.comment.added -- cverna commented on ticket fedora-infrastructure#8610: "Please link the "Bug reports" link on src.fp.o directly to RHBZ" https://pagure.io/fedora-infrastructure/issue/8610#comment-624328 19:11:33 <zodbot> nirik: Issue #8612: Outage: Upgrade of Copr servers - 2020-02-06 8:30 UTC - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8612 19:11:53 <nirik> let me see what perms our outage calendar has 19:12:31 <nirik> it's sysadmin-main now... 19:12:36 <nirik> should we just change it to sysadmin? 19:13:12 <cverna> +1 19:13:48 <smooge> +1 on my part. I do not know how to implement 19:13:59 <fm-admin> pagure.issue.comment.added -- kevin commented on ticket fedora-infrastructure#8612: "Outage: Upgrade of Copr servers - 2020-02-06 8:30 UTC" https://pagure.io/fedora-infrastructure/issue/8612#comment-624330 19:14:09 <nirik> .ticket 8613 19:14:10 <zodbot> nirik: Issue #8613: Move everything off 209.132.184.0/24 network (cloud.fedoraproject.org, fedorainfracloud.org) by 2020-03-01 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8613 19:14:21 <nirik> not much to say here. I guess I'll be working on this one. 19:14:36 <nirik> .ticket 8614 19:14:37 <zodbot> nirik: Issue #8614: Retire old OpenStack Cloud - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8614 19:14:40 <nirik> and this one likewise 19:14:54 <nirik> .ticket 8615 19:14:55 <zodbot> nirik: Issue #8615: Retire keys.fedoraproject.org 2020-02-07 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8615 19:15:08 <cverna> except for COPR there isn't much stuff that needs to be moved 19:15:10 <nirik> smooge: you gonna do this one? or should we see if nb wants to? or ? 19:15:32 <nirik> cverna: there's some stuff... maintainer-test machines, a bunch of dev instances... some apps like libravatar, etc 19:16:21 <smooge> I was going to do most of it. i was worried it would feel too much like taking old yella out for a last walk to nb 19:16:27 <cverna> oh yes :( so quite a bit of work 19:16:41 <nirik> smooge: ok. 19:16:47 <nirik> .ticket 8616 19:16:48 <zodbot> nirik: Issue #8616: How can we change emails in discussion.fedoraproject.org Discourse? - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8616 19:17:02 <nirik> so, I have no idea on this one. 19:17:10 <cverna> who is dealing with discourse ? misc maybe ? 19:17:13 <nirik> no 19:17:16 <smooge> no one is 19:17:22 <cverna> haha nice :) 19:17:26 <smooge> it is an outsourced tool 19:17:28 <nirik> mattdm I think has been the most in contact with them... 19:17:42 <nirik> but I am unsure how to contact them or ask for things. 19:17:51 <smooge> bcotton and mattdm 19:18:01 <nirik> smooge: bcotton filed the ticket. ;) 19:18:14 <smooge> yeah which is what surprised me 19:18:14 <cverna> let's assign to mattdm then lol 19:18:28 <smooge> i am wondering if the info is in openidc 19:18:39 <nirik> ? 19:18:54 <nirik> yeah, lets @mattdm and ask for a contact process or something. 19:18:57 * nirik can do that 19:19:11 <smooge> the initial email address is gotten when the person gets authenticated with our openid stuff 19:19:32 <cverna> yes but the email address is stored on discourse side, not much we can do IMHO 19:19:50 <smooge> so I was wondering if it is one of those things where fas would have sent a fedmsg for the other tool to update its info.. and well 19:20:08 <fm-admin> pagure.issue.edit -- kevin edited the priority fields of ticket fedora-infrastructure#8616 https://pagure.io/fedora-infrastructure/issue/8616 19:20:09 <fm-admin> pagure.issue.comment.added -- kevin commented on ticket fedora-infrastructure#8616: "How can we change emails in discussion.fedoraproject.org Discourse?" https://pagure.io/fedora-infrastructure/issue/8616#comment-624331 19:20:27 <nirik> smooge: they would have to write something to do that. 19:20:32 <smooge> aka similar to fedorapeople not getting the right ssh keys 19:20:38 <smooge> until we force a run 19:20:45 <smooge> understood 19:21:00 <nirik> sure, but I very much doubt there's anything listening for those currently (even tho they don't send) 19:21:08 <nirik> anyhow, thats the new stuff I think. 19:21:15 <nirik> any other tickets we want to go over 19:21:15 <smooge> yes 19:21:16 <nirik> ? 19:21:26 <smooge> not from me. I am full up on tickets 19:21:31 <cverna> .ticket 8549 19:21:32 <nirik> we could do releng for a bit. ;) 19:21:34 <zodbot> cverna: Issue #8549: Move retrace.fedoraproject.org to letsencrypt - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8549 19:21:35 * nirik runs 19:21:38 <fm-admin> pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#8606 to smooge https://pagure.io/fedora-infrastructure/issue/8606 19:21:46 <fm-admin> pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#8615 to smooge https://pagure.io/fedora-infrastructure/issue/8615 19:21:50 <cverna> we had another patch on that but I am not sure if this is good or not :) 19:21:58 <nirik> it's not goign to work 19:22:23 <nirik> The problem is that's added retrace to proxies, but it doesn't go there. we need to add the redirect directly to a apache config for retrace itself 19:22:50 <cverna> ok I can update the ticket, and try to add more pointers :) 19:23:12 <cverna> the proxies are like a magic black box to me :) 19:23:56 <cverna> I don't have anything else, and since we went overtime yesterday I guess we can finish a bit earlier today :D 19:24:10 <nirik> the problem is that retrace is also... different. we don't really maintain that... 19:24:45 <cverna> I think the person submitting the patch is from that team 19:24:53 <nirik> oh? ok. 19:25:02 <cverna> let me check the list 19:25:48 <cverna> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/TARKWBETIBZYF7LUBGIUJPZFMNTCWFQW/ 19:26:07 <cverna> ABRT stack, does that include retrace ? 19:26:13 <smooge> yep 19:26:24 <smooge> which is another ticket I need to file 19:26:31 <smooge> set up new retrace server 19:26:38 <nirik> yeah 19:26:45 <smooge> hey cverna you want to learn about retrace? 19:26:58 <cverna> smooge: ha ha why not :-) 19:27:07 <fm-admin> pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#8608 to smooge https://pagure.io/fedora-infrastructure/issue/8608 19:27:16 <cverna> where is that running currently ? 19:27:30 <fm-admin> pagure.issue.assigned.added -- smooge assigned ticket fedora-infrastructure#8607 to smooge https://pagure.io/fedora-infrastructure/issue/8607 19:29:39 <nirik> side note: noc playbook seems broken 19:29:42 <fm-admin> pagure.issue.new -- smooge opened a new ticket fedora-infrastructure#8617: "Rack, stack, and install new retrace server" https://pagure.io/fedora-infrastructure/issue/8617 19:29:52 <smooge> what is brok ein it 19:29:54 <nirik> thats where my master playbook run died. 19:30:23 <nirik> "Error: Could not fin 19:30:23 <nirik> d any host matching 'proxy30.fedoraproject.org' (config file '/etc/nagios/hostgroups/all-external.cfg', starting on line 19:30:23 <nirik> 13)", "Error: Could not expand members specified in hostgroup (config file '/etc/nagios/hostgroups/all-external.cfg', sta 19:30:23 <nirik> rting on line 13)", "Error: Service '*' on host '*' has an escalation but is not defined anywhere!", "Error: Could not re 19:30:23 <smooge> cverna, retrace is an old box in the QA network 19:30:23 <smooge> #endmeeting