18:01:07 <nirik> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting
18:01:07 <zodbot> Meeting started Thu Jun  4 18:01:07 2020 UTC.
18:01:07 <zodbot> This meeting is logged and archived in a public location.
18:01:07 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:07 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:01:07 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:01:08 <nirik> #chair cverna mboddu nirik smooge
18:01:08 <zodbot> Current chairs: cverna mboddu nirik smooge
18:01:08 <nirik> #meetingname fedora_infrastructure_ops_daily_standup_meeting
18:01:08 <nirik> #info meeting is 30 minutes MAX. At the end of 30, its stops
18:01:08 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting'
18:01:08 <nirik> #info agenda is at https://board.net/p/fedora-infra-daily
18:01:09 <nirik> #topic Tickets needing review
18:01:10 <nirik> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1
18:01:18 <smooge> morning
18:01:19 <siddharthvipul> .hello2
18:01:19 <smooge> thanks
18:01:19 <zodbot> siddharthvipul: siddharthvipul 'Vipul Siddharth' <siddharthsurfing1@gmail.com>
18:01:35 <mboddu> Hello
18:02:21 <nirik> someone want to edit today?
18:02:30 <mboddu> I can do it
18:02:39 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#8975: "pkgs01.iad2 port 22 times out" https://pagure.io/fedora-infrastructure/issue/8975#comment-656516
18:02:43 <nirik> .ticket 8975
18:02:44 <zodbot> nirik: Issue #8975: pkgs01.iad2 port 22 times out - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8975
18:03:42 <nirik> yeah, so the name hasn't changed yet... you have to use /etc/hosts right?
18:03:53 <nirik> or pkgs-iad
18:04:13 <nirik> so close ?
18:04:55 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#8975: "pkgs01.iad2 port 22 times out" https://pagure.io/fedora-infrastructure/issue/8975#comment-656519
18:05:04 <nirik> .ticket 8976
18:05:05 <zodbot> nirik: Issue #8976: Persistent storage issue on openshift in iad2 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8976
18:05:21 <smooge> nirik, there is a problem with that server
18:05:34 <nirik> so, I can explain this one in ticket... basically we have fedora_prod_mdapi, but it's ro in iad2 until we cut it over.
18:05:40 <smooge> not the one the person found.. but one with pkgs-iad
18:05:41 <nirik> which server?
18:05:50 <smooge> I added two items to the ticket
18:06:06 <smooge> I added two items to the ticket 8975
18:06:14 <smooge> geez fingers.. let me finish typing
18:06:43 <smooge> sorry was debugging and didn't see you had switched to next ticket
18:06:52 <nirik> ah, yes, might be a pagure related thing then... so mark it waiting on asignee? high-impact/medium-trouble/groomed?
18:07:05 <mboddu> +1
18:07:39 <fm-admin> pagure.issue.tag.added -- mohanboddu tagged ticket fedora-infrastructure#8975: groomed, high-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/8975
18:07:40 <fm-admin> pagure.issue.edit -- mohanboddu edited the priority fields of ticket fedora-infrastructure#8975 https://pagure.io/fedora-infrastructure/issue/8975
18:07:56 <mboddu> What about #8976?
18:07:59 <fm-admin> pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#8975: high-trouble and pagure https://pagure.io/fedora-infrastructure/issue/8975
18:08:00 <fm-admin> pagure.issue.tag.removed -- smooge removed the medium-trouble tags from ticket fedora-infrastructure#8975 https://pagure.io/fedora-infrastructure/issue/8975
18:08:15 <mboddu> Oh smooge you are updating the tickets?
18:08:17 <fm-admin> pagure.issue.edit -- kevin edited the priority fields of ticket fedora-infrastructure#8976 https://pagure.io/fedora-infrastructure/issue/8976
18:08:18 <fm-admin> pagure.issue.comment.added -- kevin commented on ticket fedora-infrastructure#8976: "Persistent storage issue on openshift in iad2" https://pagure.io/fedora-infrastructure/issue/8976#comment-656522
18:08:21 <smooge> oops sorry mboddu I missed you were editing things. I won't do any more
18:08:31 <mboddu> smooge: Okay :)
18:08:35 <nirik> I just did 8976 because I had to explain something there. ;)
18:08:37 <smooge> need to focus on 1 screen
18:08:43 <nirik> .ticket 8977
18:08:44 <zodbot> nirik: Issue #8977: Greenwave fails to access rabbitmq in iad2 - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8977
18:09:13 <nirik> I am not sure whats going on here. needs debugging.
18:09:28 <mboddu> medium trouble, high gain, groomed?
18:09:41 * mboddu not sure how big of debugging it is, so going with medium
18:09:44 <nirik> yep, and @mention abompard_ to look perhaps?
18:09:47 <mboddu> Sure
18:10:14 <nirik> .ticket 8978
18:10:15 <zodbot> nirik: Issue #8978: odcs-frontend01.iad2.fedoraproject.org cannot connect rabbitmq.fedoraproject.org - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8978
18:10:20 <nirik> this is the same issue I think.
18:10:25 <fm-admin> pagure.issue.tag.added -- mohanboddu tagged ticket fedora-infrastructure#8977: groomed, high-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/8977
18:10:26 <fm-admin> pagure.issue.edit -- mohanboddu edited the priority fields of ticket fedora-infrastructure#8977 https://pagure.io/fedora-infrastructure/issue/8977
18:10:27 <fm-admin> pagure.issue.comment.added -- mohanboddu commented on ticket fedora-infrastructure#8977: "Greenwave fails to access rabbitmq in iad2" https://pagure.io/fedora-infrastructure/issue/8977#comment-656524
18:10:47 <mboddu> Close it as dupe?
18:10:52 <nirik> so close duplicate and ask them to follow along in the last one
18:10:58 <mboddu> Okay
18:11:35 <nirik> .ticket 8979
18:11:37 <zodbot> nirik: Issue #8979: iad2/openshift: coreos-cincinnati builds are very slow and always fail - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8979
18:11:53 <nirik> needs investigation. Nothing should be slow there network wise... but something is wrong for sure.
18:12:03 <fm-admin> pagure.issue.edit -- mohanboddu edited the close_status and status fields of ticket fedora-infrastructure#8978 https://pagure.io/fedora-infrastructure/issue/8978
18:12:04 <fm-admin> pagure.issue.comment.added -- mohanboddu commented on ticket fedora-infrastructure#8978: "odcs-frontend01.iad2.fedoraproject.org cannot connect rabbitmq.fedoraproject.org" https://pagure.io/fedora-infrastructure/issue/8978#comment-656526
18:12:58 <mboddu> medium trouble, medium gain, groomed?
18:13:02 <nirik> ack
18:13:17 <nirik> .ticket 8980
18:13:18 <zodbot> nirik: Issue #8980: CentOS Jenkins: Install pungi-utils and compose-utils on F29 slave - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8980
18:13:24 <fm-admin> pagure.issue.tag.added -- mohanboddu tagged ticket fedora-infrastructure#8979: groomed, medium-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/8979
18:13:25 <fm-admin> pagure.issue.edit -- mohanboddu edited the priority fields of ticket fedora-infrastructure#8979 https://pagure.io/fedora-infrastructure/issue/8979
18:13:49 <nirik> just waiting here, move to waiting on external , med/med/groomed?
18:14:01 <mboddu> ack
18:14:17 <fm-admin> pagure.issue.tag.added -- mohanboddu tagged ticket fedora-infrastructure#8980: groomed, medium-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/8980
18:14:18 <fm-admin> pagure.issue.edit -- mohanboddu edited the priority fields of ticket fedora-infrastructure#8980 https://pagure.io/fedora-infrastructure/issue/8980
18:14:20 <nirik> .ticket 8981
18:14:21 <zodbot> nirik: Issue #8981: iad2-osbs hosts cannot reach external enpdoints - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8981
18:14:47 <nirik> needs firewall adjusting. This may be the same issue openqa is hitting... we need to allow git clones.
18:15:00 <nirik> med/med/groomed/waiting on asignee
18:15:05 <mboddu> ack
18:15:22 <fm-admin> pagure.issue.tag.added -- mohanboddu tagged ticket fedora-infrastructure#8981: groomed, medium-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/8981
18:15:23 <fm-admin> pagure.issue.edit -- mohanboddu edited the priority fields of ticket fedora-infrastructure#8981 https://pagure.io/fedora-infrastructure/issue/8981
18:15:30 <nirik> thats it from needs-review infra tickets
18:15:36 <mboddu> There is one from releng
18:15:51 <mboddu> .releng 9504
18:15:52 <zodbot> mboddu: Issue #9504: New Repos requests where "The PDC branch already exists" - releng - Pagure.io - https://pagure.io/releng/issue/9504
18:16:25 <mboddu> This just doing, removing pdc entries and running it again, this might be due to expired pagure token
18:16:33 <mboddu> I have to ping limb to check her token
18:16:47 <nirik> hum... so the packages really don't exist in pdc?
18:16:53 <mboddu> low trouble, medium gain, groomed?
18:17:03 <mboddu> So, the problem with how fedscm-admin works is:
18:17:28 <mboddu> It checks for pdc token and runs pdc tasks and then checks for src.fp.o token and then runs git tasks
18:18:06 <fm-admin> pagure.issue.comment.added -- mobrien commented on ticket fedora-infrastructure#8958: "Please issue Access keys (access key ID and secret access key) to AWS" https://pagure.io/fedora-infrastructure/issue/8958#comment-656531
18:18:18 <mboddu> So, when src.fp.o gets expired, it might create PDC entries but not git repos, but when you fix your token and run again, it thinks the repo already exists as pdc entry is already there
18:18:20 <fm-admin> pagure.issue.edit -- mobrien edited the close_status and status fields of ticket fedora-infrastructure#8958 https://pagure.io/fedora-infrastructure/issue/8958
18:18:24 <mboddu> We need to fix fedscm-admin
18:18:47 <mboddu> But it doesn't happen often, so, I never bothered to spend time on it :(
18:19:24 <nirik> ah I see.
18:20:01 <nirik> ok, fair
18:20:02 <mboddu> Thats all I got
18:20:23 <mboddu> Other tickets, I already resolved them
18:20:24 <nirik> ok, on dc move... we have begun. batcave01.phx2 is no more to be used... we have moved all the bats to batcave01.iad2.
18:20:37 <nirik> we will be doing some vpn changes soon, then wiki migration.
18:21:03 <nirik> there's still all the power builders to bring up in iad2. And we have been having a problem with the network the arm builders are on.
18:21:11 <nirik> and no doubt we will find things we need to fix... :)
18:21:23 <nirik> thats all I had.
18:21:27 <mboddu> batcave01.iad2 is still not accessible, is it still on going?
18:21:56 <mboddu> is the work still on going?*
18:22:27 <smooge> I am logged into it
18:22:28 <nirik> no, it should be accessable now...
18:22:36 <nirik> we can debug out of meeting?
18:22:51 <mboddu> (So batman cannot serve the hot city of Pheonix anymore, he is serving the only district of US now :D)
18:23:04 <mboddu> Okay
18:23:16 <nirik> frankly his work is cut out for him now being nearer to the capitol. ;)
18:23:32 <nirik> anyhow, if nothing else will close out in a min
18:23:42 <nirik> or less
18:23:44 <nirik> #endmeeting