18:00:03 <nirik> #startmeeting Fedora Infrastructure Ops Daily Standup Meeting 18:00:03 <zodbot> Meeting started Wed Jul 8 18:00:03 2020 UTC. 18:00:03 <zodbot> This meeting is logged and archived in a public location. 18:00:03 <zodbot> The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:03 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:03 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:03 <nirik> #chair pingou mboddu nirik smooge 18:00:03 <nirik> #meetingname fedora_infrastructure_ops_daily_standup_meeting 18:00:03 <nirik> #info meeting is 30 minutes MAX. At the end of 30, its stops 18:00:03 <zodbot> Current chairs: mboddu nirik pingou smooge 18:00:03 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:03 <nirik> #info agenda is at https://board.net/p/fedora-infra-daily 18:00:06 <nirik> #topic Tickets needing review 18:00:08 <nirik> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:00:19 <smooge> hello 18:00:23 <nirik> .ticket 9122 18:00:24 <zodbot> nirik: Issue #9122: Bugzilla still puts my old email address in CC for package freerdp - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9122 18:00:50 * mboddu is here 18:01:04 * cverna is around too :) 18:01:34 <nirik> so this one needs investigating. it might well be one of the ones pingou has been warning about on the mailing list. 18:01:41 <nirik> med/med/groomed/waiting? 18:01:47 <smooge> ok editing 18:02:16 <fm-admin> pagure.issue.tag.added -- smooge tagged ticket fedora-infrastructure#9122: groomed, medium-gain, and medium-trouble https://pagure.io/fedora-infrastructure/issue/9122 18:02:17 <fm-admin> pagure.issue.edit -- smooge edited the priority fields of ticket fedora-infrastructure#9122 https://pagure.io/fedora-infrastructure/issue/9122 18:02:18 <fm-admin> pagure.issue.comment.added -- smooge commented on ticket fedora-infrastructure#9122: "Bugzilla still puts my old email address in CC for package freerdp" https://pagure.io/fedora-infrastructure/issue/9122#comment-664219 18:02:32 <nirik> there's another private ticket... we can handle that in ticket. 18:02:43 <nirik> and thats all the infra ones in needs-review. 18:03:07 <mboddu> RelEng has one 18:03:22 <mboddu> .releng 9586 18:03:23 <zodbot> mboddu: Issue #9586: No pruning of repodata in EPEL-playground - releng - Pagure.io - https://pagure.io/releng/issue/9586 18:03:36 <smooge> hmmm some suspicious character opened that one 18:03:51 <mboddu> I wonder why they are not getting pruned 18:04:08 <smooge> it was brought up on the centos-devel list so I opened this 18:04:08 <nirik> :) 18:04:19 <smooge> I am not sure how these are populated so I wanted to make sure it was covered 18:04:34 <nirik> dunno. It should be making the repo new each time... so I suspect it's a bug in the sync script not doing --delete 18:05:36 <nirik> med-trouble/med-gain/groomed/waiting? 18:05:48 <mboddu> nirik: Yeah, you might be right, I will take a look at it 18:05:51 <mboddu> Yes 18:06:27 <mboddu> And there are couple other ones, unretirement, we can skip them 18:06:30 <mboddu> Thats all I got 18:06:52 <nirik> cool. any blockers or pr's or other tickets anyone wants to bring up? 18:06:58 <smooge> not from me 18:07:30 <cverna> I had the one about container build 18:07:30 <mboddu> Nothing from me 18:07:35 <cverna> let me find the number 18:07:49 <mboddu> cverna: Not building on s390x? 18:07:57 <smooge> no ppc63 18:08:12 <mboddu> cverna: https://pagure.io/releng/issue/9562 ? 18:08:16 <cverna> .ticket 9116 18:08:17 <zodbot> cverna: Issue #9116: Container builds don't work - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9116 18:08:28 <mboddu> nvm 18:08:30 <cverna> mboddu: no OSBS being broken 18:08:40 <nirik> ah, so thats that it's talking to the cdn, but that has different ips? 18:08:55 <nirik> can you change it to use the no-cache one? 18:09:27 <cverna> I think so but I am not sure what is the no-cache ip or url :P 18:09:38 <nirik> registry-no-cdn.fedoraproject.org 18:09:44 <nirik> thats it ^ 18:09:54 <cverna> cool yeah so I ll try to change that 18:10:18 <cverna> that should fix it, at least I hope ;) 18:10:22 <nirik> we could also try and adjust apache to avoid it for that, but this is easier. 18:10:46 <cverna> yeah that's a variable in ansible to change so let's try that first 18:11:26 <nirik> cool. 18:11:28 <cverna> that was all from me 18:11:46 <nirik> I'm reinstalling / spreading the ppc64le builders... hopefully after this the OOM problems will be gone. 18:12:25 <nirik> smooge: anything we can help with on the rest of the list? or wait for the list to be created? 18:13:54 <mboddu> nirik++ 18:14:28 <nirik> I'd like to make a 'whats left/still down' datacenter move wiki page and point status to that and clear it from it's outage state in the next few days also 18:15:08 <nirik> on that would be: staging, some builder capacity, packages, retrace, communishift 18:15:37 <nirik> and that reminds me, we need to do something about packages. We have 2 people working on replacements, we should pick one and deploy it. 18:16:02 <mboddu> nirik: I am not sure where does releng-odcs backend will be on the list, but can you prioritize it if possible? thanks. 18:16:52 <nirik> mboddu: oh yeah, forgot about that one... we can get it deployed soon. I think we have another bvmhost ready to be installed. 18:17:43 <mboddu> Thanks nirik, I understand staging, communishift are important, but when you have few min between, it would be great to set it up for us 18:17:50 <smooge> sorry been fighting with network issues 18:18:19 <mboddu> smooge: Did you restart the access point? :P 18:18:31 <smooge> actually yes 18:18:40 <nirik> sure. 18:18:49 <smooge> ok nirik I am working on the list. what do you want from it? 18:18:53 <mboddu> Haha, I was just joking, but the pun is on me now :) 18:19:28 <smooge> and where do you want it? 18:20:46 <nirik> smooge: well, just if there's anything we cna do to help you... or should just wait until you have the list done to start working on things. ;) 18:21:02 <nirik> I'd really like to hget some more mustangs/2 emags up as buildhw-a64's. 18:22:59 * mboddu goes to lunch before another meeting kicks off 18:23:09 <smooge> so you need to know what the mgmt ip addresses and network ips fro the mustangs for all those boxes ? 18:25:07 <nirik> I could work on the pair of emags... I think you already gave me the ips there, although I might have lost them 18:25:12 <nirik> we can take this to noc... 18:25:19 <smooge> ok sounds good 18:25:21 <nirik> unless there's anything else for this will close out in a min? 18:45:18 <lxvi> Hello, I created an account on FAS per the instructions on lists.fedoraproject.org but I cannot log into HyperKitty with the credentials. 18:45:54 <lxvi> I have logged out and logged back in to FAS a couple of times to confirm that my credentials work there 18:47:18 <lxvi> furthermore when I try to simply subscribe to epel-devel using the form I am sent to this page https://lists.fedoraproject.org/admin/lists/epel-devel.lists.fedoraproject.org/anonymous_subscribe which says "Something went wrong 18:49:45 <smooge> lxvi, are you clicking on the fedora logo to log in or some other method? 18:51:07 <lxvi> oh, I was just using the login form. when I click on the fedora button I get This site can’t be reachedThe webpage at https://id.fedoraproject.org/login/gssapi/negotiate?ipsilon_transaction_id=58740ed5-7f35-4507-a1e8-625dd96430bb might be temporarily down or it may have moved permanently to a new web address. 18:54:38 <smooge> the login form really should be removed but we are stuck in an update problem 18:55:30 <lxvi> I also tried using the Stack Exchange button and got a 500 18:58:36 <smooge> ok not sure.. have to go to a meeting 19:00:59 <lxvi> I just tried the GitHub button and it sent me to https://lists.fedoraproject.org/accounts/confirm-email/ but I have not received a confirmation email 19:05:40 <fm-admin> pagure.issue.edit -- kevin edited the close_status and status fields of ticket fedora-infrastructure#9067 https://pagure.io/fedora-infrastructure/issue/9067 19:05:41 <fm-admin> pagure.issue.comment.added -- kevin commented on ticket fedora-infrastructure#9067: "ppc64le builders are having OOM issues" https://pagure.io/fedora-infrastructure/issue/9067#comment-664244 19:15:05 <lxvi> Hi, someone on #fedora sent me here because I can't log in to lists.fedoraproject.org. Is there another channel that I should be on? 19:20:43 <nirik> lxvi: no, this is the right place. Are you using any ad blockers or javascript or cookie blockers? Can you try the fedora button again, that should be the one you should use... perhaps the error was a temp network issue? can you duplicate it? 19:22:19 <lxvi> not using blockers 19:24:11 <lxvi> I finally received some confirmation emails in my bulk folder. so I have logged in and subscribed to epel-devel. But I think someone ought to try some of these various buttons. Stack Exchange for example consistently results in a 500 server error 19:25:29 <lxvi> fedora button still sends me to This site can’t be reachedThe webpage at https://id.fedoraproject.org/login/gssapi/negotiate?ipsilon_transaction_id=e17e9913-9ebf-441c-b9e8-d9f8eef7af34 might be temporarily down or it may have moved permanently to a new web address. 19:25:36 <nirik> we are trying to upgrade there, but it's not an easy situation... I suspect some of the issues would be solved by the newer upstream versions 19:26:04 <nirik> very odd, it's up and seems working to me. can you reach id.fedoraproject.org in a browser ok? 19:27:47 <lxvi> just the domain? yeah, I get a very terse homepage :-/ 19:28:13 <nirik> ah, are you by chance on windows? and did it ask you to login in a pop up window? 19:29:01 <lxvi> I am on WinD'ohs 19:30:07 <lxvi> hmm, popus were blocked in chrome 19:30:58 <lxvi> still getting site can't be reached when clicking fedora button 19:31:00 <nirik> there's a windows bug/issue where it pops up a login/password dialog for you because it thinks it's loging into a windows kerberos realm... but it's not, so you should cancel that and enter username/pass in the web page to login 19:31:31 <lxvi> ok 19:32:28 <nirik> but I don't see why you would get cannot be reached from it. ;( The logs only show 401's... 19:38:30 <lxvi> when I open the network tab of the developer tools, I see a status of (failed) for https://id.fedoraproject.org/login/gssapi/negotiate?ipsilon_transaction_id=92e4d771-cfc9-4b79-8578-b196c202d6c7 19:47:13 <lxvi> anyway, I'm going to get back to getting redis set up for php on my new centos 8 vm.... 20:10:35 <nb> misc podcast.fedoraproject.org isn't getting a good SSL certificate 20:10:42 <nb> x3mboy I pinged misc about it 20:10:49 * nb doesn't have access to look on the wpengine side 20:13:21 <x3mboy> Thanks xD 20:23:21 <misc> mhh indeed, it seems that I had to switch the primary domain and check "get a certificate" 20:24:03 <nb> misc it seems to be working now, thank you 20:24:07 <nb> misc++ x3mboy++ 20:24:07 <zodbot> nb: Karma for misc changed to 12 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 20:24:10 <zodbot> nb: Karma for x3mboy changed to 13 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 20:24:48 <x3mboy> misc++ 20:24:50 <x3mboy> nb++ 20:24:51 <x3mboy> :D 20:25:14 <x3mboy> It's awesome!!! 20:25:36 <nb> x3mboy where is the podcast currently at? 20:25:40 * nb wants to listen to some 20:25:55 <x3mboy> https://x3mboy.fedorapeople.org/podcast 20:26:25 <x3mboy> And https://fedoraprojectpodcast.simplecast.com/ 20:26:42 <x3mboy> I will disable both when the new site based on WP is ready 20:42:29 <fm-admin> pagure.issue.comment.added -- x3mboy commented on ticket fedora-marketing#283: "[TRACKING TICKET] Fedora Podcast Website" https://pagure.io/fedora-marketing/issue/283#comment-664258 20:42:55 <fm-admin> pagure.issue.edit -- x3mboy edited the content fields of ticket fedora-marketing#283 https://pagure.io/fedora-marketing/issue/283 04:57:06 <jkaluza> mboddu: nirik: Can you ping me once that odcs releng backend will be ready? I will need to reconfigure ODCS to add it back there. 04:57:57 <jkaluza> I will probably also have to deploy odcs there myself anyway :). 08:14:11 <fm-admin> pagure.issue.comment.added -- hhorak commented on ticket fedora-infrastructure#9116: "Container builds don't work" https://pagure.io/fedora-infrastructure/issue/9116#comment-664272 08:47:36 <fm-admin> pagure.issue.assigned.added -- cverna assigned ticket fedora-infrastructure#9116 to cverna https://pagure.io/fedora-infrastructure/issue/9116 08:50:17 <fm-admin> pagure.issue.comment.added -- cverna commented on ticket fedora-infrastructure#9116: "Container builds don't work" https://pagure.io/fedora-infrastructure/issue/9116#comment-664275 09:05:22 <fm-admin> pagure.issue.edit -- churchyard edited the close_status and status fields of ticket releng#9552 https://pagure.io/releng/issue/9552 09:05:23 <fm-admin> pagure.issue.comment.added -- churchyard commented on ticket releng#9552: "Retire commits don't retire packages" https://pagure.io/releng/issue/9552#comment-664280 09:24:12 <pemensik> hi. What is a reason for closing https://pagure.io/releng/fedora-scm-requests/issue/26934 without creating it? 09:25:09 <pemensik> should I fill some infrastructure ticket for it to some component? 09:32:22 <jednorozec> pemensik, I am not sure what is your intent there. Do you want to create new distgit branch for the existing repository? or a new repository in dist-git? 09:33:16 <pemensik> tests namespace repository for bind. That issue was created by fedpkg request-tests-repo 09:33:53 <pemensik> that repository does not exist, so I want a new repository obviously :) 09:35:04 <pemensik> also, if it was unclear, I would expect a question, not closing 09:35:36 <jednorozec> pemensik, it was closed automaticali by the tool we use to process scm requests. 09:36:16 <pemensik> jednorozec, and my question was, whether to fill an issue to that tool. So where can I do that? Is there issue already? 09:37:03 <jednorozec> pemensik, https://pagure.io/fedscm-admin/issues 09:37:49 <pemensik> is upstreamurl mandatory, as it seems from that backtrace? 09:48:13 <jednorozec> pemensik, after digging in the fedscm-admin I have found that it expects the upstreamurl 09:49:39 <pemensik> jednorozec, unfortunately, fedpkg request-tests-repo can provide description, but not the upstreamurl. 09:50:16 <jednorozec> pemensik, please open issue for the fedscm-admin 09:51:17 <pemensik> jednorozec, I have created https://pagure.io/fedscm-admin/issue/36, would it be sufficient as comment there? 09:51:38 <jednorozec> pemensik, that would be great 10:10:02 <jednorozec> pemensik, I have created PR to fix this. 10:46:48 <pemensik> jednorozec, do you think description should be mandatory? 10:47:23 <pemensik> I would use issue_body_json.get('upstreamurl') if possible :) 10:48:56 <pemensik> and the same with 'description' 10:49:41 <pemensik> or maybe better: issue_body_json.get('upstreamurl', '').strip() 10:56:30 <jednorozec> pemensik, that is what I just did :) 10:57:58 <pemensik> jednorozec, okay, seems nicer, thanks 11:35:05 <siddharthvipul_> REMINDER: There will be a CPE PO Office Hours on IRC today (2020-07-09) at 13:00 UTC in #fedora-meeting-1 11:49:21 <fm-admin> pagure.issue.comment.added -- erack commented on ticket fedora-infrastructure#9116: "Container builds don't work" https://pagure.io/fedora-infrastructure/issue/9116#comment-664314 15:00:13 <nirik> meeting time in #fedora-meeting-1 15:15:07 <asaleh> .fasinfo asaleh 15:15:08 <zodbot> asaleh: User: asaleh, Name: Adam Saleh, email: asaleh@redhat.com, Creation: 2012-10-10, IRC Nick: None, Timezone: UTC, Locale: en, GPG key ID: None, Status: active 15:15:11 <zodbot> asaleh: Approved Groups: cla_fpca cla_done fi-apprentice sysadmin fedorabugs packager 15:15:13 <asaleh> .fasinfo wwoods 15:15:17 <zodbot> asaleh: User: wwoods, Name: Will Woods, email: wwoods@redhat.com, Creation: 2006-05-12, IRC Nick: wwoods, Timezone: US/Eastern, Locale: en, GPG key ID: , Status: active 15:15:20 <zodbot> asaleh: Approved Groups: cla_fedora cla_done gitpungi +gitlittlebottom @fedorabugs packager gitanaconda @gitbeaker +gitsnake sysadmin-test +bzrpreupgrade @gitpython-bugzilla hosted-content @gitautoqa @gitfedora-qa proventesters @hgsmolt gitlorax @gitbeefymiracle cla_fpca gitblivet qa @weldr @weldr-sig sysadmin +sysadmin-analysis 15:15:35 <asaleh> .fasinfo scoady 15:15:36 <zodbot> asaleh: User: scoady, Name: Stephen Coady, email: scoady@redhat.com, Creation: 2019-08-02, IRC Nick: None, Timezone: UTC, Locale: en, GPG key ID: None, Status: active 15:15:38 <zodbot> asaleh: Approved Groups: fi-apprentice cla_fpca cla_done sysadmin sysadmin-bodhi packager fedorabugs 15:17:38 <asaleh> nils, how do I get more packagers to a package? 15:18:18 <asaleh> nils, I would like to add wwoods and scoady to be packagers of our dumy-package-* :) 15:18:22 <nils> asaleh, is this a community or a configuration question=? :) 15:18:28 <nils> ahh, the latter 15:18:33 <nils> sec 15:19:33 <nils> asaleh, the dummy-test-package-rubino e.a. packages` 15:19:33 <nils> ? 15:19:52 <asaleh> nils, -gloster, -crested, e.t.c, yes 15:21:20 <nils> asaleh, you need to be admin of the package already. In the case of -crested, this would be pingou. I guess it would be best if he simply added the @fedora-infra group as co-maintainers (which in theory should include both scoady and wwoods). 15:21:44 <asaleh> nils++ ... 15:21:44 <zodbot> asaleh: Karma for nphilipp changed to 11 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:22:48 <nils> asaleh, in general, it's on the dist-git repo page (src.fp.o/rpms/...), then "Settings", "Users & Groups", "add user"/"add group" 15:26:44 <nils> nirik, re: logview (came in late), if we can use the dateutil Python module (3rd party) I guess we could forgo running/decoding /bin/date there 15:27:16 <nils> darknao ^^ 15:27:30 <nils> Just not sure how minimal these scripts need to be. 15:28:17 <Tayeh> hello 15:28:26 <nirik> well, this one just runs on batcave... so not a greal big deal on size... 15:29:56 <Tayeh> .fasinfo mohamed94 15:30:01 <zodbot> Tayeh: User: mohamed94, Name: Mohammed Tayeh, email: m.tayeh94@gmail.com, Creation: 2013-09-22, IRC Nick: Tayeh, Timezone: Asia/Gaza, Locale: en, GPG key ID: , Status: active 15:30:04 <zodbot> Tayeh: Approved Groups: ambassadors cla_fpca cla_done qa fedorabugs 15:30:04 <nirik> Tayeh: you wanted to be added to the apprentice group? 15:30:23 <Tayeh> yes 15:30:29 <Tayeh> please 15:31:24 <nirik> done. will take ~15min to sync out... 15:31:39 <darknao> nils, yeah I was also thinking about that, this shell trick is bugging me :p 15:31:58 <Tayeh> thanks nirik :) 15:32:37 <nirik> python3-dateutil is on batcave01... :) so feel free to use it. 15:32:53 <nirik> I merged that existing PR... so you can do a new one for that if you all want to? 15:33:23 <nirik> and many thanks for the porting! 15:33:35 <darknao> you're welcome 15:34:10 <Tayeh> what is the next step?, to be more efficient on the fedora infrastructure team ! 15:34:21 <darknao> nirik, actually, since you're on it, can I also have access to the apprentice group ? 15:34:29 <nirik> we should go over ansible PR's. I hate old pr's lingering around... 15:34:41 <nirik> darknao: sure, whats your fas id? 15:34:46 <nirik> oh, I can look 15:34:51 <darknao> .fasinfo darknao 15:34:52 <zodbot> darknao: User: darknao, Name: Francois Andrieu, email: naolwen@gmail.com, Creation: 2015-09-04, IRC Nick: darknao, Timezone: Europe/Paris, Locale: en, GPG key ID: 1E435DAF6A394AA115877CC132349F8C63803B15, Status: active 15:34:55 <zodbot> darknao: Approved Groups: cla_fpca cla_done 15:35:47 <nirik> done 15:35:57 <darknao> nirik++ thanks! 15:35:58 <zodbot> darknao: Karma for kevin changed to 27 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:36:01 <fm-admin> pagure.issue.new -- scoady opened a new ticket fedora-infrastructure#9124: "[Request] Access to monitor-gating on os.fedoraproject.org" https://pagure.io/fedora-infrastructure/issue/9124 15:55:59 <Tayeh> mohamed94@bastion-iad01.fedoraproject.org: Permission denied (publickey). 16:06:20 <Tayeh> nirik: 16:06:57 <Tayeh> i'm setup my ssh access like https://fedora-infra-docs.readthedocs.io/en/latest/sysadmin-guide/sops/sshaccess.html#ssh-sop 16:07:21 <Tayeh> but i get a error mohamed94@bastion-iad01.fedoraproject.org: Permission denied (publickey). 17:17:04 <fm-admin> pagure.issue.comment.added -- pingou commented on ticket fedora-infrastructure#9124: "[Request] Access to monitor-gating on os.fedoraproject.org" https://pagure.io/fedora-infrastructure/issue/9124#comment-664400 17:20:35 <fm-admin> pagure.issue.comment.added -- pingou commented on ticket fedora-infrastructure#9122: "Bugzilla still puts my old email address in CC for package freerdp" https://pagure.io/fedora-infrastructure/issue/9122#comment-664403 17:40:57 <fm-admin> pagure.issue.comment.added -- kevin commented on ticket fedora-infrastructure#9122: "Bugzilla still puts my old email address in CC for package freerdp" https://pagure.io/fedora-infrastructure/issue/9122#comment-664409 18:00:02 <zodbot> nirik: Error: Can't start another meeting, one is in progress. 18:00:02 <nirik> #chair pingou mboddu nirik smooge 18:00:02 <zodbot> Current chairs: mboddu nirik pingou smooge 18:00:02 <nirik> #meetingname fedora_infrastructure_ops_daily_standup_meeting 18:00:02 <nirik> #info meeting is 30 minutes MAX. At the end of 30, its stops 18:00:02 <zodbot> The meeting name has been set to 'fedora_infrastructure_ops_daily_standup_meeting' 18:00:02 <nirik> #info agenda is at https://board.net/p/fedora-infra-daily 18:00:04 <nirik> #topic Tickets needing review 18:00:07 <nirik> #info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1 18:00:07 <smooge> hello 18:00:10 <nirik> oh, mast one didn't end? 18:00:16 <nirik> zodbot: listmeetings 18:00:16 <zodbot> nirik: ('#fedora-admin', 'freenode') 18:00:21 <nirik> #endmeeting