15:00:48 <smooge> #startmeeting Infrastructure (2019-05-23)
15:00:48 <zodbot> Meeting started Thu May 23 15:00:48 2019 UTC.
15:00:48 <zodbot> This meeting is logged and archived in a public location.
15:00:48 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:48 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:48 <zodbot> The meeting name has been set to 'infrastructure_(2019-05-23)'
15:00:48 <smooge> #meetingname infrastructure
15:00:48 <zodbot> The meeting name has been set to 'infrastructure'
15:00:48 <smooge> #topic aloha
15:00:48 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs
15:00:48 <zodbot> Current chairs: abompard bowlofeggs cverna mizdebsk mkonecny nirik pingou puiterwijk relrod smooge tflink
15:01:34 <nirik> morning everyone.
15:01:58 <smooge> good morning
15:02:07 <smooge> I think everyone has started 5 day weekends
15:02:21 <nirik> oh, that sounds nice
15:02:38 <smooge> #topic announcements and information
15:02:39 <smooge> #info only ~90 days until F31 freeze and ~60 days until F31 rebuild please land changes appropriately.
15:02:39 <smooge> #info bowlofeggs will be going on extended leave in June
15:02:39 <smooge> #info nirik will be on leave from 2019-06-03 to 2019-06-07
15:02:39 <smooge> #info many people will be at a community platform engineering f2f 2019-06-10 to 14
15:02:40 <smooge> #info Flock2Fedora 2019-08-08 -> 2019-08-11
15:02:42 <smooge> #info update reboots happened this week. last set today at 2100 UTC
15:02:44 <smooge> #info Anitya (release-monitoring.org) 0.15.1 is now deployed in production
15:02:46 <smooge> #info the-new-hotness 0.11.2 is now deployed on production
15:02:49 <mkonecny> .hello zlopez
15:02:51 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu>
15:04:15 <clime> hi
15:06:07 <bowlofeggs> .hello2
15:06:08 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:07:18 <smooge> #topic Oncall
15:07:18 <smooge> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:07:18 <smooge> #info smooge is on call from 2019-05-16 -> 2019-05-23
15:07:18 <smooge> #info relrod is on call from 2019-05-23 -> 2019-05-30
15:07:19 <smooge> #info smooge is on call from 2019-05-30 -> 2019-06-06
15:07:20 <smooge> #info no one is on call from 2019-06-06 -> 2019-06-20
15:07:21 <smooge> #info pingou is on call from 2019-06-20 -> 2019-06-27
15:07:23 <smooge> #info Summary of last week: (from smooge )
15:08:03 <smooge> This week was dealing with outage related items and broken playbooks. Most of the work was actually done by nirik
15:08:06 <smooge> thanks nirik
15:08:22 * relrod checks in a few minutes late, sorry
15:08:32 <smooge> i hand you the coffee cup
15:08:47 <relrod> .alias add oncall "echo relrod (Rick Elrod) is oncall. If not around, please file a ticket: https://pagure.io/fedora-infrastructure/issues"
15:08:47 <zodbot> relrod: Kneel before zod!
15:09:15 <smooge> #topic Monitoring discussion
15:09:15 <smooge> #info https://nagios.fedoraproject.org/nagios
15:09:15 <smooge> #info Go over existing out items and fix
15:09:23 * nirik looks
15:09:48 <smooge> virthost06 should be fixed today during a reboot
15:10:00 <smooge> virthost05 should be listed as it is having fan issues
15:10:34 <smooge> I don't know about notifs
15:10:54 <nirik> I'm getting notifications...
15:10:59 <nirik> might just need playbook run?
15:11:38 <nirik> the koji thing is kde chainbuild all failing on epel7
15:12:35 <nirik> storinator01 is the maching that was being moved... did it get finally moved?
15:13:07 <smooge> it got turned off.
15:13:16 <smooge> I have no info if it has been moved
15:13:44 <nirik> ☹
15:13:50 <smooge> It is on my list to deal with today
15:14:10 <smooge> Basically I was out last night and it was turned off. I did tell them they could do so at any time
15:14:31 <smooge> so I just need to get some time and chat with them
15:14:43 <smooge> #topic Tickets discussion
15:14:43 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:15:04 <mkonecny> https://pagure.io/fedora-infrastructure/issue/7828
15:15:24 <mkonecny> I noticed this few hours ago and created ticket
15:16:19 <mkonecny> It's not nice, because it's printing failed builds to bugzilla - https://bugzilla.redhat.com/show_bug.cgi?id=1713383
15:16:24 <nirik> mkonecny: does: https://src.fedoraproject.org/rpms/rust-lsd.git work?
15:16:35 <nirik> cgit was retired...
15:16:41 <mkonecny> nirik: Yes, I tried it locally
15:16:51 <nirik> no no, I mean without /cgit/ in it.
15:17:30 <mkonecny> Ok, I probably found bad example
15:18:04 <mkonecny> I will look
15:18:32 <nirik> ok. That might be it... if not, not sure what it could be...
15:19:08 <mkonecny> sh-4.4$ git clone https://src.fedoraproject.org/rpms/rust-lsd.git
15:19:08 <mkonecny> fatal: could not create work tree dir 'rust-lsd': Permission denied
15:19:16 <mkonecny> This is the output from pod
15:20:11 <puiterwijk> mkonecny: right. You can only write to volumes that are readwrite
15:20:17 <nirik> ah, needs a writable local dir? does cd /tmp work? or does it need storage?
15:20:26 <mkonecny> puiterwijk: My bad
15:20:26 <puiterwijk> In openshift, you run as a random, high, userid that does not have write access outside of /tmp and volumes
15:20:38 <puiterwijk> nirik: emptyDir storage would work
15:20:44 * nirik nods.
15:20:53 <puiterwijk> so it doesn't need NFS. /tmp *might* work, but not sure
15:21:09 <mkonecny> puiterwijk: Yeah, cloning to /var/tmp is working
15:21:15 <puiterwijk> (/tmp working depends on how big it is)
15:21:17 <nirik> large projects could be large too...
15:21:21 <puiterwijk> mkonecny: yes, but /tmp is limited in storage.
15:21:46 <puiterwijk> And you cannot see how much storage it has from inside the container.
15:21:52 <bowlofeggs> i.e., you might run out of memory
15:21:55 <mkonecny> I need to change the url, that is used to clone the project it seems
15:21:57 <bowlofeggs> /tmp might be a tmpfs, no?
15:22:06 <bowlofeggs> (haha i wrote the second message first, but forgot to escape the /)
15:22:35 <puiterwijk> bowlofeggs: I think that in Openshift, they didn't go for tmpfs /tmp, since there's usually lots of containers :)
15:22:56 <bowlofeggs> cool
15:23:02 <bowlofeggs> yeah i guess that's something systemd does anyway
15:23:06 * puiterwijk goes hope.... and check....
15:23:48 <mkonecny> What could I do about this, I need to download sources for each project
15:24:09 <puiterwijk> mkonecny: add an emptyDir volume, and make sure you clone in there.
15:24:14 <puiterwijk> And please make sure to clean up afterwards
15:25:08 <mkonecny> puiterwijk: Thanks, this will need some changes in the-new-hotness. I'm not sure if this is configurable
15:26:35 <mkonecny> I closed the ticket as Fixed
15:26:57 <smooge> ok thanks.
15:27:03 <smooge> any other tickets for today?
15:27:21 <smooge> #topic Open Floor
15:27:52 <nirik> #info everyone interested please weigh in on the paste thread on the infra list
15:28:08 <nirik> #info everyone think of flock talks they would like to give or go to.
15:29:20 <smooge> #info One last reminder. We will be rebooting systems starting at 21:00 UTC today. Expect outages
15:30:13 <bowlofeggs> so i miiiight be able to release bodhi 4.0.0 this week upstream
15:30:22 <bowlofeggs> next monday is a US holiday
15:30:38 <bowlofeggs> so i might deploy it on tuesday, but that gives a shorter week if there are weird issues
15:30:52 <bowlofeggs> any thoughts on tuesday vs monday of the following week for deployment?
15:31:00 <bowlofeggs> i'm also not 100% sure we will have it ready by tuesday
15:31:11 <nirik> I will be out that week entirely.
15:31:18 <bowlofeggs> i need to verify one last test
15:31:34 <bowlofeggs> nirik: next week or the second week?
15:31:36 <puiterwijk> bowlofeggs: I'd be around any day equally I think
15:31:44 <nirik> this week or next week would be good for me, the first week of june I am out...
15:31:53 <nirik> and the next week is the f2f meeting
15:31:58 <bowlofeggs> ah
15:32:19 <bowlofeggs> well i def wouldn't do a major backwards incompatible deployment on a thursday or friday haha
15:32:23 * nirik notes he doesn't need to be around, just noting it.
15:32:24 <bowlofeggs> so this week is no good imo
15:32:36 <bowlofeggs> but i could do it on tuesday if people don't mind it being a 4 day week
15:32:45 <smooge> do it on tuesday
15:32:47 <puiterwijk> I'd say that's fine
15:32:50 <bowlofeggs> cool
15:32:54 <nirik> +1
15:32:57 <smooge> if it doesn't work fudge it
15:33:10 <bowlofeggs> nothing ever works, so that's just par for the course ☺
15:33:14 <bowlofeggs> this_is_fine.jpg
15:34:45 <smooge> ok anything else for this week?
15:35:01 <smooge> I need to get an icepick removed from my head
15:36:02 <smooge> ok thank you all for coming this week. We have 2 meetings and then we will have a week of
15:36:09 <smooge> see you next week
15:36:10 <bowlofeggs> nirik: how important is it to you to switch bodhi to f30?
15:36:18 <bowlofeggs> that might delay things a bit
15:36:24 <bowlofeggs> because i haven't done any testing there
15:36:25 <nirik> bowlofeggs: meh, just a thought... nothing we have to do
15:36:28 <bowlofeggs> ok
15:36:35 <bowlofeggs> we can do that any time
15:36:38 <nirik> we could punt that
15:36:41 <bowlofeggs> doesnt' have to correspond to a bodhi release
15:36:45 * nirik nods
15:36:50 <bowlofeggs> migth atually be best to *not* do with a bodhi release
15:37:06 <bowlofeggs> so if there are problems we more easily know if its the change in OS or change in code
15:37:31 <bowlofeggs> perhaps we can do taht the following week (if you want - i'm still fine staying on f29 until EOL too)
15:37:43 <bowlofeggs> oh you are gone
15:37:48 <puiterwijk> bowlofeggs: so what's the ETA for 4.0.1? Tuesday night or Wednesday morning? :)
15:37:49 <bowlofeggs> well, we can do that when you aren't gone ☺
15:37:58 <bowlofeggs> puiterwijk: i sure hope not ☺
15:38:08 <smooge> #endmeeting