15:00:56 <smooge> #startmeeting Infrastructure (2019-03-21)
15:00:56 <zodbot> Meeting started Thu Mar 21 15:00:56 2019 UTC.
15:00:56 <zodbot> This meeting is logged and archived in a public location.
15:00:56 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:56 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:56 <zodbot> The meeting name has been set to 'infrastructure_(2019-03-21)'
15:00:56 <smooge> #meetingname infrastructure
15:00:56 <smooge> #topic aloha
15:00:56 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink threebean cverna mizdebsk mkonecny
15:00:56 <zodbot> The meeting name has been set to 'infrastructure'
15:00:56 <zodbot> Current chairs: cverna mizdebsk mkonecny nirik pingou puiterwijk relrod smooge tflink threebean
15:01:05 <mizdebsk> .hello2
15:01:06 <zodbot> mizdebsk: mizdebsk 'Mikolaj Izdebski' <mizdebsk@redhat.com>
15:01:12 <abompard> .hello2
15:01:13 <zodbot> abompard: abompard 'Aurelien Bompard' <aurelien@bompard.org>
15:01:13 <cverna> hello
15:01:20 <frailtyy> .hello2
15:01:21 <zodbot> frailtyy: frailtyy 'Garrett Jansen' <garrett@jansen.sh>
15:01:25 <frailtyy> :O
15:01:43 <mkonecny> .hello zlopez
15:01:45 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu>
15:02:18 <tflink> .hello2
15:02:19 <zodbot> tflink: tflink 'Tim Flink' <tflink@redhat.com>
15:02:22 <bowlofeggs> .hello2
15:02:23 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:02:34 <bowlofeggs> look at me, almost on time for once
15:02:58 <smooge> hello everyone
15:03:11 <smooge> #topic New folks introductions
15:03:11 <smooge> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:03:11 <smooge> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:05:02 <smooge> #topic announcements and information
15:05:02 <smooge> #info Beta Freeze Is Currently Going On. Please get FBR for frozen systems.
15:05:49 <nirik> morning
15:05:57 <mkonecny> hi nirik
15:05:57 * nirik got distracted and lost time
15:05:59 <bowlofeggs> #info bodhi 4.0.0 is on the horizon - we are wrapping up the last few tickets that we plan to include
15:06:35 <smooge> #info pagure 5.3.92 was installed on stg.pagure.io and src.stg.fpo
15:06:54 <smooge> #topic Oncall
15:06:55 <smooge> #info bowlofeggs is on call from 2019-03-14 -> 2019-03-21
15:06:55 <smooge> #info nirik is on call from 2019-03-21 -> 2019-03-28
15:06:55 <smooge> #info relrod is on call from 2019-03-28 -> 2019-04-04
15:06:55 <smooge> #info smooge is on call from 2019-04-04 -> 2019-04-11
15:06:56 <smooge> #info ????? is on call from 2019-04-11 -> 2019-04-18
15:06:57 <smooge> #info Summary of last week: (from bowlofeggs )
15:07:06 <smooge> bowlofeggs, how was oncall this week?
15:07:27 <mizdebsk> sorry for not summarizing my last call on the previous meeting, something come up and i was not able to join
15:07:36 <mizdebsk> my last oncall*
15:07:47 * bowlofeggs passes the ceremonial oncall coffee mug to nirik
15:08:13 <bowlofeggs> smooge: oncall was fairly quiet this round - i mostly just triaged bugs and none of them were urgent
15:08:33 * nirik drinks deep.
15:09:06 <bowlofeggs> i am probably the least useful oncall since i'm not a sysadmin though ☺
15:09:18 <bowlofeggs> i sometimes notice that people .oncall, see it's me, and then go "oh well" :)
15:09:21 <smooge> well thank you for your time. getting tickets traiged
15:09:28 <bowlofeggs> heh, yeah i think doing that is useful
15:09:33 <smooge> it was
15:09:35 <nirik> it does help...
15:09:38 <bowlofeggs> i can usually tell if soemthing is worth interrupting others or not
15:09:46 <bowlofeggs> which i think is the main goal
15:10:11 <bowlofeggs> (the secret: it's almost always not worth interrupting others!)
15:10:22 <smooge> #topic Monitoring discussion
15:10:22 <smooge> #info https://nagios.fedoraproject.org/nagios
15:10:22 <smooge> #info Go over existing out items and fix
15:11:26 * nirik looks
15:11:53 <smooge> pinging the abrt people about retrace servers
15:12:18 <nirik> yeah, send email...
15:12:40 <nirik> some disks needing replacement still...
15:12:49 <smooge> and our usual suspects
15:12:54 <nirik> pkgs02 no swap (because I tried to unswap/swap)
15:13:11 <smooge> I thought I rebooted that
15:13:14 <nirik> pkgs02 needs more space (but that should wait until after freeze)
15:13:15 <mizdebsk> i think pkgs02 needs more memory and disk
15:13:27 <nirik> yeah.
15:14:44 <smooge> nirik, did you do the swap/noswap after the last reboot?
15:15:47 <smooge> just checking in case there was a different problem with swap
15:16:07 <nirik> yes, but it is unable to swapon due to memory fragmentation
15:16:48 <smooge> ok thanks
15:16:51 <nirik> swapon: /dev/vda2: swapon failed: Cannot allocate memory
15:17:04 <smooge> #topic Tickets discussion
15:17:04 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:17:38 <nirik> I didn't mark anything.
15:17:43 <mkonecny> https://pagure.io/fedora-infrastructure/issue/7646
15:17:43 <smooge> #topic Priorities for next week?
15:17:43 <smooge> #info please put tickets needing to be focused on here
15:17:46 <nirik> if anyone has a pet ticket to discuss, feel free.
15:17:50 <smooge> oops sorry
15:17:55 <nirik> thats fine
15:18:06 <mkonecny> I tried it today and the fork is working
15:18:23 <mkonecny> But the forked repo is stuck as read-only
15:18:51 <mkonecny> With this message: "The permissions on this repository are being updated. This may take a while. During this time, you or some of the project's contributors may not be able to push to this repository."
15:19:45 <mkonecny> Sorry I forgot to mark the ticket before meeting
15:20:04 * nirik has no status there, smooge or puiterwijk might know.
15:20:40 <smooge> the updated version which was supposed to help fix this was applied earlier today
15:20:44 <puiterwijk> The read-only bit is a red herring, and a consequence of the former. The Hook ID is wrong
15:21:11 <smooge> I don't know why the hook-id is wrong because i thought I  set it 'right' using the commands earlier
15:21:18 <smooge> well not that fork
15:21:43 <mkonecny> Here is the fork, if you want to look at it - https://src.stg.fedoraproject.org/fork/zlopez/rpms/0ad
15:21:47 <puiterwijk> NEw forks get the Hook ID from config
15:22:25 <smooge> so after we updated the config to make sure it had the right hook-id in it, I ran that command which was to change all the forks to it
15:23:02 <puiterwijk> Right. Did you also restart all of the services that set this up? (one of the celery runners)
15:23:05 <mkonecny> smooge: This fork was created after the update
15:24:08 <puiterwijk> Anyway, we can get that fixed out-of-meeting
15:24:28 <smooge> puiterwijk, I did what you told me to do. Asking me if I didn't do something will just be 'did you tell me to do that?'
15:24:42 <puiterwijk> That is totalyl fair
15:25:02 <puiterwijk> Anyway, let's talk out-of-meeting. It should be simple to fix, just need to look what step I forgot
15:25:09 <smooge> ok focus this week:
15:25:29 <mkonecny> Ok, I just wanted to ask about it, it's blocking one of my work in progress issue
15:25:30 <smooge> 1. repospanner/pagure deployments for post freeze
15:25:47 <smooge> 2. rawhide gating items in staging
15:25:53 <smooge> 3. ???
15:26:16 <smooge> mkonecny, understood. we will fix it after this meeting
15:26:16 <nirik> things in no order for me: f30 beta, power9 installs, community openshift
15:26:16 <bowlofeggs> nothing in staging for rawhide gating yet
15:26:40 <smooge> 3. community openshift
15:26:42 <nirik> mizdebsk: oh, is the stg koji sync all done? I thought so, but didn't look...
15:26:42 <smooge> 4. power9
15:26:47 <bowlofeggs> bodhi 4.0.0 will have a few thigns for rawhide gating, but not enough to actually turn it on
15:27:04 <mizdebsk> nirik, it is done, but last time i checked there was a problem with nfs on s390x builder
15:27:25 <nirik> it shouldn't need nfs?
15:27:50 <mizdebsk> not for rpm builds, it may need it for images and stuff
15:27:56 * mizdebsk is not sure
15:27:57 <nirik> I guess it would if we wanted runroot/compose tasks... but we never have in the past.
15:28:04 <puiterwijk> 1. repospanner/pagure deployment help and repospanner fixes
15:28:44 <puiterwijk> 2. nothing until 1 is considered "done enough"
15:28:58 <nirik> speaking of, where are we on the ansible repospanner? still need to upgrade pagure.io and repospanner on batcave01?
15:29:12 <pingou> bowlofeggs: getting the new resultsdb and greenwave in staging would be good
15:29:17 <pingou> and it relates to rawhide gating
15:29:18 <puiterwijk> nirik: I'd say yes
15:29:29 <pingou> (the resultsdb PR that should improve things just got merged)
15:30:31 * pingou not sure if we can run our test scripts against greenwave in staging
15:30:44 <pingou> the koji db has been synced recently but is it new enough?
15:30:47 <pingou> I guess we'll see :)
15:31:11 <nirik> note that (I think) greenwave in stg is using prod data/listening to prod bus?
15:31:31 <pingou> resultsdb is for sure
15:31:35 <pingou> I'm not sure for greenwave
15:31:46 <smooge> i think it is also.
15:31:56 <pingou> then, should work :)
15:31:59 <pingou> cool
15:32:08 <nirik> well, it's querying src.stg git for objects that only exist in prod... ;)
15:32:08 <bowlofeggs> pingou: staging isn't very useful with greenwave, because there are no tests in staging
15:32:13 <mizdebsk> pingou, koji db in stg was synced on 2019-03-12 - it's pretty recent
15:32:17 <nirik> (and thus causing repospanner messages)
15:32:24 <bowlofeggs> i've long wanted tests in staging, but alas
15:33:24 <smooge> ok anymore items needing more than one person to focus on?
15:33:27 <pingou> bowlofeggs: resultsdb in staging is full of test results (from prod)
15:33:49 <pingou> nirik: that's likely going to be the issue yeah :(
15:33:51 <bowlofeggs> pingou: that's not what i need - i need to see tests on builds that i make in staging, so i can force failures, successes, etc.
15:34:08 <bowlofeggs> and use staging gating.yamls
15:34:25 <pingou> I agree this would be good to have
15:35:46 <smooge> ok I am going to move onto the next topic
15:35:54 <smooge> #topic Apprentice Open office minutes
15:35:54 <smooge> #info A time where apprentices may ask for help or look at problems.
15:36:21 <smooge> any items that new people are looking at wanting help on?
15:36:46 <smooge> I want to thank the person who gave a fix for the http certgetter
15:38:50 <nirik> they also sent another patch in ticket for another item. ;)
15:38:59 <nirik> hopefully we can get that one applied soon too.
15:39:06 <smooge> cool will go look
15:39:10 <smooge> #topic Open Floor
15:39:35 <smooge> Anything for the open floor htis week?
15:39:53 <smooge> If not I will give you back the rest of your day and will go eat some dinner
15:40:19 <nirik> breakfast, its whats for dinner!
15:40:35 <smooge> well I have been up for a whee bit
15:40:49 <smooge> you on the other hand are at 7:40?
15:40:58 <bowlofeggs> i ate dinner at america's restaurant last night
15:41:00 <nirik> 8:40now...
15:41:08 <smooge> dennys?
15:41:13 <bowlofeggs> smooge: yep!
15:41:28 <smooge> I am an internationalist myself
15:41:28 <bowlofeggs> cinammon roll pancakes. devastatingly delicious
15:41:34 <nirik> they have the silliest name for a dish I think anywhere...
15:41:43 <bowlofeggs> the slam?
15:41:44 <nirik> Moons over my hammy
15:41:48 <bowlofeggs> oh yeah, haha
15:41:50 <frailtyy> moons over my hammy is a good touch
15:41:59 <frailtyy> ive always been a grand slammer
15:42:08 <nirik> https://www.dennys.com/food/breakfast-faves/moons-over-my-hammy/
15:42:25 <smooge> thankfully they never went with my idea of Sam and Ella Eggs
15:42:25 <bowlofeggs> it used to be called Danny's originally
15:42:31 <bowlofeggs> smooge: lol
15:42:57 <smooge> ok and with that.. good night folks. Please tip the wait staff on your way out.. they have to put up with me
15:43:03 <smooge> #endmeeting