15:01:48 <smooge> #startmeeting Infrastructure (2019-02-21)
15:01:48 <zodbot> Meeting started Thu Feb 21 15:01:48 2019 UTC.
15:01:48 <zodbot> This meeting is logged and archived in a public location.
15:01:48 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:48 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:48 <zodbot> The meeting name has been set to 'infrastructure_(2019-02-21)'
15:01:48 <smooge> #meetingname infrastructure
15:01:48 <smooge> #topic aloha
15:01:48 <zodbot> The meeting name has been set to 'infrastructure'
15:01:48 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink threebean cverna mkonecny mizdebsk
15:01:48 <zodbot> Current chairs: cverna mizdebsk mkonecny nirik pingou puiterwijk relrod smooge tflink threebean
15:02:01 <tflink> morning
15:02:03 <nirik> morning.
15:02:06 <mkonecny> .hello zlopez
15:02:07 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu>
15:02:07 <smooge> morning everyone
15:02:10 <pingou> .hello
15:02:12 <zodbot> pingou: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
15:02:18 <pingou> meh
15:02:21 <pingou> .hello pingou
15:02:22 <mizdebsk> .hello2
15:02:22 <zodbot> pingou: pingou 'Pierre-YvesChibon' <pingou@pingoured.fr>
15:02:25 <zodbot> mizdebsk: mizdebsk 'Mikolaj Izdebski' <mizdebsk@redhat.com>
15:05:02 <smooge> #topic New folks introductions
15:05:02 <smooge> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:05:02 <smooge> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:05:10 <smooge> Hello any new people this weke?
15:05:47 <nirik> I'm so old I could be new again? :)
15:05:49 * nirik runs
15:06:40 <smooge> nirik, you got hit by the 1970-01-01 fedmsg bus?
15:07:09 <nirik> I even predate that. :) but I digress...
15:07:17 <smooge> #topic announcements and information
15:07:17 <smooge> #info nirik will have sparse hours due to house move
15:07:17 <smooge> #info Mass branching happened and it was ...
15:07:17 <smooge> #info Mass update/reboots planned for 2019-02-26 -> 2019-02-28
15:07:17 <smooge> #info Beta Freeze Begins 2019-03-05
15:07:18 <smooge> #info Anitya (release-monitoring.org) 0.15.0 was released and deployed on staging
15:07:20 <smooge> #info Pagure 3.2.90 was deployed to stg.pagure.io. Please test
15:07:22 <smooge> #info Pagure 3.2.90 was deployed to src.stg.fedoraproject.org. some manual work was needed
15:07:24 <smooge> #info Taskotron Staging re-deployed on F29, mostly working
15:07:38 <pingou> 5.2.93 even :)
15:07:53 <smooge> you can tell I wrote this earlier this week
15:08:03 <nirik> oh cool.
15:08:11 <puiterwijk> #info Koji 1.16.2 was deployed today. No new features, just a CVE fix.
15:08:13 <nirik> did that include repospanner 0.4?
15:08:18 <smooge> no
15:08:22 <nirik> ok
15:08:32 <smooge> repospanner 0.4 may be tomorrow or Monday
15:09:08 <nirik> we should schedule a mass update/reboot next week... tue/wed or just marathon wed.
15:09:28 <smooge> Yep. I would prefer Tue/Wed
15:09:42 <pingou> puiterwijk: nice cve
15:09:43 <smooge> Unless we are going to do it during working hours
15:10:20 <smooge> because I am finding that working after 00:00 UTC hard
15:10:28 <nirik> I probibly will finally be moving next week (I sure hope), but still unclear when, and I can help as time permits
15:11:37 <nirik> might know more tomorrow, or we could just schedule tue/wed and I can try and help as much as I can. .
15:11:45 <bowlofeggs> .helo2
15:11:47 <bowlofeggs> .hello2
15:11:48 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:11:54 <bowlofeggs> (sorry i'm late)
15:12:22 <smooge> ok that does change the calculations a bit
15:12:28 <smooge> puiterwijk, how does your next week look?
15:12:44 <puiterwijk> smooge: hopefully better than the last few?
15:12:47 <smooge> you are on call so it would be your say on what days and times you want to deal with outages
15:12:50 <bowlofeggs> #info bodhi-3.12.{0,1,2} were deployed this week (1 and 2 were sad)
15:12:55 * tflink plans to be around to help, at least with the qa stuff
15:13:14 <bowlofeggs> #undo
15:13:16 <smooge> and I realized.. I need to do this later in the meeting
15:13:17 <bowlofeggs> #info bodhi-3.13.{0,1,2} were deployed this week (1 and 2 were sad)
15:13:18 <puiterwijk> smooge: meh, whatever. I'll end up dealing with it anyway probably.
15:13:38 <smooge> so let's put off scheduling in the section where we do that
15:13:52 <smooge> #topic Oncall
15:13:52 <smooge> #info smooge is on call from 2019-02-14 -> 2019-02-21
15:13:52 <smooge> #info puiterwijk is on call from 2019-02-21 -> 2019-02-28
15:13:52 <smooge> #info ?????? is on call from 2019-02-28 -> 2019-03-07
15:13:53 <smooge> #info ?????? is on call from 2019-03-07 -> 2019-03-14
15:13:54 <smooge> #info ?????? is on call from 2019-03-14 -> 2019-03-21
15:13:55 <smooge> #info Summary of last week: (from smooge )
15:14:09 <smooge> So last week had a couple of issues.. almost all of them were fixed by mizdebsk
15:14:11 * relrod gets home from class in time for meeting for once! Ooh yay just in time to be given on-call duties :P
15:14:44 <smooge> thank you very much mizdebsk
15:14:55 <pingou> mizdebsk++
15:14:55 <zodbot> pingou: Karma for mizdebsk changed to 17 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:15:07 <nirik> yes indeed. appreciated. ;)
15:15:13 <mkonecny> nice
15:15:18 <mizdebsk> my pleasure
15:15:37 <bowlofeggs> mizdebsk++
15:16:14 <smooge> mizdebsk, is going to help me on getting something in to have fedmsg-hub and fedmsg-hub3 restart if it grows too much in memory
15:16:26 <smooge> and then our large amount of pages will be that
15:16:41 <smooge> #topic Monitoring discussion
15:16:41 <smooge> #info https://nagios.fedoraproject.org/nagios
15:16:41 <smooge> #info Go over existing out items and fix
15:16:59 <nirik> I think it's mostly similar to last week.
15:17:02 * nirik looks
15:17:30 <nirik> someone could power back up kvm01 (not that we use it much)
15:18:08 <nirik> osbs-master01.stg is firewall (fixing)
15:18:35 <nirik> perhaps we should move back to autosign01 later today?
15:18:40 <smooge> should we turn swamp back on pkgs02.phx2.fedoraproject.org
15:19:08 <relrod> nirik: autosign01 is on my list, will work on it today
15:19:17 <nirik> relrod: cool.
15:19:34 <nirik> smooge: if you can, I tried to clear it, but it won't go back on because memory is too fragmented.
15:19:45 <smooge> ah ok.. that is a reboot then
15:19:48 <nirik> swapon: /dev/vda2: swapon failed: Cannot allocate memory
15:20:00 <smooge> which it iwll get next week
15:20:13 <nirik> we should remove the askbot fedmsg check
15:20:34 <smooge> ok will do that
15:20:50 <nirik> ppc8-02/03 and virthost06 have disks to replace
15:21:01 <mizdebsk> smooge, did you have time to look at openshift nagios plugin?
15:21:31 <smooge> no sadly the proxy 503 issue ate all my time
15:21:38 <smooge> which reminds me
15:21:47 <mizdebsk> no problem
15:22:03 <smooge> #info mirrorlist proxies are much lower on 503's but proxy01 still generates much more than any other
15:22:14 <smooge> #topic Tickets discussion
15:22:14 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:22:38 <smooge> No one marked a ticket for discussion so we will move to the next.
15:22:44 <pingou> nice work on the 503s smooge :)
15:23:07 <nirik> how much did it decrease on non proxy01?
15:23:09 <smooge> Please people (and audience members) if you have a ticket that you want us to work on/talk beyond anything else.. please mark it
15:23:22 <smooge> from several thousand to 0
15:23:45 <smooge> but I want to talk about that next
15:23:55 <mizdebsk> and if you can't mark the ticket for meeting, you can add a comment asking us to do so
15:23:58 <smooge> #topic Priorities for next week?
15:23:58 <smooge> #info please put tickets needing to be focused on here
15:24:06 <smooge> mizdebsk++
15:24:21 <smooge> OK the top priority for next week will be updates/reboots
15:24:42 <pingou> https://pagure.io/fedora-infrastructure/issue/7523 may be nice to have
15:24:54 <pingou> so we can move ahead with testing some of the already ported apps in stg
15:25:03 <mkonecny> +1
15:25:25 <nirik> I was going to do that, but I had questions... which folks have replied to, but I haven't gotten back around to it.
15:25:28 <mkonecny> Anitya and the-new-hotness are waiting for this already :-)
15:25:57 <smooge> There are a TON of packages queued up for systems.. so I am actually going to ask to update staging hosts sooner and test that we don't have a BOOM
15:25:58 <nirik> basically how we want to name them: $service, $fqdn, something else
15:26:01 <bowlofeggs> i wouldn't mind seeing some movement on the rabbitmq certificates
15:26:05 <pingou> nirik: thanks for that :)
15:26:17 <bowlofeggs> bodhi in production *should* be able to publish with fedora-messaging (but we should test in stg first, of course ☺)
15:26:33 <pingou> pagure 5.3 comes with fedora-messaging support as well
15:26:34 <nirik> it sounds like we want to do $service I suppose...
15:26:35 <puiterwijk> nirik: let's call them $service.
15:26:41 <pingou> (to be tested as well)
15:26:47 <nirik> that seems... less exact, but ok
15:27:02 <smooge> $service.$fqdn ?
15:27:13 <nirik> is there a schema? perhaps we could name them by that?
15:27:18 <puiterwijk> smooge: well, the thing is that we don't often have an fqdn...
15:27:30 <nirik> org.fedoraproject.bodhi or whatever
15:27:32 <puiterwijk> Due to the fact that many things are moving to openshift. And in addition, as Jeremy said, per-host certs add nothing
15:27:41 <puiterwijk> Sure, that's fine.
15:28:00 <smooge> $domainname.$service ?
15:28:05 <pingou> io.pagure.pagure?
15:28:24 <puiterwijk> I would suggest to not make it harder than need be...
15:28:24 <nirik> if there's a source of truth for that, I can use that...
15:28:47 <nirik> or we can just do servicename I guess.
15:28:51 <nirik> we can always redo it
15:28:55 <pingou> +1
15:28:58 <smooge> ok I am going to give people 1 more minute to call out shed colours
15:28:59 <mkonecny> +1
15:29:03 <puiterwijk> Just servicename is fine I'd say. Nobody else will see it in the messages anyway
15:29:08 <nirik> well, $servicename$env
15:29:13 <pingou> +1
15:29:15 <puiterwijk> It's just username for rabbit.
15:29:22 <pingou> pagure.stg - srcfp.stg
15:29:22 <nirik> we do want seperate stg and prod certs right? they are different cas I think?
15:29:43 <nirik> so we could just name them the same, but that might be confusing
15:30:15 <smooge> nirik, puiterwijk would any eventual leakage between environments cause problems?
15:30:16 <mkonecny> How it is done now for fedmsg?
15:30:21 <smooge> because we know they will mix
15:30:23 <puiterwijk> Different CAs, yeah. But adding .stg or .prod might be useful for quick determination
15:30:31 <puiterwijk> smooge: there's entirely separate CA chains.
15:30:38 <puiterwijk> So the stg cert just will not work for prod
15:30:48 <nirik> mkonecny: mostly fqdn, but with openshift... thats not very useful
15:31:07 <nirik> right, so I will do $servicename.stg for stg and $servicename for prod
15:31:15 <mizdebsk> with fedmsg, staging services sometimes listen to prod messages; we don't want to allow it with fedora-messaging?
15:31:16 <smooge> puiterwijk, someday someone will sign something wrong.. OR we will have one of our systems which needs to be on both
15:31:17 <nirik> (so we can use the ansible $env thing)
15:31:25 <puiterwijk> Sure
15:31:36 <puiterwijk> smooge: if it needs to be on both, it'll need to connect to two brokers anyway
15:31:54 <puiterwijk> i.e. two different entire connection chains, which includes certs
15:31:55 <nirik> mizdebsk: we can... I guess for those we would do specific more descriptive certs?
15:32:12 <puiterwijk> I *think* that we had plans to do a broker-shovel.
15:32:17 <nirik> openqa-stg-listening-on-prod
15:32:24 <puiterwijk> So have the prod broker send stuff to the stg broker
15:32:25 <nirik> or that could work
15:32:43 <smooge> ok anything more on this?
15:32:47 <puiterwijk> And then apps can just ignore or not specific topics
15:32:51 * nirik will get these today
15:33:12 <smooge> OK I marked the ticket as next-meeting just to say it was this meeting
15:33:32 <smooge> thanks nirik and everyone else
15:33:39 <bowlofeggs> nirik: let's deploy to prod at EOB friday in your TZ
15:33:51 <smooge> Any other high priority items for next week?
15:34:07 <bowlofeggs> smooge: yeah, is next week going to be the week of the linux desktop?
15:34:20 <smooge> that can wait for open flood
15:34:29 <tflink> bowlofeggs: isn't it always?
15:34:37 <smooge> #topic Discuss: Is the Fedora pastebin still useful? - relrod
15:34:37 <smooge> #info how many users are using it?
15:34:37 <smooge> #info should we look at converging with CentOS one so simpler setup?
15:34:41 <smooge> relrod, you are up
15:34:42 <bowlofeggs> haha
15:35:00 <bowlofeggs> i use the pastebin sometimes, fwiw
15:35:04 <relrod> So our pastebin has become a cesspit of spam
15:35:04 <nirik> I use it all the time
15:35:08 <bowlofeggs> so i'd say yes
15:35:12 * pingou uses it most frequently
15:35:15 <relrod> and we are also duplicating efforts with centos who runs one
15:35:26 <bowlofeggs> i would also say that there's no reason people couldn't just use another public one though
15:35:37 <bowlofeggs> and yeah, duplicating with centos is not ideal either
15:35:39 <nirik> relrod: can fpaste work with the centos one?
15:35:40 <smooge> I would like to offer the following optin
15:35:43 <bowlofeggs> but why should either project host one?
15:35:48 <pingou> I'm fine with consolidate on one
15:35:49 <nirik> and can the centos one be banded/answer for ours?
15:35:57 <relrod> Also upstream is... somewhat dead? They talked about doing a rewrite/version 2 of it and then went silent
15:36:04 <smooge> just run two instances of the same thing
15:36:09 <nirik> sadly the way of such things.
15:36:25 <tflink> relrod: isn't this the second or third upstream that we've used?
15:36:31 <relrod> tflink: yeah :(
15:36:42 <smooge> we run with Bahhumbug's playbooks and modify fpaste to work with the php stikked
15:36:46 <tflink> yikes, short lifetimes
15:36:49 <relrod> nirik: fpaste right now can't, but I think it would be pretty easy to get it to hit the centos one
15:37:12 <pingou> relrod: the cli ?
15:37:17 <relrod> pingou: yeah
15:37:20 <pingou> ok
15:37:27 <smooge> the CentOS one is setup to flush everything after 1 day to deal with spam
15:37:32 <bowlofeggs> is there a reason for either project to host one of these thought?
15:37:43 <bowlofeggs> there are other hosted one on the interweb right?
15:37:46 <nirik> if there was a nice one we could run in openshift that could be an option, but of course we would still be running it.
15:37:46 <smooge> A LOT of people use them
15:37:58 <nirik> bowlofeggs: branding I guess?
15:38:06 <pingou> even spammers, and that's telling something :)
15:38:07 <bowlofeggs> i know people use them, but why do they need to use ours and not someone else's?
15:38:14 <bowlofeggs> yeah branding
15:38:15 <nirik> show your pride in fedora by using paste.fedoraproject.org or something?
15:38:30 <bowlofeggs> it's fun to host stuff to an extent, but this doesn't seem like a wheel house to me
15:38:34 <smooge> bowlofeggs, the reason Fedora community asked us was for 2 reasons: 1 branding and 2 they trusted us more than most of the paste sites
15:38:38 <bowlofeggs> what even is a wheel house…
15:38:38 <nirik> also we have a cli and I don't know how some places would feel about allowing it?
15:38:50 <bowlofeggs> must be a water wheel thing?
15:39:02 <nirik> bowlofeggs: the top cabin of the ship with the wheel that directs it and the captain.
15:39:04 <bowlofeggs> the cli is nice for sure
15:39:07 <bowlofeggs> i use it
15:39:08 <smooge> a lot of those other sites go dodgy with ads and otehr stuff
15:39:12 <pingou> bowlofeggs: http://taylormarshall.com/wp-content/uploads/2013/09/hamster-ball.jpg
15:39:14 <bowlofeggs> nirik: oooh
15:39:16 <bowlofeggs> haha
15:39:30 <bowlofeggs> pingou: haha
15:39:33 <nirik> yeah, there's a bunch with ads and trackers and other malware
15:39:44 <bowlofeggs> well i'm not against hosting it, just thought it was worth asking the question
15:39:50 <bowlofeggs> doesn't affect me really ☺
15:39:52 <smooge> so we were asked to set this up to avoid that for fedora qa and other related tasks
15:39:57 <bowlofeggs> i do use ours for sure and i like it
15:40:00 <nirik> if the centos one could be branded or we could both run the same one that might be good
15:40:17 <smooge> info bowlofeggs is now the PM for the paste project because it didn't affect him before
15:40:17 <pingou> how does the centos one look like?
15:40:21 <bowlofeggs> lol
15:40:24 <bowlofeggs> hahahah
15:40:25 <pingou> https://paste.centos.org/
15:40:36 <tflink> I suspect that this question has been asked but would it be possible to re-purpose something like spamassassin to help filter out some of the spam?
15:40:44 <bowlofeggs> is anyone from centos here today? bstinson?
15:40:50 <smooge> tflink, the stikked has several plugins
15:40:52 <nirik> tflink: possible, but work...
15:41:04 <relrod> tflink: possibly but then we diverge from upstream and have to maintain a fork of modernpaste
15:41:06 <smooge> but it is a constant job of Bahhumbug on that
15:41:22 <bowlofeggs> relrod: we might be able to contribute to upstream
15:41:24 <smooge> tflink, it is a lot of work.
15:41:26 <tflink> nirik: yeah, I hadn't gotten to the "would it be worth the effort" part :)
15:41:26 <bowlofeggs> i'm sure that'd be generally beneficial
15:41:27 <nirik> proposal: gather info for a week and discuss next time?
15:41:35 <bowlofeggs> of course, writing it at all takes $resources
15:41:52 <bowlofeggs> nirik: +1
15:41:52 <smooge> +1 to nirik's proposal
15:42:00 <relrod> nirik: sure. I usually have class at meeting time, but if someone else can bring it up next week, +1
15:42:10 <smooge> we can and should discuss this on the list then
15:42:15 <nirik> relrod: can you do the gathering of info? or would you like someone else to?
15:42:16 <pingou> the branding on the CentOS paste is minimal
15:42:25 <smooge> relrod, I think you opened up an email thread on this previously?
15:42:26 <pingou> so we could likely just replace CentOS by Fedora and done
15:42:47 <relrod> smooge: Not that I recall, at least not recently
15:42:52 <nirik> if we run a Stikked in openshift we could make it use EmprtyDir for storage, and just gets wiped everytime we deploy.
15:43:16 <pingou> nirik: :s may make paste dispear sooner than a day
15:43:16 <bowlofeggs> it might be even possible to do some proxy server tricks where it's the same paste instance, but depending on the CNAME you use you see a different logo
15:43:20 <smooge> pingou, the centos site is incredibly simple set up and most of the options are turned off to make it fast as possible
15:43:32 <nirik> they don't seem really active either...
15:43:41 <smooge> no paste is active
15:44:03 <pingou> smooge: do we need/want any of the options that are actually off?
15:44:06 <relrod> bowlofeggs: or just do a theme that has both logos somehow and make paste.fp.o and paste.c.o both work, if they are amenable to that
15:44:13 <smooge> pingou, I don't think so
15:44:20 <bowlofeggs> we could use twitterfs to store the pastes in twitter as tweets! then we don't have to solve storage issues in openshift!
15:44:29 <bstinson> there are a couple of things we do to guard against crawling/indexing too
15:44:33 <smooge> ok guys.. we have a motion
15:44:42 <bstinson> but smooge is better connected about this than i am
15:45:00 <smooge> well I just pinned Bahhumbug down last week on it
15:45:11 <relrod> nirik: I can try to gather some info...
15:45:37 <nirik> cool. If you like I can assist, or stay out of your way
15:45:50 <nirik> I think we have more options that we did before now...
15:45:50 <smooge> relrod,what nirik just said
15:46:14 <nirik> because if we just run it in openshift we can avoid packaging it
15:46:40 <nirik> so we could even look at hastebin or something
15:46:48 <smooge> #info relrod will gather more info on pastes and we will discuss options next week
15:47:06 <relrod> My main thing is the spam problem. The 7 day thing hacks around that, but makes it less useful imo (there are pastebins online with no expiry). So just wanted to toss the question out.
15:47:30 <pingou> I'm fine with up to 2 or 3 days
15:47:32 <nirik> yeah, might be worth looking at what other ones do for that...
15:47:42 <pingou> centos offers 1 which may be a little short
15:47:43 <nirik> it must be a common problem over them all
15:47:56 <smooge> most of the pastebins don't care about spam
15:48:12 <smooge> the ones with ads actually like it because they make money off it
15:48:20 <nirik> many of them have anti-features we don't want either like password protected or client encrypted
15:48:26 <mizdebsk> perhaps we could offer 1 day to anonymous users and more to authenticated users
15:49:04 <smooge> ok lets put those ideas on the future email thread
15:49:07 <relrod> mizdebsk: well that's the other thing, we never got auth working on modernpaste. puiterwijk had a patch that integrated with oidc stuff, but ran out of cycles to work on it
15:49:21 <smooge> we are coming up to an hour and I would like to see if we have other items to discuss
15:49:27 <relrod> smooge: +1
15:49:41 <smooge> #topic mirrorlist 503's
15:50:00 <smooge> #info we had a big increase in 503's on proxy01 starting 2019-01-20
15:50:31 <smooge> #info we worked on a couple of wsgi patches which dropped the number in 1/2 on proxy01
15:51:19 <nirik> was 2019-01-20 the day we moved it to f29?
15:51:20 <smooge> #info the other hosts which have large numbers are in EU where they get hit more due to DNS
15:51:52 <smooge> nirik, I don't know.. I was focusing on some other items at the time
15:52:02 <nirik> nope, it was moved in december
15:52:16 <smooge> when did the mirrorlist image get updates
15:53:16 <smooge> at the moment I am mostly writing about this to document in the meeting minutes
15:53:22 <nirik> "2 months ago"
15:53:47 <nirik> is it worth checking if we have a single ip or network hitting things hard? (that showed up on 01-20...)
15:53:59 <puiterwijk> nirik: "docker image inspect"?
15:54:20 <nirik> "created": "2018-10-15T11:04:36Z",
15:54:28 <smooge> when I did it.. it was all over the place
15:54:30 <nirik> then the next change was the ones we did about a week ago
15:54:45 <smooge> basically at the beginning of the hour 20,000 systems checkin
15:55:36 <smooge> they hit the server at the end of the :59 minute to the beginning of the :00 minute and then some number of them get a BOOM 503
15:55:49 <bowlofeggs> i've g2g to another meeting - have a great day everyone!
15:55:56 <smooge> see you bowlofeggs
15:56:22 <smooge> I will put more info in another email. I think we have run out of time to discuss this longer
15:56:29 <smooge> #topic Open Floor
15:57:30 <smooge> anyone have anything for the floor..
15:57:47 <smooge> if not, thank you all for coming this week. I hope to see you all next week
15:57:48 <mklvr> I was previously a member of the fedora-apprentice group but was removed due to inactivity. I think I'm going to have some spare time coming up and was hoping I could get readded to that group.
15:58:15 <smooge> ok get with me after the meeting in #fedora-admin
15:58:17 <nirik> I'll hopefully have some items next week on community openshift, but we can save that for next week. ;)
15:58:29 <mklvr> I wanted to look into #7376 among others.
15:58:30 <smooge> nirik, I was going to say 2 weeks
15:58:37 <smooge> because next week is going to be boxes
15:58:41 <smooge> sooo many boxes
15:58:58 <nirik> yeah, and me moving probibly, or imploding due to not moving
15:59:31 <smooge> #endmeeting