15:02:58 <smooge> #startmeeting Infrastructure (2018-09-13)
15:02:58 <zodbot> Meeting started Thu Sep 13 15:02:58 2018 UTC.
15:02:58 <zodbot> This meeting is logged and archived in a public location.
15:02:58 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:58 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:02:58 <zodbot> The meeting name has been set to 'infrastructure_(2018-09-13)'
15:02:59 <smooge> #meetingname infrastructure
15:02:59 <zodbot> The meeting name has been set to 'infrastructure'
15:02:59 <smooge> #topic aloha
15:02:59 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink threebean
15:02:59 <zodbot> Current chairs: nirik pingou puiterwijk relrod smooge tflink threebean
15:03:13 <bcotton> .hello2
15:03:14 * relrod waves
15:03:14 <zodbot> bcotton: bcotton 'Ben Cotton' <bcotton@redhat.com>
15:03:15 <bowlofeggs> .hello2
15:03:16 <zodbot> bowlofeggs: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:03:21 <aikidouke> ./hello2
15:03:34 <aikidouke> crud lol
15:03:38 <aikidouke> .hello2
15:03:39 <zodbot> aikidouke: aikidouke 'None' <zach@mailup.net>
15:03:41 * cverna is around
15:03:47 <queue-tip> .hello2
15:03:47 <zodbot> queue-tip: Sorry, but you don't exist
15:03:58 <smooge> NOTE: I will be stopping this meeting at :55 because the next meeting starts at 00
15:03:59 * mkonecny here
15:04:06 <queue-tip> I always wondered what that command does
15:04:23 <mkonecny> .hello2
15:04:24 <zodbot> mkonecny: Sorry, but you don't exist
15:04:41 <smooge> #topic New folks introductions
15:04:41 <smooge> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:04:42 <nirik> morning
15:05:29 <smooge> hello any new people here this week?
15:06:34 <smooge> #topic announcements and information
15:06:35 <smooge> #info tflink is on extended PTO
15:06:35 <smooge> #info Beta Freeze is in effect. All changes to frozen systems in Infrastructure will require +1
15:06:35 <smooge> #info Bodhi 3.10.0 beta in staging https://bodhi.stg.fedoraproject.org/docs/user/release_notes.html
15:06:35 <smooge> #info odd network problem on arm network worked around.
15:06:36 <smooge> #info odd install problem on aarch64 systems is ongoing.
15:06:38 <smooge> #info Anitya 0.13.0 deployed on staging https://stg.release-monitoring.org - mkonecny
15:06:40 <smooge> #info Fedora Product Center repo created https://github.com/fedora-infra/fdc - cverna
15:06:52 <smooge> been a busy week so far
15:06:55 <smooge> any other announcements
15:07:16 <smooge> #info Hurricane Florence may affect some servers like fedorapeople
15:07:46 <bowlofeggs> queue-tip, mkonecny: the .hello2 command looks you up in Fedora's Account System (FAS) by your IRC nick and then says your name to everyone
15:07:59 <bowlofeggs> so it requires you to have a FAS account, and for your IRC nick to be registered with it
15:08:26 <jcline> #info fedora-messaging-1.0.0b1 is available in infra-stg repos as far back as epel7 and in rawhide, f29 Fedora repos
15:08:43 <smooge> cool
15:08:58 <smooge> jcline, that updates/replaces fedmsg?
15:09:09 <smooge> or something completely different
15:09:10 <mkonecny> bowlofeggs: Strange, I have it there
15:09:33 <jcline> smooge, basically
15:10:02 <jcline> Replaces, that is
15:11:35 <bowlofeggs> mkonecny: yeah it does look like you have it set up correctly to me
15:11:37 <smooge> cool
15:11:37 <bowlofeggs> weird
15:11:42 <smooge> #topic Oncall
15:11:42 <smooge> #info Relrod is on call from 2018-09-06->2018-09-13
15:11:42 <smooge> #info Smooge is on call from 2018-09-13->2018-09-20
15:11:42 <smooge> #info Nirik is on call from 2018-09-20->2018-09-27
15:11:42 <smooge> #info ??? is on call from 2018-09-27->2018-10-04
15:11:42 <mkonecny> jcline: I wanted to look at recording from the workshop about fedora-messaging that was on the Fedora Flock, but it isn't on youtube
15:11:43 <smooge> #info Summary of last week: (from Relrod)
15:12:00 <relrod> It's been pretty quiet, probably because of freeze
15:12:03 <smooge> cool
15:12:14 <jcline> mkonecny, I think there wasn't any recording equipment in the room we were in
15:12:25 <smooge> I will take over after this meeting. I would like to have an alternative because I may be without power
15:12:38 <relrod> I can be backup
15:12:50 <mkonecny> jcline: :-(
15:13:24 <smooge> thanks relrod
15:13:31 <smooge> ok next up
15:13:42 <smooge> #topic recent openshift additions - kevin
15:13:47 <smooge> nirik, you are up
15:14:05 <nirik> oh yeah, wanted to mention my recent stuff with openshift stg...
15:14:22 <nirik> I enabled metrics and logs... so now we can see cpu/load/etc.
15:14:35 <nirik> and look at logs, but it's kinda complex... kibana
15:14:57 <nirik> Also multitenant is enabled in stg (meaning all projects are network isolated from each other)
15:15:11 <nirik> I'm going to roll these out to prod after freeze.
15:15:15 <cverna> nirik++
15:15:17 * nirik digs up some urls.
15:15:48 <nirik> oh, and puiterwijk fixed logs and such from the web interface, but non firefox as firefox does something bad with websockets.
15:16:32 <nirik> https://kibana.app.os.stg.fedoraproject.org/app/kibana
15:16:44 <bowlofeggs> nirik: could the multitenant stuff be used to move bodhi staging into prod openshift, while continuing to keep it isolated from production stuff (like db)
15:16:47 <nirik> the monitoring appears next to each app (small graphs)
15:17:03 <nirik> bowlofeggs: well, partly, but it still needs more...
15:17:16 <bowlofeggs> cool
15:17:27 <bowlofeggs> if that is achievable, it could be nice ☺
15:17:28 <nirik> it needs egress stuff to make sure it goes out on the stg network too... but that does exist.
15:17:59 <nirik> you can basically set that per node...
15:18:06 <nirik> and then label that node as say 'staging'
15:18:15 <nirik> and then tell the staging apps to use that label
15:18:26 <bowlofeggs> ah interesting
15:18:27 <nirik> so we will get there, just more testing and working
15:18:38 <bowlofeggs> so the pods can't be on the same host as prod, but can be on the same openshift deployment
15:18:59 <bowlofeggs> cool, i look forward to it (no rush, it's a nice to have ☺)
15:19:00 <smooge> but pods get moved around at times right?
15:19:21 <nirik> yeah, but based on node selector/label.
15:19:34 <nirik> or you can tell it only to put those on nodes labeled etc.
15:19:36 <smooge> we would need more than 2 nodes then?
15:19:40 <nirik> yep.
15:19:56 <nirik> I added 2 nodes in stg also... it has 4 now.
15:20:01 <smooge> ok cool..
15:20:06 <nirik> oh and https://registry-console-default.app.os.stg.fedoraproject.org/registry should now work too
15:20:11 <nirik> so you can see images/etc
15:20:43 <nirik> Anyhow, I'd love for us to start moving more things to openshift.
15:20:51 <nirik> perhaps we could do a VFAD after f29 is out...
15:21:33 <smooge> cool
15:21:34 <smooge> ok
15:21:38 <smooge> next up?
15:21:46 <smooge> #topic python3 porting help needed for supybot-plugins - kevin
15:21:58 <smooge> hey nirik.. can you come back?
15:22:18 <nirik> sure!
15:22:33 <nirik> so, we have several supybot plugins we use in zodbot.
15:22:44 <nirik> they are currently python2 and there's bugs to retire them in f30
15:22:54 <nirik> it would be nice if we could port them to python3 and prevent that
15:23:25 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=1627142
15:23:33 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=1627228
15:23:45 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=1627225
15:23:55 <nirik> so, any help welcome. it should be pretty easy I hope
15:24:21 <cverna> that could be some good material for the office hours if an apprentice want to look at it
15:24:31 <nirik> indeed
15:25:19 <aikidouke> would like to do take one or more - looks like there are some comments from Miro on the first one?
15:25:35 <nirik> yeah, miro offered to help too
15:26:20 <aikidouke> ok - I'll follow the bugs and see if I can do anything to help :)
15:26:30 <nirik> thanks!
15:27:05 <mkonecny> Are there any tests ready for python3?
15:27:56 <nirik> tests for these? or apps in general ?
15:27:57 <x3mboy> .hello2
15:27:57 <zodbot> x3mboy: x3mboy 'Eduard Lucena' <eduardlucena@gmail.com>
15:28:33 <smooge> mkonecny, ^^
15:29:07 <cverna> aikidouke: let me know if you need help :)
15:29:12 <mkonecny> nirik: in general
15:29:24 <aikidouke> cverna++ ty!
15:29:54 <smooge> normally tests for apps have been the province of the developer of the app
15:30:20 <nirik> yeah... any normal tests should work I would think... nothing specific I know of to test python3 ability
15:30:52 <mkonecny> nirik: thanks, this is what I wanted to know
15:31:07 <smooge> #topic Tickets discussion
15:31:08 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:31:22 <smooge> .ticket 7233
15:31:24 <zodbot> smooge: Issue #7233: RFE: Machine-readable freeze flag? - fedora-infrastructure - Pagure - https://pagure.io/fedora-infrastructure/issue/7233
15:32:25 <smooge> so this is a discussion on how to know when we are in freeze or not
15:32:25 <nirik> threebean asked for this...
15:32:29 <nirik> yep.
15:33:19 <smooge> i can see that if we put it in PDC replacement then both ops, dev, qa and releng can all query one place for what they are doing
15:33:31 <smooge> but so would be having it in the ansible tree
15:34:14 <bowlofeggs> it could be a nice feature for bodhi too, because bodhi could ask "are you sure?" if someone tries to push during a freeze
15:34:16 <smooge> I think we can do the ansible tree now and the PDC thing in how many months?
15:34:42 <bowlofeggs> honestly the ansible tree is probably good enough for a permanent solution
15:34:57 <cverna> pdc replacement is officially FDC (Fedora Product Center) :)
15:35:02 <bowlofeggs> it's just a URL for some json/yaml/toml/data_type_of_the_month
15:36:34 <smooge> yep
15:36:59 <smooge> anything more here?
15:37:45 <smooge> .ticket 7226
15:37:47 <zodbot> smooge: Issue #7226: DNS for fedoraproject.org email alias forwarding - fedora-infrastructure - Pagure - https://pagure.io/fedora-infrastructure/issue/7226
15:37:54 <nirik> I was thinking just an ansible var... but if people have better ideas I am all ears.
15:37:54 <nirik> if no ideas we can move on.
15:37:54 <nirik> I'll probibly wait and do this after freeze
15:38:25 <nirik> I left this on from last week...
15:38:35 <nirik> no change, waiting on reporter I think
15:38:47 <smooge> okie dokie
15:38:59 <smooge> #topic Apprentice Open office minutes
15:39:00 <smooge> #info A time where apprentices may ask for help or look at problems.
15:39:10 <smooge> ok anything for this part?
15:39:52 <aikidouke> should I resend my clear 2fa request?
15:40:21 <nirik> aikidouke: yeah, I don't think I saw the updated one?
15:40:21 * relrod still hasn't done any onboard videos yet, will try to do some soonish.
15:40:34 <aikidouke> ok thanks nirik will do
15:40:40 <aikidouke> (hopefully correctly)
15:40:58 <nirik> aikidouke: is your key up on the keyservers?
15:41:04 <aikidouke> it is sir
15:41:18 <nirik> cool.
15:41:21 <smooge> relrod, thanks
15:41:27 <aikidouke> relrod: liked your talk :)
15:41:36 <relrod> aikidouke: thank you!
15:41:44 <smooge> I am hopefully going to watch it tomorrow.
15:42:25 <aikidouke> I've been trying to think about ways that an apprentice could clone the infra repo locally and then cut out some variables and run just a mini version of the app that they wanted to work on, if that makes sense
15:42:42 <smooge> that would be useful for many things
15:43:01 <aikidouke> obv wouldnt work well for builders, etc
15:43:02 <mkonecny> or use openshift as a playground :-)
15:43:36 <aikidouke> what level of os ninja would one need for that mkonecny?
15:45:02 <aikidouke> in a playground, local OR openshift, what things would not be needed? fedmsg? stg? just trying to think about things that would cut down on complexity
15:45:44 <mkonecny> I'm just thinking about using one instance of openshift for apprentice training, maybe prepared with some test application
15:45:50 <aikidouke> but thats probably a bigger discussion for an apprentice workday?
15:46:06 <smooge> i would agree on that
15:46:08 <nirik> yeah, as more of our apps use openshift that should be doable...
15:46:23 <nirik> and people could just use oc cluster up locally.
15:46:33 * aikidouke nods
15:46:56 <aikidouke> is after freeze a good time for a workday? after F29 beta/final release?
15:47:37 <nirik> yeah, after final might be...
15:47:44 <smooge> Octoberish
15:48:04 <smooge> I am going to move to Open Floor
15:48:07 <aikidouke> 👌🏼
15:48:08 <smooge> #topic Open Floor
15:48:38 <smooge> any open projects?
15:49:16 <smooge> if not I can close out and give you all coffee time
15:49:29 <aikidouke> coffee!
15:49:34 <bcotton> #info Release Readiness and Go/No-Go meetings are later today!
15:50:00 <nirik> day of meetings. what fun!
15:50:07 <aikidouke> #info Stay Safe US Southeast :)
15:50:14 <bcotton> aikidouke++
15:50:14 <zodbot> bcotton: Karma for aikidouke changed to 3 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:50:14 <smooge> ah yes. these are important meetings that need concentration... so please just ping the oncall person and have them contact whoever you needed to see when they are available
15:50:50 <nirik> yeah, hopefully everyone has their bread and milk stockpiled.
15:50:59 <smooge> don't forget your cinnamon
15:51:08 <bowlofeggs> i've got a month's worth of soylent on hand!
15:51:08 <smooge> because you will need it for the french toast
15:51:10 <bcotton> milk will not keep well :-)
15:51:22 <bcotton> vanilla extract is important, too
15:51:27 <smooge> oh yes
15:51:48 <smooge> what I love is that canned and powdered milk is rarely bought
15:51:50 <smooge> ok anyway
15:51:57 <smooge> have a good week everyone
15:52:00 <smooge> say good night bowlofeggs
15:52:26 <smooge> #endmeeting