16:01:35 <eddiejennings> #startmeeting Infrastructure (2022-04-14)
16:01:35 <zodbot> Meeting started Thu Apr 14 16:01:35 2022 UTC.
16:01:35 <zodbot> This meeting is logged and archived in a public location.
16:01:35 <zodbot> The chair is eddiejennings. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:01:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:01:35 <zodbot> The meeting name has been set to 'infrastructure_(2022-04-14)'
16:01:48 <dtometzki> .hi
16:01:48 <aheath1992> .hello
16:01:48 <zodbot> dtometzki: dtometzki 'Damian Tometzki' <linux@tometzki.de>
16:01:51 <zodbot> aheath1992: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
16:01:51 <eddiejennings> #meetingname infrastructure
16:01:51 <zodbot> The meeting name has been set to 'infrastructure'
16:01:57 <aheath1992> .hello anheath1992
16:01:58 <zodbot> aheath1992: Sorry, but user 'anheath1992' does not exist
16:02:01 <eddiejennings> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak computerkid
16:02:01 <zodbot> Current chairs: bodanel computerkid dtometzki eddiejennings jnsamyak mobrien nirik pingou siddharthvipul zlopez
16:02:01 <eddiejennings> #info Agenda is at: https://board.net/p/fedora-infra
16:02:02 <eddiejennings> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:02:02 <eddiejennings> #topic greetings!
16:02:03 <mobrien> .hi
16:02:04 <zodbot> mobrien: mobrien 'Mark O'Brien' <markobri@redhat.com>
16:02:04 <aheath1992> .hello aheath1992
16:02:07 <zodbot> aheath1992: aheath1992 'Andrew Heath' <aheath1992@gmail.com>
16:02:08 <eddiejennings> .hi
16:02:10 <zodbot> eddiejennings: eddiejennings 'Eddie Jennings' <eddie@eddiejennings.net>
16:02:50 <nirik> morning
16:02:55 <eddiejennings> #topic New folks introductions
16:02:55 <eddiejennings> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:02:55 <eddiejennings> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:02:57 <darknao> .hi
16:02:59 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:03:18 <eddiejennings> Good [time appropriate greeting] all!
16:03:33 <petebuffon> .hello petebuffon
16:03:34 <zodbot> petebuffon: petebuffon 'Peter Buffon' <pabuffon@gmail.com>
16:03:35 <prakashmishra> Hello everyone
16:03:38 <eddiejennings> Do we have any new folks with us today?  If so, introduce yourselves!
16:03:46 <eddiejennings> We won't bit!  Not even petebuffon
16:03:55 <eddiejennings> /s/bit/bite
16:05:39 <eddiejennings> Last call for new folks introductions!
16:06:04 <prakashmishra> Hi. I'm Prakash. I've worked as an SRE in the past. I look forward to contributing to Fedora Infra
16:06:13 <nirik> welcome prakashmishra
16:06:18 <Saffroni1ue> o/
16:06:18 <eddiejennings> Welcome Prakash!
16:06:20 <dtometzki> hi
16:06:32 <prakashmishra> o/
16:06:38 <mkonecny> .hello zlopez
16:06:39 <zodbot> mkonecny: zlopez 'Michal Konecny' <michal.konecny@psmail.xyz>
16:06:51 <eddiejennings> Moving along. :)
16:06:52 <eddiejennings> #topic Next chair
16:06:53 <eddiejennings> #info magic eight ball says:
16:06:53 <eddiejennings> ##info chair 2022-04-14 - eddiejennings
16:06:53 <eddiejennings> ##info chair 2022-04-21 - ??
16:06:53 <eddiejennings> ##info chair 2022-04-28 - ??
16:07:11 <eddiejennings> So I'm willing to chair 4/21, since that'll be the end of my on-call week
16:07:12 <dtometzki> i will do 28
16:07:23 <eddiejennings> 28th sold to dtometzki
16:07:31 <mkonecny> Hi Prakash
16:07:34 <mobrien> I can do 21
16:07:48 <eddiejennings> 21st sold to mobrien
16:07:55 <dtometzki> oh no i cant eddiejennings
16:08:05 <dtometzki> iam not available
16:08:06 <eddiejennings> sorry, all bids final :P
16:08:18 <eddiejennings> I'll do the 28th then :D
16:08:35 <eddiejennings> ##info chair 2022-04-21 - mobrien
16:08:36 <dtometzki> should i do 21
16:08:43 <dtometzki> ok
16:08:52 <eddiejennings> ##info chair 2022-04-28 - eddiejennings
16:09:05 <eddiejennings> How about May 5th?
16:09:24 <dtometzki> yes that is ok
16:09:36 <dtometzki> great
16:09:38 <eddiejennings> ##info chair 2022-05-05 dtometzki
16:09:59 <eddiejennings> Excellent.
16:10:08 <eddiejennings> I think we're well covered for the new few weeks.
16:10:25 <eddiejennings> #topic announcements and information
16:10:34 <eddiejennings> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:10:34 <eddiejennings> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:10:34 <eddiejennings> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request
16:10:34 <eddiejennings> #info Fedora 36 Beta is out
16:10:35 <eddiejennings> #info Fedora Final freeze is on going
16:10:37 <eddiejennings> #info thread on fedoraplanet on infrastructure list, chime in if you have thoughts on it
16:10:39 <eddiejennings> #info Easter Holidays on Friday 15th April and Monday 18th April, plenty of Red Hat folks will be unavailable
16:11:20 <mobrien> #info please help us with improving contribution to fedora infra https://discussion.fedoraproject.org/t/improving-contribution-to-fedora-infrastructure/38294/8
16:11:53 <eddiejennings> Noted.  Any other announcements?
16:12:31 <mobrien> Above is a discussion thread about ways to possibly improve contribution, any and all feedback is wanted and welcomed
16:13:02 <eddiejennings> I'll take a look at that thread :)
16:13:19 <eddiejennings> Ok. Shifting gears.
16:13:27 <eddiejennings> #topic Oncall
16:13:40 <eddiejennings> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:13:40 <eddiejennings> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:13:40 <eddiejennings> ## .oncalltakeeu .oncalltakeus
16:13:40 <eddiejennings> #info petebuffon on call from 2022-04-01 to 2022-04-07
16:13:40 <eddiejennings> #info mobrien on call from 2022-04-08 to 2022-04-14
16:13:41 <eddiejennings> #info eddiejennings on call from 2022-04-15 to 2022-04-21
16:13:43 <eddiejennings> #info ? on call from 2022-04-22 to 2022-04-28
16:13:43 <mobrien> Thank you 🙂
16:13:51 <eddiejennings> You're welcome :D
16:14:07 <eddiejennings> Any takers for 4/22 - 4/28?
16:15:11 <dtometzki> i can takit but iam unavailable on 4/28
16:15:25 <eddiejennings> Just that one day?
16:15:32 <dtometzki> yes
16:15:40 <eddiejennings> I'll cover for you on the 28th
16:15:59 <dtometzki> perefct
16:16:00 <eddiejennings> #info dtometzki on call from 2022-04-22 to 2022-04-28 (eddiejennings covering 2022-04-28)
16:16:20 <eddiejennings> Any takers for 4/29 - 5/5?
16:17:40 <nirik> I can if no one else wants it.
16:17:43 <mobrien> I think 2 weeks in advance is probably ok
16:17:44 <eddiejennings> I think my cat may be volunteering.  He just brought me his little toy :P
16:17:53 <eddiejennings> Yeah.  We can leave it open until next week
16:18:10 <eddiejennings> Moving along.
16:18:22 <eddiejennings> #info Summary of last week: (from current oncall )
16:18:36 <eddiejennings> mobrien, take it away!
16:19:14 <mobrien> I was sick at the end of last week so didn't take it until start of this week but I received no pings
16:19:48 <eddiejennings> No pings are the best pings :D
16:19:54 <nirik> things have been quiet...
16:20:09 <eddiejennings> #topic Monitoring discussion [nirik]
16:20:09 <eddiejennings> #info https://nagios.fedoraproject.org/nagios
16:20:09 <eddiejennings> #info Go over existing out items and fixs
16:20:16 <mobrien> I'm guessing Freeze is the reason behind the quiet
16:20:25 <eddiejennings> nirik, you're up
16:20:47 <nirik> lets see... I think we are in better shape than before...
16:21:15 <nirik> 2 "down" hosts are just mgmt interfaces misbehaving...
16:21:30 <nirik> once f36-test is sorted, we should be pretty green.
16:21:43 <nirik> We have been getting lots of badges and resultsdb alerts....
16:21:55 <nirik> resultsdb is on the way into openshift, so hopefully that will fix that.
16:22:13 <nirik> not sure what to do about badges. ;( Perhaps we should adjust the alerting
16:22:37 <nirik> thats about it, unless there's any questions...
16:22:50 <eddiejennings> Badges broken or alerting needlessly?
16:23:05 <Saffroni1ue> nirik: whats the eta on resultsdb going in? is anyone working on that atm?
16:23:26 <mobrien> f36-test playbook is running as we speak(although looking like it will fail)
16:23:55 <nirik> eddiejennings: it's getting stuck from time to time and we get alerts on the queue growing, so someone has to go wipe it's queue and restart it.
16:24:09 <eddiejennings> ah
16:24:28 <nirik> Saffroni1ue: yeah, odra (lrossett) is working on it. not sure how much time he has for it tho... it's been a long road.
16:24:38 <nirik> we almost have it up in stg.
16:25:08 <Saffroni1ue> kk might see how hes geting on, PrakashMishra[m] and I were looking to work on some openshift related tasks
16:26:19 <nirik> yeah, you might be more in his time zone to help with it too.
16:26:34 <nirik> also, we have a mini-iniative to move stuff from ocp3 to 4...
16:27:20 <nirik> I'd like to retire the ocp3.11 cluster soon
16:28:18 <nirik> anyhow, we can move on from monitoring...
16:28:18 <prakashmishra> Yes. Saffroni1ue and I were talking about working on some Openshift related tasks. I also have experience working with OCP 3.11 and 4. I will sync with him to see where and how I can help
16:28:28 <eddiejennings> Noted.
16:28:53 <eddiejennings> #topic Learning topic
16:28:53 <eddiejennings> #topic Tor and Tor Services [eddiejennings & petebuffon] - 2022-04-14
16:29:00 <eddiejennings> So for those that are new.
16:29:19 <eddiejennings> We generally alternate weeks between someone presenting a learning topic and looking at the ticket backlog.
16:29:24 <eddiejennings> This week is a learning topic week.
16:29:36 <eddiejennings> petebuffon and I are going to tag-team it for this week.
16:30:08 <eddiejennings> petebuffon will start the topic about Tor itself and Tor services, then I'll jump in and start the discussion as it relates to an open ticket we have.
16:30:15 <eddiejennings> petebuffon, take it away! :D
16:30:38 <petebuffon> Okay! So let's enter the world of Tor and the darkweb
16:30:56 * nirik puts on a hoodie.
16:31:06 <petebuffon> There is a lot of opinions and politics surrounding Tor, I will focus on how it works.
16:31:10 * eddiejennings secures his tinfoil hat.
16:31:28 <petebuffon> Tor was developed in the 1990s by the United States Navy. The Onion Routing Project simply became Tor and was released under a free license in 2004. The Tor Project was founded for maintaining Tor.
16:31:40 <petebuffon> https://www.torproject.org/
16:31:55 <petebuffon> Tor relies on the concept of onion routing, where messages are encapsulated in multiple layers of encryption.
16:32:06 <petebuffon> The encrypted data is transmitted through a series of network nodes called onion routers, each of which decrypts a single layer, uncovering the data’s next destination.
16:32:20 <petebuffon> Tor relies on TLS for encryption.
16:32:28 <petebuffon> The Tor network is operated by a group of worldwide volunteers who each run their own onion router.
16:32:39 <petebuffon> When the final layer is decrypted, the message has arrived at its destination.
16:32:48 <petebuffon> Each onion router only knows the locations of the preceding and following nodes, resulting in anonymity for both the sending and receiving parties.
16:33:05 <petebuffon> images for demonstration
16:33:07 <petebuffon> https://en.wikipedia.org/wiki/Tor_(network)#/media/File:How_Tor_Works_2.svg
16:33:11 <petebuffon> https://en.wikipedia.org/wiki/Onion_routing#/media/File:Onion_diagram.svg
16:33:31 <petebuffon> And for those wanting to go on a deep dive, the crypto is really interesting: https://www.onion-router.net/Publications/CACM-1999.pdf
16:34:06 <petebuffon> any questions so far?
16:34:22 * nirik is looking at diagrams
16:35:06 <petebuffon> basically tor traffic instead of hopping through normal routers, hops through onion routers.
16:35:35 <eddiejennings> A key concept, at least from my reading, is the idea of the Tor circuit.
16:35:48 <petebuffon> each onion router can only see encrypted payloads because they don't have the encryption keys to decrypt the payload
16:36:14 <nirik> huh, so the sender client knows all the servers and decides the path on sending? I guess that makes sense because then they can do all the encryption up front. Just the nodes don't know the full path.
16:36:24 <eddiejennings> Traffic enters the Tor network via a "guard", will be relayed at least once through another note, then exits to the destination via an "exit node"
16:37:17 <petebuffon> ya the client decides on a path, or circuit. It's also changed about once every minute
16:38:03 <petebuffon> anything else to add eddiejennings before I talk about services?
16:38:33 <eddiejennings> I think that's about it for an overview.  Understanding the idea of the circuit is important for the services.  Good job so far :D
16:38:48 <petebuffon> nice okay
16:38:57 <petebuffon> Servers configured to receive inbound connections only through Tor are called onion services (formerly, hidden services).
16:39:07 <petebuffon> Onion services are accessed through a .onion top level domain, which are not actual DNS names.
16:39:15 <petebuffon> These sites can only be accessed through the Tor network where the onion address is used to lookup public keys and introduction points (located in a distributed hash table) for the service.
16:39:44 <petebuffon> Setting up an onion service is as easy as installing Tor (either from package repo or from https://torproject.org), editing the Tor config (/etc/tor/torrc), and then starting the service (systemctl start tor).
16:39:55 <petebuffon> Set your web server to listen only on localhost (127.0.0.1)  and add the following lines to /etc/tor/torrc:
16:39:58 <petebuffon> HiddenServiceDir /var/lib/tor/hidden_service/http
16:40:02 <petebuffon> HiddenServicePort 80 127.0.0.1:80
16:40:08 <petebuffon> You can get your .onion address with:
16:40:12 <petebuffon> $ sudo cat /var/lib/tor/hidden_service/http/hostname
16:40:15 <petebuffon> o9asojd8aymqqtoa.onion
16:40:43 <petebuffon> this is a small exert from a linuxjournal article: https://www.linuxjournal.com/content/tor-hidden-services
16:41:27 <petebuffon> clients can either access onion services via the Tor browser
16:41:46 <petebuffon> or through socks5 proxies as well
16:42:37 <petebuffon> anything to add eddiejennings?
16:43:07 <eddiejennings> In the example of the webserver, from my understanding, using a Tor service is kind of like running a reverse-proxy on the same host as your web server.
16:43:33 <petebuffon> right
16:43:41 <eddiejennings> You reverse-proxy (the Tor service) is listening for traffic on port 80 on your public NIC.
16:44:05 <eddiejennings> Then it hands the traffic off to the loopback interfact on Port 80, which the web server hosting content is listening on
16:44:33 <eddiejennings> interfact = interface
16:44:49 <eddiejennings> Questions about the general idea of Tor services?
16:45:13 <nirik> so I assume there's some way to prevent duplicate names... ie, we advertise o9asojd8aymqqtoa.onion as a pointer to mirrors.fedoraproject.org, something would prevent someone from just setting that hostname and hyjacking traffic?
16:46:07 <petebuffon> I believe that has to do with the distributed hash table for .onion names, but I'm not exactly sure how that works
16:46:32 <nirik> ok, just wondering.
16:46:48 <eddiejennings> Speaking of, this brings us to this ticket.
16:47:05 <eddiejennings> https://pagure.io/fedora-infrastructure/issue/9549
16:47:19 <eddiejennings> .ticket9549
16:47:25 <eddiejennings> .ticket 9549
16:47:27 <zodbot> eddiejennings: Issue #9549: Tor hidden service for update metadata - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9549
16:47:46 <mkonecny> Does Tor has something like DNS?
16:48:44 <eddiejennings> From what I gathered, yes it does for your Tor services.  There is a mechanism that publishes a directory of sorts.  petebuffon may have another way of describing it.
16:50:02 <nirik> ah from a quick poking around... the .onion hostname is a key hash... so it needs to match the key you generate for you to get the traffic/have that valid name
16:50:28 <petebuffon> right, it's a private key / public key kinda deal
16:51:07 <eddiejennings> To determine the feasibility of the request for 9549, there are a few things that have to be considered.
16:51:12 <petebuffon> the hash table of onion addresses is stored at each onion router
16:52:04 <eddiejennings> One question I have, and maybe nirik can shed some light.  I learned there's a rust application that is what generates the metalinks used by dnf.  Is that the application that's accepting incoming requests, or is a webserver getting the request and handing it to that application?
16:53:14 <nirik> it's apache -> mirrorlist2 server
16:53:27 <nirik> so it does hit apache on the proxies first and proxies to mirrorlist2 server
16:54:21 <eddiejennings> and mirrorlist2 server is what generates the metalinks?
16:54:42 <nirik> yep
16:54:49 <nirik> thats the rust app
16:55:45 <eddiejennings> Which would bring up the question of can that app function behind a Tor hidden service?  I don't know the answer to that, but that's something that I think has to be answered to determine if we should move forward.
16:56:28 <nirik> I would think it could... we could test in staging...
16:56:35 <eddiejennings> The other thing, is if that's behind a Tor hidden service and functioning, will there be any impact with traffic received by that service that's not coming from the Tor network.  Again, don't have the answer for that (my hunch is "no"), but that would have to be known.
16:56:37 <nirik> I'm not sure how you would enable this in dnf?
16:58:01 <nirik> (but it sounds like the ticket reporter knows some way?)
16:58:22 <eddiejennings> My guess on that would be when dnf reaches out to the mirrors, there's a way for the mirrors to see this is coming from Tor and serve up the appropriate metalinks.
16:59:02 <eddiejennings> So the tl;dr for this ticket is, I don't think it's a hard "no," but there are answers that need to be had to really wrap our heads around it.
16:59:28 <nirik> well, you can't use .onion addresses in /etc/yum.repos.d/ files... so I don't know how you tell dnf to contact the metalink server over tor...
16:59:58 <nirik> yeah. Great investigation eddiejennings and petebuffon! Lots to consider and great describing it...
17:00:26 <mobrien> Great talk eddiejennings & petebuffon
17:00:35 <eddiejennings> The question then becomes, is this worth the time and effort, which we can discuss in #fedora-admin or other channels :)
17:01:16 <petebuffon> and if the goals wanted by using tor can be achieved by another method
17:01:26 <eddiejennings> So we've hit 13:00, any other quick announcements or other things or note?
17:01:41 <dtometzki> no many thanks
17:01:45 <eddiejennings> #topic Open Floor
17:02:21 <eddiejennings> Thank you to my partner in crime petebuffon for this week's meeting :D
17:02:34 <petebuffon> cheers, what a wild ride
17:02:35 <Saffronique> thanks eddiejennings, petebuffon was fun
17:02:43 <eddiejennings> And with that you may return to your regularly scheduled Thursday!
17:02:45 * nirik has to run to another meeting. thanks.
17:02:57 <mkonecny> Thanks for running this eddiejennings
17:03:02 <eddiejennings> #endmeeting