20:00:15 <sgallagh> #startmeeting Fedora Server SIG Weekly Meeting (2018-06-12)
20:00:15 <zodbot> Meeting started Tue Jun 12 20:00:15 2018 UTC.
20:00:15 <zodbot> This meeting is logged and archived in a public location.
20:00:15 <zodbot> The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:15 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:15 <zodbot> The meeting name has been set to 'fedora_server_sig_weekly_meeting_(2018-06-12)'
20:00:15 <sgallagh> #meetingname serversig
20:00:15 <sgallagh> #topic Roll Call
20:00:15 <zodbot> The meeting name has been set to 'serversig'
20:00:25 <dperpeet> .hello2
20:00:26 <zodbot> dperpeet: dperpeet 'None' <dperpeet@redhat.com>
20:00:30 <sgallagh> .hello2
20:00:33 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
20:01:17 <nirik> morning
20:02:26 * langdon lurks
20:03:15 <sgallagh> OK, let's get started.
20:03:19 <sgallagh> #topic Agenda
20:03:25 <sgallagh> We have only one topic on the agenda so far:
20:03:39 <sgallagh> #info Agenda Topic: Remote Logging requirements
20:03:48 <sgallagh> Does anyone have another topic for this week>?
20:04:46 <dperpeet> I don't (just so you don't question your internet connectivity if nobody else answers)
20:04:53 <sgallagh> Heh
20:05:03 <sgallagh> Well, if anyone does, we'll have time for Open Floor, I expect
20:05:11 <sgallagh> #topic Remote Logging Requirements
20:05:47 <sgallagh> As hopefully everyone saw, Lukas Ruzicka sent out a proposed new test case for F29 blockers.
20:06:29 <sgallagh> This was based on our original (now-ancient) PRD for the Server Edition. We had a requirement that Server Edition must be capable of transmitting all logs to a remote logging source, as this is a common requirement in many deployments.
20:07:08 <sgallagh> The PRD intentionally did not specify the protocol that was needed for this, as when it was being written we were still considering whether we wanted to back the journald-remote horse or stay conservative with rsyslog.
20:07:52 <nirik> there's also a systemd-netlogd now...
20:07:53 <sgallagh> So, now that three years have passed, I think it's worth discussing whether we should come down firmly on one side or the other... or any other outcome
20:08:02 <sgallagh> Oh? That's a new one on me.
20:08:12 <nirik> (that transmits a journal via syslog to a remote machine)
20:08:15 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=1580121
20:08:22 <nirik> https://github.com/systemd/systemd-netlogd/blob/master/systemd-netlogd.spec
20:08:25 <sgallagh> hmm
20:08:41 <nirik> but thats using the same transport a rsyslog would use
20:08:51 <sgallagh> OK, interesting
20:09:18 <sgallagh> So I think there are several questions before us, and I'll try to enumerate them. Then we can walk through each of them
20:10:09 <sgallagh> 1) Do we want to require that the default installation of Server Edition is capable of transmitting logs to a remote server out of the box (modulo config settings to point the way)?
20:10:44 <sgallagh> 2) Do we want the Server Edition to be capable of aggregating logs from one or more remote servers out of the box (same caveat with required config)?
20:10:58 <sgallagh> 3) Do we want to mandate a specific protocol for 1) and 2)?
20:11:25 <sgallagh> I think I had a 4) when I started typing, but I forgot it...
20:12:17 <sgallagh> So, let's start with 1).
20:12:19 <nirik> I think 1 is something large deployments definitely want, but are we targeting them?
20:12:40 <dperpeet> wouldn't large deployments have a custom roll-out?
20:13:09 <dperpeet> I would say this is for the smaller deployments, or even trial ones ("would Fedora Server work for me?")
20:13:23 <sgallagh> That's a fair question; should we assume that anyone who is doing this sort of setup would have custom scripts to configure it anyway?
20:13:36 <sgallagh> ( I think "no", but the question should be asked and answered.)
20:14:01 <nirik> yeah, I would expect large sites to be using CM of some kind...
20:14:10 <dperpeet> I think with containers and virtual machines, it should be trivial to get the logs for anyone
20:14:32 <sgallagh> Jumping to the end-state, I think my ideal situation would be for Cockpit to have a config dialog that amounted to "send all my logs to x.x.x.x" and that would be answer to 1)
20:14:48 <sgallagh> (With Cockpit capable of doing the necessary package installation and configuration as needed)
20:15:06 <sgallagh> dperpeet: Sorry, can you expand on that last statement?
20:15:12 <sgallagh> I'm not sure I follow your logic.
20:15:28 <dperpeet> that's a very flexible definition of "out of the box" when you propose to have that done via Cockpit
20:16:03 <sgallagh> dperpeet: It's *soft*ware. Flexibility is right in the name!
20:16:07 <dperpeet> sure, I meant that when I set up a virtual machine with Fedora Server, it should be trivial to access those logs (have them pushed somewhere or pull from there)
20:16:10 <dperpeet> :)
20:16:22 <dperpeet> sgallagh, I wasn't criticizing, just pointing out
20:16:38 <dperpeet> I think that option would be a good compromise between initial footprint and ease of use
20:16:45 <sgallagh> Right, I wasn't necessarily answering question 1) there; I was throwing out a straw man for discussion
20:17:27 <sgallagh> dperpeet: Pulling logs from a server is prohibitively complex; I think we need to limit ourselves to "push" solutions for now
20:18:35 <sgallagh> dperpeet: The VM case I think is fundamentally identical to the bare metal case. Containers are a bit more interesting though.
20:18:52 <sgallagh> I'm not sure we necessarily want to try to solve that generically here in this meeting though.
20:19:09 <dperpeet> sgallagh, ah, I read the log aggregation as pulling logs
20:19:21 <nirik> I think (unless its changed) that sending logs via rsyslog/syslog proto is much easier than the systemd native one (which I think used http post?)
20:19:30 <sgallagh> No, I meant having multiple machines all pushing to a common machine that aggregates them
20:19:50 <sgallagh> nirik: It used an HTTPS transport in some fashion, yes
20:20:02 <sgallagh> I don't know all the details
20:20:46 <sgallagh> I don't think it was terribly important to know the difference, since theoretically the systemd-journal-upload and systemd-journal-remote tools should have handled the complexities under the hood
20:21:20 <sgallagh> nirik: Speaking from your perspective, did that protocol ever take root anywhere, or has the state of the art remained with rsyslog
20:21:20 <dperpeet> in that case I'm not so sure 2) is essential :)
20:21:46 <dperpeet> since combining information is a very broad field...
20:21:48 <nirik> sgallagh: I have never heard of anyone using systemd-journal-upload/remote.
20:22:06 <sgallagh> dperpeet: Well, rsyslog can do this
20:22:14 <nirik> I recall the devel thread had a bunch of improvement suggestions, but I don't know if those ever happened.
20:22:26 <sgallagh> nirik: That sounds to me like we should probably stick with rsyslog in Server Edition for the time being then
20:22:40 <nirik> yeah, that would be my vote
20:23:05 <sgallagh> dperpeet: rsyslog has a mode of operation where it can listen on a well-known port for messages sent from other rsyslog systems
20:23:25 <sgallagh> It "aggregates* them mostly in terms of serializing the data coming in and logging them to a common place tagged with origin
20:23:42 <dperpeet> yeah, I've seen that before
20:23:53 <dperpeet> it sounds like rsyslog is a good candidate
20:24:02 <dperpeet> maybe with an enabler via Cockpit or the likes?
20:24:26 <sgallagh> Yeah, that would be ideal in my book
20:24:40 <sgallagh> dperpeet: Does cockpit support on-demand package install yet?
20:25:05 <dperpeet> sgallagh, I honestly have no idea
20:25:26 <sgallagh> Probably best to just stick with installing rsyslog by default, I suppose
20:25:40 <sgallagh> And then have a cockpit configuration enabler
20:25:45 <dperpeet> yeah
20:26:25 <sgallagh> So, with all that in mind, I'd say that https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_Remote_logging as currently written would be appropriate and acceptable for F29 release criteria.
20:27:26 <sgallagh> (leaving the cockpit enabler as aspirational at this moment; though it's probably not going to be too hard to write -- the config is simple)
20:28:16 <dperpeet> +1
20:28:47 <nirik> yep. agree
20:29:55 <sgallagh> OK, good enough for me.
20:30:25 <sgallagh> #agreed Server SIG endorses rsyslog for remote logging and aggregation and approves the proposed rsyslog test cases for F29
20:30:30 <sgallagh> #topic Open Floor
20:30:36 <sgallagh> Other topics for this week?
20:31:06 * nirik has nothing off hand
20:31:56 <sgallagh> OK, I'll give it another minute or so and then close the meeting
20:34:22 <sgallagh> #endmeeting