I’m working on a system that will flexibly route syslog data from its various sources (Cisco logs, proxy logs, linux server logs) to two or three endpoints.
One endpoint is QRadar, which already has numerous DSMs and parsers defined. We don’t want to rewrite any of those.
Most of the parsing will actually happen downstream, we just need to post the logs to kafka, and QRadar, in the formats they arrive in.
I seem to be fighting Fluentd’s tendency to convert syslog header to JSON fields for the header, leaving the message field with only part of the data I want to send along. But it’s all JSON encoded, so it breaks the “don’t change the Syslog format” requirement.
I actually want to send the original syslog header fields, such as timestamp, and source hostname, intact.
fluentd Input: timestamp hostname message
fluentd Output: timestamp hostname message (unaltered)
Since we have a lot of log sources, we don’t control which syslog format they use (RFC3164 or RFC5424)
The ideal situation would be:
TCP input (TCP or UDP port 514)
for each message:
TCP or UDP output in exactly the same format it arrived in (with no added JSON from fluentd)
(We’d prefer to have TCP for everything)
Is this possible?