Confusing some configurations in ClusterLogForwarder kind (DO380)

[GiangPham]

Hi everyone,

Recently, I have been practicing exercises from Chapter 7 in DO380. I modified some configurations using my own parameters, which are a bit different from the original setup. However, after making the changes, I don’t fully understand why the sample configuration in this chapter works effectively, while mine doesn’t (specifically pushing app, infra, and audit logs to the syslog endpoint).

I hope someone who has experience can share some insights or point out any mistakes I might have made.

Thanks in advance!

Initial code: 

apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
    name: instance
    namespace: openshift-logging
spec:
    inputs:
    - name: critical-apps
      application:
        selector:
          matchLabels:
            logging: critical

    outputs:
    - name: audit-syslog
      type: syslog
      url: tcp://utility.lab.example.com:514
      syslog:
        msgID: audit
        appName: ocp-lab
        facility: user
        procID: vector
        rfc: RFC5424
        severity: informational

    - name: apps-syslog
      type: syslog
      url: tcp://utility.lab.example.com:514
      syslog:
        msgID: apps
        appName: ocp-lab
        facility: user
        procID: vector
        rfc: RFC5424
        severity: informational

    - name: infra-syslog
      type: syslog
      url: tcp://utility.lab.example.com:514
      syslog:
        msgID: infra
        appName: ocp-lab
        facility: user
        procID: vector
        rfc: RFC5424
        severity: informational

    pipelines:
    - name: critical-apps-syslog
      inputRefs:
        - critical-apps
      outputRefs:
        - apps-syslog

    - name: infra-syslog
      inputRefs:
        - infrastructure
      outputRefs:
        - infra-syslog

    - name: audit-syslog
      inputRefs:
        - audit
      outputRefs:
        - audit-syslog

My code:

apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
    name: instance
    namespace: openshift-logging
spec:
  inputs:
    - application:
        selector:
          matchLabels:
            logging: critical
      name: critical-apps
  outputs:
    - name: apps-tcp-syslog
      syslog:
        appName: ocp-console
        facility: user
        msgID: apps
        procID: apps
        rfc: RFC5424
        severity: informational
      type: syslog
      url: 'tcp://utility.lab.example.com:514'
    - name: apps-udp-syslog
      syslog:
        appName: ocp-console
        facility: user
        msgID: apps
        procID: apps
        rfc: RFC5424
        severity: informational
      type: syslog
      url: 'udp://utility.lab.example.com:514'
    - name: infra-tcp-syslog
      syslog:
        appName: ocp-console
        facility: user
        msgID: infra
        procID: infra
        rfc: RFC5424
        severity: informational
      type: syslog
      url: 'tcp://utility.lab.example.com:514'
    - name: infra-udp-syslog
      syslog:
        appName: ocp-console
        facility: user
        msgID: infra
        procID: infra
        rfc: RFC5424
        severity: informational
      type: syslog
      url: 'udp://utility.lab.example.com:514'
    - name: audit-tcp-syslog
      syslog:
        appName: ocp-console
        facility: user
        msgID: audit
        procID: audit
        rfc: RFC5424
        severity: informational
      type: syslog
      url: 'tcp://utility.lab.example.com:514'
    - name: audit-udp-syslog
      syslog:
        appName: ocp-console
        facility: user
        msgID: audit
        procID: audit
        rfc: RFC5424
        severity: informational
      type: syslog
      url: 'udp://utility.lab.example.com:514'
  pipelines:
    - inputRefs:
        - critical-apps
      name: apps-pipeline
      outputRefs:
        - apps-tcp-syslog
    - inputRefs:
        - infrastructure
      name: infra-pipeline
      outputRefs:
        - infra-tcp-syslog
    - inputRefs:
        - audit
      name: audit-pipeline
      outputRefs:
        - audit-tcp-syslog

Best regards!
Giang

[sam2019]

@GiangPham ,  i understand  that you are taking an example from the course that works fine with a specific scenario and has been tried and tested - replacing with your own version and attempting to get it working on the lab cluster. That may not work due to a number of reasons. To provide a better context, pls explain what the original configuration does and how you are changing it and what were you hoping your changes will do ( if they had worked ). 

[GiangPham]

Hi Sam,

Thank you spent time to my feedback. At there, all contexts of this exercise is required:

--------------------

The syslog server is set up to separate OpenShift audit, infrastructure, and application logs into individual files within the /var/log/openshift directory. This is done by using the msgID attribute in syslog, with values audit, infra, and apps for each log type respectively.

Set up the Cluster Log Forwarder with three distinct pipelines—one for each log type—to forward logs to the syslog server based on the corresponding msgID value.

Make sure that only application logs labeled with logging: critical are included.

The syslog server is accessible at the DNS address utility.lab.example.com, and it accepts connections over TCP port 514.

Beside that, only vector as collector using in ClusterLogging

--------------------

My modification:

1. Change name, proid, msgid in ClusterLogForwarder, specially, i want to add a new option related to udp protocol (Temporarily i remove this section in pipeline why still error)

Expected results: 

1. Create eventrouter prior
2. Logs can be found at /var/log/openshift path in utility.lab.example.com (such as audit.log, infra.log, app.log)