Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-7183

[release-6.0] Syslog: merging data from 'message' to the root of log makes log event inconsistent with ViaQ data model

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • NEW
    • Hide
      Before this update, merging data from the 'message' field into the root of a Syslog log event caused the log event to be inconsistent with the ViaQ data model, potentially leading to overwritten system information, data duplication, or event corruption. With this update, the parsing and merging behavior for Syslog output has been revised to align with other output types, resolving the inconsistency.
      Show
      Before this update, merging data from the 'message' field into the root of a Syslog log event caused the log event to be inconsistent with the ViaQ data model, potentially leading to overwritten system information, data duplication, or event corruption. With this update, the parsing and merging behavior for Syslog output has been revised to align with other output types, resolving the inconsistency.
    • Bug Fix
    • Log Collection - Sprint 271, Log Collection - Sprint 272
    • Important

      Description of problem:

      This bug drive to several issues introduced during the migration from Fluentd to Vector. Behavior related to parsing and merging data from the message field into the log event, which could lead to system information being overwritten, data duplication, or event corruption.

      Steps to Reproduce:

      1. Deploy a syslog server
      2. Deploy the Cluster Logging Operator with Vector as collector type
      3. Configure clusterLogForwarder for log forwarding to a syslog server with the next configuration: 
        apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: instance
  namespace: openshift-logging
spec:
  outputs:
  - name: logs
    type: syslog
    url: tcp://rsyslog-server.rsyslog-pj.svc:6514
  pipelines:
  - inputRefs:
    - application
    - audit
    name: syslog-pl
    outputRefs:
    - logs

      How to fix:

      Need to avoid using this VRL construction:

      . = merge(., parse_json!(string!(.message))) ?? .

      Instead of, to retain the ability to extract values from JSON-formatted logs, a JSON Parser Filter must be added to the Log Forwarder spec.
      To access specific fields from the parsed JSON, use the field names with supports template syntax with structured prefix

              vparfono Vitalii Parfonov
              vparfono Vitalii Parfonov
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: