Handle multiline log records

Important

Enable either autoMultilineDetection or a Recombine operator using is_first_entry—never both at the same time.

Note

The File Log Receiver flushes every 5 seconds by default. The 5 seconds is measured from when a log line matching either the default auto-detection regex or the provided is_first_entry regex is detected. Lines emitted outside that window are stored as separate records.

Automatic detection

When autoMultilineDetection is enabled, the agent groups related log lines into a single entry whenever the first line starts with one of these timestamp formats:

  • YYYY-MM-DD HH:MM:SS  → 2025-06-16 13:45:30

  • YYYY-MM-DDTHH:MM:SS.sssZ  → 2025-06-16T14:33:53.123456Z

  • Mon DD HH:MM:SS  → Jun 16 20:16:01

  • YYYY/MM/DD HH:MM:SS  → 2025/06/16 20:46:15

  1. Create a file named auto-multiline-detection-values.yaml with the following contents:

node:
  containers:
    logs:
      autoMultilineDetection: true
      enabled: true

  1. Redeploy the Observe Agent.

Run the following command to redeploy the Observe Agent in the observe namespace.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values auto-multiline-detection-values.yaml

  1. Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe

Custom pattern with the Recombine operator

autoMultilineDetection covers the most common timestamp formats, but some applications use different markers to signal the start of a log entry. When that happens, you can switch to the Recombine Operator and define your own rule with the is_first_entry setting. The operator tells the Observe Agent to begin a new record whenever a line matches the pattern you provide.

Suppose your logs lines that start with [

[2025-06-18T18:52:24.089689Z] INFO: User login successful. User ID: 92065
[2025-06-18T18:52:24.561449Z] INFO: User login successful. User ID: 65749
[2025-06-18T18:52:25.309899Z] DEBUG: Starting backup process.
Directory: /data/backup
Estimated files: 556
[2025-06-18T18:52:26.226822Z] DEBUG: Starting backup process.
Directory: /data/backup
Estimated files: 446
[2025-06-18T18:52:27.538841Z] ERROR: Failed to load configuration file.
File path: /etc/app/config.yaml
Cause: FileNotFoundError
Stack trace:
    File "/app/main.py", line 23, in load_config
    config = open(config_path, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/etc/app/config.yaml'
[2025-06-18T18:52:29.004533Z] ERROR: Failed to load configuration file.
File path: /etc/app/config.yaml
Cause: FileNotFoundError
Stack trace:
    File "/app/main.py", line 23, in load_config
    config = open(config_path, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/etc/app/config.yaml'
[2025-06-18T18:52:30.190177Z] INFO: User login successful. User ID: 30051

  1. Create a file named multiline-detection-values.yaml with the following contents:

node:
  containers:
    logs:
      autoMultilineDetection: false
      enabled: true

agent:
  config:
    nodeLogsMetrics:
      receivers:
        filelog:
          operators:
          - id: container-parser
            max_log_size: 102400
            type: container
          # Recombine lines until the next one that starts with "["
          - id: multiline-recombine
            type: recombine
            combine_field: body
            is_first_entry: body matches "^\\["

  1. Redeploy the Observe Agent.

Run the following command to redeploy the Observe Agent in the observe namespace.

helm upgrade --reuse-values observe-agent observe/agent -n observe --values multiline-detection-values.yaml

  1. Restart the pods.

kubectl rollout restart deployment -n observe
kubectl rollout restart daemonset -n observe