[iptables] Add mapping for journald.custom fields

[mjwolf]

Fixes this error in system tests:

test case failed: one or more errors found in documents stored in logs-iptables.log-95566 data stream: [0] field "journald.custom.seqnum" is undefined
[1] field "journald.custom.seqnum_id" is undefined

e.g. https://buildkite.com/elastic/integrations/builds/17669

Proposed commit message

Add mappings for journald.custom fields, which will fix an automated system test failure.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Run elastic-package test system, there should be no failures

Related issues

• Closes [Stack 8.16.0-SNAPSHOT] [iptables] Failing test daily: system test: journald in iptables.log #11337
• Closes [Stack 8.17.0-SNAPSHOT] [iptables] Failing test daily: system test: journald in iptables.log #11559

[elasticmachine]

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[andrewkroh]

SEQNUM_ID is not a "custom" field. This is built-in to journald. IMO this is an input bug that it writes it to the "journald.custom" namespace. We should fix that problem.

To address the test failure, could we ship an update to the ingest pipeline that drops this specific field instead of using the flattened? The field is not particularly useful IMO and takes up a lot of space.

• https://www.freedesktop.org/software/systemd/man/latest/systemd.journal-fields.html#Address%20Fields

[mjwolf]

SEQNUM_ID is not a "custom" field. This is built-in to journald. IMO this is an input bug that it writes it to the "journald.custom" namespace. We should fix that problem.

To address the test failure, could we ship an update to the ingest pipeline that drops this specific field instead of using the flattened? The field is not particularly useful IMO and takes up a lot of space.

• https://www.freedesktop.org/software/systemd/man/latest/systemd.journal-fields.html#Address%20Fields

Thanks, that's good to know. I've removed the seqnum fields in the ingest pipeline

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: a693372917cdea7a81c6f8fff4bf9f3a13357982

History

• 💚 Build #18628 succeeded 7046cb81f21b01036de289be5aa2953af691e5d2

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[andrewkroh]

LGTM. Will you please open an elastic/beats issue to track the underlying input problem that neither field should be sent through as "custom".

[andrewkroh]

Suggested change

	link: https://github.com/elastic/integrations/pull/999999
	link: https://github.com/elastic/integrations/pull/11839

[andrewkroh]

Suggested change

	- description: Remove journald seqnum fields
	- description: Remove journald.custom.seqnum and journald.custom.seqnum_id which are misclassified as custom fields by the journald input.

[andrewkroh]

@mjwolf Sorry, I didn't notice that auto-merge was enabled. I did have a suggestion and one request.