Skip to content

Make prospector.scanner.fingerprint configurable #12274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

MichaelKatsoulis
Copy link
Contributor

@MichaelKatsoulis MichaelKatsoulis commented Jan 8, 2025

Proposed commit message

  • WHAT: In kubernetes container logs, make usage of fingerprint optional and configurable
  • WHY: The default values of prospector.scanner.fingerprint do not suit all users

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

fingerprint

@MichaelKatsoulis MichaelKatsoulis requested a review from a team as a code owner January 8, 2025 14:55
@andrewkroh andrewkroh added Integration:kubernetes Kubernetes Team:Cloudnative-Monitoring Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring] labels Jan 8, 2025
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Jan 8, 2025

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

- name: fingerprint
title: Fingerprint settings
show_user: false
description: Configuration options for prospector.scanner.fingerprint. By default fingerprint is enabled, offset is 0 and length 1024.
Copy link
Contributor

@tetianakravchenko tetianakravchenko Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe instead of listing defaults in description, would be better to add a link to the doc instead - https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#filebeat-input-filestream-scan-fingerprint

multi: false
required: true
show_user: false
description: If prospector.scanner.fingerprint.enabled is set to false, file_identity fingerprint must be switched off. If it set to true, file_identity fingerprint must be switched on.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those conditions might be quite confusing, wondering if suggested option wouldn't be easier

file_identity.fingerprint: ~
{{/if}}
Copy link
Contributor

@tetianakravchenko tetianakravchenko Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid conditions in fields description, we could use smth like below instead:

{{#if useFingerprint}}
file_identity.fingerprint: ~
prospector:
  scanner:
    fingerprint:
      enabled: true
      {{ fingerprintConfig }}
{{/if}}

(not tested)
fingerprintConfig variable then would be:

offset: 0
length: 1024

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this but then it would be confusing to the users as to what options exactly they can set in fingerprintConfig yaml. For example they can set offset and length but not enabled. And if they set enabled as well then there will be an error.
Maybe theenabled could also be part of the available configuration options, to avoid errors, but it won't have any effect as it will be under the {{#if useFingerprint}}

Like this:

{{#if useFingerprint}}
file_identity.fingerprint: ~
prospector:
  scanner:
    {{ fingerprint }}
{{/if}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tetianakravchenko check the new commit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe theenabled could also be part of the available configuration options, to avoid errors, but it won't have any effect as it will be under the {{#if useFingerprint}}

but does it change much from the previous option? I mean that we still allow config like:

useFingerprint: true
fingerprint:
  enabled: false

that is not supported, no?

As an option in useFingerprint description can be mentioned that prospector.scanner.fingerprint will be enabled automatically, for configuring additional prospector.scanner.fingerprint parameters please use Fingerprint settings. WDYT?

Another reason to put prospector.scanner.fingerprint to the template - now we allow to configure all prospector.scanner.*, not just the prospector.scanner.fingerprint is it expected behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another reason to put prospector.scanner.fingerprint to the template - now we allow to configure all prospector.scanner.*, not just the prospector.scanner.fingerprint is it expected behavior?

In theory they could configure all prospector.scanner.* but that was not the idea. The idea was to configure fingerprint only. It was just more clear if they also have to set the fingerprint key.
We could change that and go one level deeper.

useFingerprint: true
fingerprint:
  enabled: false

That is not supported but can never happened as if the set useFingerprint false, then the prospector.scanner.fingerprint won't be include in the final config. If we do not include the enabled parameter, then it is error prone. First because users do not know exactly what are the options and second because if the add the enabled for whatever reason there will be an error about key duplication.

I suggest we just do that:

{{#if useFingerprint}}
file_identity.fingerprint: ~
prospector:
  scanner:
    fingerprint:
      {{ fingerprintConfig }}
{{/if}}

Copy link
Contributor

@tetianakravchenko tetianakravchenko Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is not supported but can never happened as if the set useFingerprint false, then the prospector.scanner.fingerprint won't be include in the final config.

why? it can happen
useFingerprint: true, but variable
fingerprintYaml is:

enabled: false

If we do not include the enabled parameter, then it is error prone.

Yes, it makes sense, I understand your point

I suggest we just do that:

{{#if useFingerprint}}
file_identity.fingerprint: ~
prospector:
 scanner:
   fingerprint:
     {{ fingerprintConfig }}
{{/if}}

👍 yes, I think it looks better, just to prevent unexpected scanners configuration.
To the default value I would add a comment:

enabled: true    # This must be set to `true`
offset: 0
length: 1024

or smth like that, so to make it explicitly said. wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if enabled is set to false but the file_identity.fingerprint is configured then there is an error. But I will add this comment to be more clear

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment added in b04da0a

@andrewkroh andrewkroh added the enhancement New feature or request label Jan 9, 2025
@elasticmachine
Copy link

💚 Build Succeeded

History

  • 💚 Build #20274 succeeded e7fb7dc5451351815bfe221eae46d9e49f57ee11
  • 💔 Build #20272 failed 34fb21a1dc614ffcd85d7718fb34bdaa8351fb9d
  • 💚 Build #20208 succeeded 8287dab663ec0d037db6af93f9c4b4b52fc020c9
  • 💚 Build #20183 succeeded 4f66657a6e0fb9940b699a4eafeb7fba6b5abb0b

Copy link

@MichaelKatsoulis MichaelKatsoulis merged commit 5a747d5 into elastic:main Jan 10, 2025
5 checks passed
@elastic-vault-github-plugin-prod

Package kubernetes - 1.70.0 containing this change is available at https://epr.elastic.co/package/kubernetes/1.70.0/

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
* Make prospector.scanner.fingerprint configurable
---------

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
* Make prospector.scanner.fingerprint configurable
---------

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
@MichaelKatsoulis MichaelKatsoulis deleted the make-fingerprint-configurable branch February 7, 2025 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:kubernetes Kubernetes Team:Cloudnative-Monitoring Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[kubernetes] Make fingerprinting optional or make fingerprint length configurable
4 participants