Loading

Configure the EDOT Java agent

Stack Serverless Observability EDOT Java

The minimal configuration section provides a recommended starting point for EDOT Java configuration.

See configuration options for details on the supported configuration options and configuration methods for how to provide them.

Note

Declarative configuration is not supported. Using it deactivates many agent features.

This configuration is provided using environment variables, other configuration methods are also supported.

# service name: mandatory for integration in UI and correlation
OTEL_SERVICE_NAME=my-service

# resource attributes: recommended for integration in UI and correlation, can also include service.name
OTEL_RESOURCE_ATTRIBUTES='service.version=1.0,deployment.environment.name=production'

# exporter endpoint: mandatory if not using a local Collector accessible on http://localhost:4317
OTEL_EXPORTER_OTLP_ENDPOINT=https://my-otel-collector

# exporter authentication: mandatory if endpoint requires authentication
OTEL_EXPORTER_OTLP_HEADERS='Authorization=ApiKey mySecretApiKey'

For authentication, the OTEL_EXPORTER_OTLP_HEADERS can also be used with an APM secret token:

OTEL_EXPORTER_OTLP_HEADERS='Authorization=Bearer mySecretToken'

EDOT Java instrumentation agent is based on OpenTelemetry Java SDK and Instrumentation, and thus supports the following configuration options:

EDOT Java uses different defaults than the OpenTelemetry Java instrumentation for the following configuration options:

Option EDOT Java default OpenTelemetry Java agent default EDOT Java version
OTEL_RESOURCE_PROVIDERS_AWS_ENABLED true false (docs) 1.0.0+
OTEL_RESOURCE_PROVIDERS_GCP_ENABLED true false (docs) 1.0.0+
OTEL_RESOURCE_PROVIDERS_AZURE_ENABLED true false (docs) 1.4.0+
OTEL_INSTRUMENTATION_RUNTIME-TELEMETRY_EMIT-EXPERIMENTAL-TELEMETRY true false (docs) 1.4.0+
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE delta (*) cumulative (docs) 1.0.0+

(*) default value set to delta only if not already explicitly set.

The EDOT Java instrumentation agent also provides configuration options for each of the supported features. This table only contains minimal configuration, see each respective feature for exhaustive configuration options documentation.

Option Default Feature EDOT Java version
OTEL_INFERRED_SPANS_ENABLED false Inferred spans 1.0.0+
OTEL_JAVA_EXPERIMENTAL_SPAN_STACKTRACE_MIN_DURATION 5ms Span stacktrace 1.0.0+
ELASTIC_OTEL_UNIVERSAL_PROFILING_INTEGRATION_ENABLED auto Elastic Universal profiling integration 1.0.0+
OTEL_INSTRUMENTATION_OPENAI_CLIENT_ENABLED true OpenAI client instrumentation 1.4.0+
ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL INFO Agent logging 1.5.0+
ELASTIC_OTEL_VERIFY_SERVER_CERT true Exporter certificate verification 1.5.0+

The following settings are available:

Option default description
OTEL_INSTRUMENTATION_OPENAI_CLIENT_ENABLED true enables or disable OpenAI instrumentation
ELASTIC_OTEL_JAVA_INSTRUMENTATION_GENAI_EMIT_EVENTS value of OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT If set to true, the agent will generate log events for OpenAI requests and responses. Potentially sensitive content will only be included if OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is true
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT false If set to true, enables the capturing of OpenAI request and response content in the log events outputted by the agent. ↪

Stack Preview 9.1.0 Serverless Unavailable EDOT Java Preview 1.5.0

APM Agent Central Configuration lets you configure EDOT Java instances remotely, see Central configuration docs for more details.

To activate central configuration, set the ELASTIC_OTEL_OPAMP_ENDPOINT environment variable to the OpAMP server endpoint.

export ELASTIC_OTEL_OPAMP_ENDPOINT=http://localhost:4320/v1/opamp

To deactivate central configuration, remove the ELASTIC_OTEL_OPAMP_ENDPOINT environment variable and restart the instrumented application.

You can modify the following settings for EDOT Java through APM Agent Central Configuration:

Setting Central configuration name Type
Logging level logging_level Dynamic
Turn off instrumentations deactivate_instrumentations Dynamic
Turn off all instrumentations deactivate_all_instrumentations Dynamic
Send traces send_traces Dynamic
Send metrics send_metrics Dynamic
Send logs send_logs Dynamic

Dynamic settings can be changed without having to restart the application.

Configuration can be provided through multiple configuration methods:

Configuration options are applied with the following priorities:

Important

Declarative configuration is not supported.

Environment variables provide a cross-platform way to configure EDOT Java and is especially useful in containerized environments.

Define environment variables before starting the JVM:

export OTEL_SERVICE_NAME=my-service
java ...

These configuration options can be seen by anything that can see the executed command-line.

Define system properties at the JVM start, usually on the command-line:

java -Dotel.service.name=my-service ...

When modifying the JVM command line options is not possible, using the JAVA_TOOL_OPTIONS environment variable could be used to provide the system properties, for example:

export JAVA_TOOL_OPTIONS='-Dotel.service.name=my-service'

EDOT Java can be configured using a java properties configuration file.

Before starting the JVM, create and populate the configuration file and specify where to find it:

echo otel.service.name=my-service > my.properties
java -Dotel.javaagent.configuration-file=my.properties ...

The EDOT Java agent provides the ability to control the agent log verbosity by setting the log level with the ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL configuration option (INFO by default).

The following log levels are supported: TRACE, DEBUG, INFO, WARN, ERROR, FATAL and OFF.

For troubleshooting, the ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL=DEBUG is a recommended alternative to OTEL_JAVAAGENT_DEBUG=true as it provides span information in JSON format.

This feature relies on the OTEL_JAVAAGENT_LOGGING configuration option to be set to elastic (default), the simple value from upstream is not supported.

Setting OTEL_JAVAAGENT_LOGGING=none or ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL=OFF disables agent logging feature.

Setting OTEL_JAVAAGENT_LOGGING=application will disable EDOT agent logging feature and attempt to use the application logger. As documented here in the upstream documentation, support for this depends on the application and logging libraries used.

The EDOT Java agent provides the ability to toggle the exporter endpoint certificate verification with the ELASTIC_OTEL_VERIFY_SERVER_CERT configuration option (true by default).

When the endpoint certificate is not trusted by the JVM where the agent runs, the common symptom is security-related exceptions with the following message: unable to find valid certification path to requested target.

This is common in the following scenarios:

  • When endpoint uses a self-signed certificate not trusted by the JVM
  • When the certificate authority used by the endpoint certificate is not trusted by the JVM

One solution is to add the certificate or certificate authority to the JVM trust store, which requires modifying the JVM trust store.

If trust store modification is not possible or not practical, for example when troubleshooting or working with a local deployment, certificate verification can be disabled by setting ELASTIC_OTEL_VERIFY_SERVER_CERT to false. This however need to be evaluated carefully as it lowers the communication security and could allow for man-in-the-middle attacks where the data could be intercepted between the agent and the collector endpoint.