otelcol.exporter.kafka

The authentication block holds the definition of different authentication mechanisms to use when connecting to Kafka brokers. It doesn’t support any arguments and is configured fully through inner blocks.

The kerberos block configures Kerberos authentication against the Kafka broker.

The following arguments are supported:

When use_keytab is false, the password argument is required. When use_keytab is true, the file pointed to by the keytab_file argument is used for authentication instead. At most one of password or keytab_file must be provided.

disable_fast_negotiation is useful for Kerberos implementations which don’t support PA-FX-FAST (Pre-Authentication Framework - Fast) negotiation.

The plaintext block configures plain text authentication against Kafka brokers.

The following arguments are supported:

The sasl block configures SASL authentication against Kafka brokers.

The following arguments are supported:

You can set the mechanism argument to one of the following strings:

• "PLAIN"
• "SCRAM-SHA-256"
• "SCRAM-SHA-512"
• "AWS_MSK_IAM_OAUTHBEARER"

When mechanism is set to "AWS_MSK_IAM_OAUTHBEARER", the aws_msk child block must also be provided.

You can set the version argument to either 0 or 1.

The aws_msk block configures extra parameters for SASL authentication when using the AWS_MSK_IAM_OAUTHBEARER mechanisms.

The following arguments are supported:

The following arguments are supported:

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If you set reload_interval to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

• ca_pem and ca_file
• cert_pem and cert_file
• key_pem and key_file

If cipher_suites is left blank, a safe default list is used. Refer to the Go TLS documentation for a list of supported cipher suites.

The curve_preferences argument determines the set of elliptic curves to prefer during a handshake in preference order. If not provided, a default list is used. The set of elliptic curves available are X25519, P521, P256, and P384.

The following arguments are supported:

The trusted platform module (TPM) configuration can be used for loading TLS key from TPM. Currently only TSS2 format is supported.

The path attribute is not supported on Windows.

Example

alloy Copy

otelcol.example.component "<LABEL>" {
    ...
    tls {
        ...
        key_file = "my-tss2-key.key"
        tpm {
            enabled = true
            path = "/dev/tpmrm0"
        }
    }
}

In the above example, the private key my-tss2-key.key in TSS2 format will be loaded from the TPM device /dev/tmprm0.

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

disable_high_cardinality_metrics is the Alloy equivalent to the telemetry.disableHighCardinalityMetrics feature gate in the OpenTelemetry Collector. It removes attributes that could cause high cardinality metrics. For example, attributes with IP addresses and port numbers in metrics about HTTP and gRPC connections are removed.

The metadata block configures how to retrieve and store metadata from the Kafka broker.

The following arguments are supported:

When full is set to false, the client does not make the initial request to broker at the startup.

Retrieving metadata may fail if the Kafka broker is starting up at the same time as the Alloy component. The retry child block can be provided to customize retry behavior.

The retry block configures how to retry retrieving metadata when retrieval fails.

The following arguments are supported:

The following arguments are supported:

When enabled is true, failed batches are retried after a given interval. The initial_interval argument specifies how long to wait before the first retry attempt. If requests continue to fail, the time to wait before retrying increases by the factor specified by the multiplier argument, which must be greater than 1.0. The max_interval argument specifies the upper bound of how long to wait between retries.

The randomization_factor argument is useful for adding jitter between retrying Alloy instances. If randomization_factor is greater than 0, the wait time before retries is multiplied by a random factor in the range [ I - randomization_factor * I, I + randomization_factor * I], where I is the current interval.

If a batch hasn’t been sent successfully, it’s discarded after the time specified by max_elapsed_time elapses. If max_elapsed_time is set to "0s", failed requests are retried forever until they succeed.

The following arguments are supported:

The blocking argument is deprecated in favor of the block_on_overflow argument.

When block_on_overflow is true, the component will wait for space. Otherwise, operations will immediately return a retryable error.

When enabled is true, data is first written to an in-memory buffer before sending it to the configured server. Batches sent to the component’s input exported field are added to the buffer as long as the number of unsent batches doesn’t exceed the configured queue_size.

queue_size determines how long an endpoint outage is tolerated. Assuming 100 requests/second, the default queue size 1000 provides about 10 seconds of outage tolerance. To calculate the correct value for queue_size, multiply the average number of outgoing requests per second by the time in seconds that outages are tolerated. A very high value can cause Out Of Memory (OOM) kills.

The sizer argument could be set to:

• requests: number of incoming batches of metrics, logs, traces (the most performant option).
• items: number of the smallest parts of each signal (spans, metric data points, log records).
• bytes: the size of serialized data in bytes (the least performant option).

The num_consumers argument controls how many readers read from the buffer and send data in parallel. Larger values of num_consumers allow data to be sent more quickly at the expense of increased network traffic.

If an otelcol.storage.* component is configured and provided in the queue’s storage argument, the queue uses the provided storage extension to provide a persistent queue and the queue is no longer stored in memory. Any data persisted will be processed on startup if Alloy is killed or restarted. Refer to the exporterhelper documentation in the OpenTelemetry Collector repository for more details.

The following arguments are supported:

max_size must be greater than or equal to min_size.

The sizer argument can be set to:

• items: The number of the smallest parts of each span, metric data point, or log record.
• bytes: the size of serialized data in bytes (the least performant option).