Rotel
High Performance, Resource Efficient OpenTelemetry Collection
Install / Use
/learn @rotel-dev/RotelREADME
About
Rotel provides an efficient, high-performance solution for collecting, processing, and exporting telemetry data. Rotel is ideal for resource-constrained environments and applications where minimizing overhead is critical.
Features:
- Supports metrics, logs, and traces
- OTLP receiver supporting gRPC, HTTP/Protobuf, and HTTP/JSON
- OTLP exporter supporting gRPC and HTTP/Protobuf
- Built-in batching and retry mechanisms
- Additional exporters: ClickHouse, Datadog, AWS X-RAY, AWS EMF, and Kafka
- Kafka Receiver
- Kmsg Receiver (Linux kernel messages)
Rotel can be easily bundled with popular runtimes as packages. Its Rust implementation ensures minimal resource usage and a compact binary size, simplifying deployment without the need for a sidecar container.
Runtime integrations:
- Python: streamfold/pyrotel
- Node.js: streamfold/rotel-nodejs
We also have a custom AWS Lambda Extension layer that embeds the Rotel collector to provide OpenTelemetry collection with minimal coldstart latency:
- Rotel Lambda Extension: streamfold/rotel-lambda-extension
Rotel is fully open-sourced and licensed under the Apache 2.0 license.
Getting Started
To quickly get started with Rotel you can leverage the bundled Python or NodeJS packages. Or if you'd like to use Rotel directly from Docker, follow these steps:
-
Running Rotel
- We use the prebuilt docker image for this example, but you can also download a binary from the releases page.
- Execute Rotel with the following arguments. To debug metrics or logs, add
an additional
--debug-log metrics|logs.
docker run -ti -p 4317-4318:4317-4318 streamfold/rotel --debug-log traces --exporter blackhole- Rotel is now listening on localhost:4317 (gRPC) and localhost:4318 (HTTP).
-
Verify
- Send OTLP traces to Rotel and verify that it is receiving data:
go install github.com/open-telemetry/opentelemetry-collector-contrib/cmd/telemetrygen@latest telemetrygen traces --otlp-insecure --duration 5sAlternatively, use the built-in
generate-otlptool:# Generate and send traces directly to Rotel cargo run --bin generate-otlp -- traces --http-endpoint localhost:4318 # Or generate a trace file for testing cargo run --bin generate-otlp -- traces --file trace.pb # Then send it with curl curl -X POST http://localhost:4318/v1/traces \ -H "Content-Type: application/x-protobuf" \ --data-binary @trace.pb- Check the output from Rotel and you should see several "Received traces" log lines.
Configuration
Rotel is configured on the command line with multiple flags. See the table below for the full list of options. Rotel will also output the full argument list:
rotel start --help
All CLI arguments can also be passed as environment variable by prefixing with ROTEL_ and switching hyphens to
underscores. For example, --otlp-grpc-endpoint localhost:5317 can also be specified by setting the environment
variable ROTEL_OTLP_GRPC_ENDPOINT=localhost:5317.
Any option above that does not contain a default is considered false or unset by default.
| Option | Default | Options | |-----------------------------------|----------------------|--------------------------------------------------------------------| | --daemon | | | | --log-format | text | json | | --pid-file | /tmp/rotel-agent.pid | | | --log-file | /tmp/rotel-agent.log | | | --debug-log | | metrics, traces, logs | | --debug-log-verbosity | basic | basic, detailed | | --otlp-grpc-endpoint | localhost:4317 | | | --otlp-http-endpoint | localhost:4318 | | | --otlp-grpc-max-recv-msg-size-mib | 4 | | | --exporter | otlp | otlp, blackhole, datadog, clickhouse, awsxray, awsemf, kafka, file | | --otlp-receiver-traces-disabled | | | | --otlp-receiver-metrics-disabled | | | | --otlp-receiver-logs-disabled | | | | --otlp-receiver-traces-http-path | /v1/traces | | | --otlp-receiver-metrics-http-path | /v1/metrics | | | --otlp-receiver-logs-http-path | /v1/logs | | | --otel-resource-attributes | | | | --enable-internal-telemetry | | |
The PID and LOG files are only used when run in --daemon mode.
See the section for Multiple Exporters for how to configure multiple exporters
OTLP exporter configuration
The OTLP exporter is the default, or can be explicitly selected with --exporter otlp.
| Option | Default | Options | |----------------------------------------|--------------------------------|------------------| | --otlp-exporter-endpoint | | | | --otlp-exporter-protocol | grpc | grpc, http | | --otlp-exporter-custom-headers | | | | --otlp-exporter-compression | gzip | gzip, none | | --otlp-exporter-authenticator | | sigv4auth, basic | | --otlp-exporter-tls-cert-file | | | | --otlp-exporter-tls-cert-pem | | | | --otlp-exporter-tls-key-file | | | | --otlp-exporter-tls-key-pem | | | | --otlp-exporter-tls-ca-file | | | | --otlp-exporter-tls-ca-pem | | | | --otlp-exporter-tls-skip-verify | | | | --otlp-exporter-request-timeout | 5s | | | --otlp-exporter-pool-idle-timeout | 30s | | | --otlp-exporter-pool-max-idle-per-host | 100 | | | --otlp-exporter-retry-initial-backoff | (uses global exporter default) | | | --otlp-exporter-retry-max-backoff | (uses global exporter default) | | | --otlp-exporter-retry-max-elapsed-time | (uses global exporter default) | |
Any of the options that start with --otlp-exporter* can be set per telemetry type: metrics, traces or logs. For
example, to set a custom endpoint to export traces to, set: --otlp-exporter-traces-endpoint. For other telemetry
types their value falls back to the top-level OTLP exporter config.
Cloudwatch OTLP Export
The Rotel OTLP exporter can export to the Cloudwatch OTLP endpoints for traces and logs. You'll need to select the HTTP protocol and enable the sigv4auth authenticator.
The sigv4auth authenticator requires the AWS authentication credentials. See the AWS Authentication docs for supported methods.
Traces
_Tracing requires that you enable [Transaction Search](https://
