Security overview

This page gathers, for a security review, what Perf Sentinel and its companion service PerfSentinelHub expose, send, store and sign. Each statement gives the default and what is left to the operator, and points to the document that details it. Facts are checked against the engine at 0.25.4 and the Hub at 0.3.4.

Perf Sentinel is a single self-hosted binary that analyzes OpenTelemetry traces, in CI or as a daemon (watch). The Hub is an optional self-hosted service that keeps the history of findings across a fleet of daemons. Neither is a SaaS, and neither sends usage telemetry.

At a glance

Network flows

Inbound

The engine daemon listens on OTLP gRPC 4317 and OTLP HTTP 4318. The query API (/api/*), /metrics and /health are served on the same 4318 listener, so a firewall rule or a NetworkPolicy cannot let traces in while keeping the query API closed. A local JSON socket (/tmp/perf-sentinel.sock by default) is created with mode 0600. The capture command receives OTLP the same way, on 127.0.0.1 by default. The other commands listen on nothing.

The binary binds to 127.0.0.1 by default and logs a warning on any other address. The Helm chart sets 0.0.0.0, which a pod needs behind its Service, and moves access control to the Service and to the chart's NetworkPolicy, which is off by default.

The Hub listens on plain HTTP 8080. TLS is expected at the ingress.

Outbound

The engine makes no outbound call unless the configuration or the command line names a destination:

Reference data (carbon intensities, power tables) is compiled into the binary and never fetched at runtime.

Behind a corporate proxy, the engine's HTTPS calls, verify-hash --url included, follow the usual variables: an https:// destination goes through HTTPS_PROXY (or ALL_PROXY) over a CONNECT tunnel unless NO_PROXY exempts it, and the PEM certificates of SSL_CERT_FILE are trusted next to the bundled Mozilla roots, which covers a TLS-inspecting proxy and an internal PKI. http:// destinations always connect directly, HTTP_PROXY included, and only an http:// proxy URL is used, so a SOCKS or https:// proxy never receives a plaintext CONNECT. The operating system's certificate store is not read. See Outbound proxy and private CA.

The Hub calls its configured daemon sources (poll, ack relay, live view), and the engine subprocess it launches reaches the trace backend set in its configuration. With sign-in enabled, it calls the identity provider's token and userinfo endpoints. Once a day, it asks the GitHub releases API for the latest engine and Hub versions. This check is on by default, and hub.updateCheck.enabled: false turns it off for a cluster with no egress.

Data handled and stored

Normalization. Spans are processed in memory, inside a streaming window (30 s TTL, 10,000 traces at most by default). SQL literals become ?, numeric and UUID path segments become {id} and {uuid}, and the query string is dropped. A finding carries the resulting template and a count of distinct values, not the values. A few things stay verbatim, such as SQL comments, non-numeric path segments and, outside MySQL and MariaDB, text between double quotes: see What stays verbatim in a template. A finding also carries the service name, the grouping attribute values, the code location and the trace id.

What the daemon writes to disk.

• The ack log, acks.jsonl, under the user's local data directory, mode 0600.
• The per-window NDJSON archive and the incidents archive, both opt-in, mode 0600, opened without following symlinks.
• Never raw spans.

The capture command is the exception by design: it writes the raw spans it receives, literals included, to an NDJSON file. Treat that file as sensitive.

What the Hub stores. The Hub keeps each finding as the engine sent it, in a SQLite database (/data/hub.db): template, trace id, grouping attribute values, code location, ack author and reason. It does no masking of its own. Findings are kept 180 days, analysis reports 24 hours and analysis runs 30 days by default.

Secrets.

• Environment variables take precedence over the configuration file for the engine's API keys and tokens.
• Endpoint URLs carrying user:pass@ are rejected at load time, and credentials are redacted from logs.
• The Hub passes a source's credential to the engine subprocess through an environment variable, never on the command line.

Access control

Engine

These are never authenticated, whatever the configuration:

• OTLP ingest, gRPC and HTTP.
• /metrics and /health.

The read endpoints /api/findings, /api/findings/{trace_id}, /api/explain/{id}, /api/correlations, /api/status, /api/config, /api/energy and /api/export/report are open by default. [daemon] require_read_key = true puts them, and every other GET of the query API, behind [daemon] read_api_key or the write key the route's handler accepts (Read gate). Every reader then needs the key, the Hub, Grafana, perf-sentinel query, the live HTML report and the JetBrains plugin among them, so do not turn it on for a daemon developers' IDEs read before the plugin installed there sends a per-endpoint credential.

The daemon trusts its trace senders. This is the stated threat model: No authentication.

Key-protected routes:

• Ack writes (POST and DELETE /api/findings/{signature}/ack) are open until [daemon.ack] api_key or PERF_SENTINEL_ACK_API_KEY is set.
• GET /api/acks then accepts that key or [daemon] read_api_key.
• GET and POST /api/incidents require [daemon.incidents] api_key when incidents are enabled, and GET also accepts [daemon] read_api_key.

Keys travel in X-API-Key or Authorization: Bearer and are compared in constant time. The ack author (by) is declared by the caller, not authenticated.

CORS is off by default, and a wildcard origin combined with a write key or with require_read_key is refused at startup. TLS covers both OTLP listeners when tls_cert_path and tls_key_path are set in [daemon]. Client certificates (mTLS) are not supported.

Hub

• Findings push: requires the source's X-API-Key (at least 32 characters, bound to its source, compared in constant time). A source without a key cannot push.
• UI and API: open by default.
• Sign-in: with hub.auth.enabled, the UI requires a session obtained through OAuth2 authorization code with PKCE against your identity provider (Keycloak, Entra ID, Google, GitLab and others). The session cookie is Secure, SameSite=Lax, 8 hours sliding. This is not full OpenID Connect: the identity comes from the userinfo endpoint, the ID token is not validated, and roles come only with access control, below.
• Access control: with hub.access.enabled, which needs sign-in and hub.metricsPort, each caller reaches only what its grants cover, per team, on sources, services or grouping attributes. CI jobs and dashboards read with machine tokens declared in the Hub's configuration, their values in a Secret, sent as Authorization: Bearer and holding findings.read at most. A session then ends 8 hours after sign-in. A machine token in a shared tool such as Grafana serves its whole scope to every user of that tool, so give each team its own.
• Routes open even with sign-in: /api/findings and /api/findings/{traceId} unless access control is on, the import routes (each with its own key, the disclosure import included), /health and /metrics (on hub.metricsPort alone when it is set). Keep them on an internal network.
• Share links (0.7.0, disclosures on): /share/<token>/<file> serves a published disclosure's five files to whoever holds the token, with no session: never a page, never a redirect, one identical 404 for an unknown, expired, revoked or withdrawn token. The token is stored as its SHA-256, expires within 30 days and stays out of the Hub's request log, but an ingress access log sees it. Internal reports are never shared.
• Launcher: it runs the bundled engine binary as a subprocess with an argument list, without a shell. The binary path and the trace endpoints come from the configuration only. User inputs are validated, and a run stops after 300 s by default.

Details: Hub authentication, Hub access control, Hub limitations.

Hardening

Engine code.

• The core library carries #![forbid(unsafe_code)]. The CLI holds four unsafe blocks, all libc calls (killpg to stop a captured process group, getrusage for bench).
• On musl, the allocator is mimalloc, and the TLS stack (ring) includes C and assembly.
• Clippy runs in pedantic mode with warnings as errors, and CodeQL analyzes the Rust code.
• The SQL normalizer, the JSON ingest and the disclosure PDF reader have fuzzing targets in fuzz/. CI runs them for 20 minutes each every week, and for 2 minutes on pull requests that touch them.
• verify-hash --report report.pdf parses an untrusted PDF: a PDF over 4 MiB is refused before parsing, a file declaring more than 32 object or cross-reference streams too, every stream is decoded under a cap (64 MiB for the report, 1 MiB for the attestation and the bundle), and the name tree walk is bounded in depth and size. The attestation and bundle it reads, from a PDF or beside a --url report, go to a private temporary directory removed when the command returns.

Engine limits.

• Payloads are capped at 16 MiB, compressed and decompressed (max_payload_size).
• Concurrency is capped: 32 OTLP HTTP requests, 32 gRPC requests, 256 gRPC streams and 128 socket connections.
• A request times out after one minute.
• A response holds at most 1,000 rows.
• TLS handshakes time out after 10 s, with at most 128 in flight.
• Memory admission control (cgroup v2) is opt-in.
• There is no per-client rate limiting.

Engine HTML report. It sets a Content Security Policy and renders every value with textContent, and a test fails the build if the template uses an unsafe DOM API.

Hub code.

• .NET 10 compiled with NativeAOT, nullable references, warnings as errors.
• Two runtime packages (Microsoft.Data.Sqlite, SQLitePCLRaw) and parameterized SQL.
• Request bodies are capped at 2 MiB (import, analyses) and 8 KiB (ack), and concurrency gates return 503 with Retry-After.
• The Hub sets no security header (CSP, HSTS, X-Frame-Options) besides Content-Security-Policy: frame-ancestors 'self' with access control on: add the rest at the ingress.

Containers and Helm. Both charts:

• run as non-root with a read-only root filesystem
• drop all capabilities and disallow privilege escalation
• apply the RuntimeDefault seccomp profile

The engine chart also disables the service account token. Its NetworkPolicy template is off by default and, once enabled, opens 4317 and 4318 together, so any pod allowed to send traces can also read the query API. The Hub chart ships no NetworkPolicy and no Ingress.

Supply chain

Engine.

• Every GitHub Action is pinned to a commit SHA.
• Cargo.lock is committed, and cargo audit and cargo deny run daily.
• Trivy scans the image before each release and blocks on fixable HIGH or CRITICAL vulnerabilities.
• Gitleaks scans the full history.
• Release tags are signed, and GitHub shows them as verified.

Release binaries carry:

• a SLSA build provenance attestation, Build Level 2 by GitHub's own documentation
• embedded cargo-auditable dependency data
• an attested SPDX SBOM

The Helm chart is signed with Cosign. The container images carry no Cosign signature and the release workflow issues no attestation for them. BuildKit adds an unsigned provenance document to the image index, not tied to the binary's attestation. See Supply chain pinning policy for the details and Software supply chain for the chart.

# releases built since the move to the INR organization
gh attestation verify perf-sentinel-linux-amd64 --repo Institut-du-Numerique-Responsable/perf-sentinel
gh attestation verify perf-sentinel-linux-amd64 --repo Institut-du-Numerique-Responsable/perf-sentinel \
  --predicate-type https://spdx.dev/Document/v2.3
# v0.26.2 and earlier, attested under the original account
gh attestation verify perf-sentinel-linux-amd64 --owner robintra --signer-repo robintra/perf-sentinel
cargo audit bin perf-sentinel-linux-amd64

Hub.

• Every release artefact, the OCI image tarball and the chart included, is signed with Cosign (sign-blob) and carries a GitHub build provenance and an SPDX SBOM attestation.
• The release builds the image twice and compares the results byte for byte.
• NuGet restores are locked.
• Actions are pinned by SHA.
• CodeQL, SonarCloud, Trivy, OSV-Scanner, Gitleaks, TruffleHog and OpenSSF Scorecard run in CI.

The image pushed to GHCR is not signed in the registry: verify the release artefacts as described in Hub releasing.

Both repositories require signed commits on their main branch, the engine through branch protection and the Hub through its Protect default branch ruleset. On both, the Administrator role can bypass the rule. On the engine, commits merged before 3 October 2026 were rewritten by GitHub's rebase merge and carry no signature. Release tags are signed on both.

Reporting a vulnerability

Both projects take reports through GitHub private vulnerability reporting.

• Engine: acknowledgment within 72 hours, best effort since the project has a single maintainer for now. Initial assessment within 7 days. A CVE is requested from Medium severity up. Only the latest minor release receives fixes. See SECURITY.md.
• Hub: acknowledgment within 3 business days, assessment within 7 business days. Only the latest 0.x release receives fixes. See the Hub security policy.

Deployment recommendations

1. Keep OTLP ingestion on a trusted network. The daemon trusts its senders.
2. Enable the engine chart's NetworkPolicy with namespace or pod selectors. To keep the query API from everything that sends traces, put a reverse proxy in front of 4318 that filters /api/*.
3. If you use acks, set PERF_SENTINEL_ACK_API_KEY from a Secret. Without it, anyone who reaches the port can acknowledge a finding.
4. Encrypt the traffic with tls_cert_path and tls_key_path in [daemon], a service mesh or the ingress.
5. For the Hub, turn on hub.auth, terminate TLS and add the security headers at the ingress, and write a NetworkPolicy, since the chart ships none.
6. In a cluster with no egress, set hub.updateCheck.enabled: false.
7. Set http.route in your instrumentation, and keep personal data out of URL path segments and SQL comments.
8. Treat the files written by capture as sensitive. They hold raw spans.
9. If your policy requires signed images, verify the release binary and build or sign the image in your own registry.
10. Behind a TLS-inspecting proxy, set HTTPS_PROXY and NO_PROXY, and put the proxy's root CA in the SSL_CERT_FILE bundle.