Normalization: SQL and HTTP

Normalization is the second pipeline stage. It transforms raw SpanEvents into NormalizedEvents by extracting a template (parameterized query or URL pattern) and the concrete parameter values.

Why not use sqlparser?

The sqlparser crate is a full SQL parser that builds an AST. We chose a homemade tokenizer instead:

• Binary size: a full parser and its dialect grammars add code to the release binary. Perf Sentinel targets < 15 MB total.
• Dependency weight: sqlparser pulls in additional crates and increases compile time.
• No dialect required: sqlparser requires specifying a SQL dialect (PostgreSQL, MySQL, etc.). Our tokenizer produces a template for any dialect because it only replaces literals and never needs to understand query structure. The engine named in db.system only adjusts quoting rules (MySQL double-quoted strings, backslash escapes), and an unknown engine gets the privacy-safe reading.
• Performance: a full parser builds an AST we would immediately discard. Our single-pass tokenizer processes input in O(n) with no intermediate data structure.
• Simplicity: a few hundred lines of code vs a full parser and its dialect grammars.

The trade-off is documented in Limitations: the delimiters and keywords the tokenizer acts on (quotes, $, digits, IN) must be ASCII, while literal content can be any UTF-8, and it does not perform semantic analysis. It supports CTEs, double-quoted identifiers, PostgreSQL dollar-quoted strings and CALL statements.

This "never understands query structure" property is sufficient for every stage of the pipeline. Every detector (n_plus_one, redundant, fanout, …) and the sanitizer_aware classifier reason over trace shape (occurrence counts, timing variance, span ordering, ORM instrumentation scope) and the query fingerprint, never the SQL's internal grammar. Structural SQL analysis would only pay off if Perf Sentinel pivoted into single-query static analysis (a different product category), and even then an EXPLAIN plan beats re-parsing the logged text.

The tokenizer is also total. It always emits a best-effort template, even on truncated or unknown-dialect SQL (see the unterminated-literal handling), whereas a real parser rejects input it cannot parse. Since trace SQL is whatever a driver happened to log, a strict parser would need this tokenizer as a fallback anyway.

SQL tokenizer: single-pass state machine

normalize_sql() processes the query byte-by-byte through six states. normalize_sql_for(query, db_system) runs the same machine, reading " as a string quote when db_system is mysql or mariadb. The pipeline passes the span's db.system, mysql-stat passes mysql and pg-stat passes postgresql:

Whether a backslash escapes inside InString is decided when the string opens. The engines of STANDARD_STRING_SYSTEMS (PostgreSQL, CockroachDB, Vertica, SQL Server, Oracle, DB2, SQLite, H2, HSQLDB, Derby, Trino, Presto, Teradata) follow the standard and escape only in an E'...' string, an E or e right before the quote with no identifier byte before it. Every other engine, MySQL, MariaDB and an unnamed one included, escapes in every string. For an unknown engine this is the privacy-safe side: a missed escape closes the string early and leaks the rest of it into the template, while a wrong one only masks more than needed. The param keeps the escape as written.

Batch push_str optimization

Instead of pushing characters one at a time with template.push(b as char), the tokenizer tracks a normal_start index:

// On entering InString or InNumber:
if i > normal_start {
    template.push_str(&query[normal_start..i]);
}
// On returning to Normal:
normal_start = i;
// At end of input (still in Normal):
template.push_str(&query[normal_start..len]);

This batches contiguous Normal-state runs into a single push_str call. For a typical query like SELECT * FROM player WHERE game_id = 42, the SELECT * FROM player WHERE game_id = prefix is flushed in one call instead of 38 individual push calls.

The Rust String::push_str implementation copies bytes with memcpy, which is significantly faster than repeated push calls that each check capacity and potentially reallocate.

IN-list regex skip

Most SQL queries do not contain IN (...) clauses. The tokenizer tracks whether the IN keyword appears:

if !has_in_list
    && (b == b'I' || b == b'i')
    && i + 1 < len
    && (bytes[i + 1] == b'N' || bytes[i + 1] == b'n')
    && (i == 0 || bytes[i - 1].is_ascii_whitespace())
    && (i + 2 >= len || !bytes[i + 2].is_ascii_alphanumeric())
{
    has_in_list = true;
}

If has_in_list is false after the main loop, the regex post-pass (IN_LIST_RE.replace_all) is skipped entirely. Queries with no IN clause never run the regex.

Cow::Borrowed optimization

When the regex does run but makes no replacements (e.g., IN (?) is already collapsed), Regex::replace_all returns Cow::Borrowed. The code checks for this:

let template = if has_in_list {
    match IN_LIST_RE.replace_all(&template, "IN (?)") {
        Cow::Borrowed(_) => template,    // no allocation
        Cow::Owned(s) => s,              // one allocation
    }
} else {
    template                              // no regex at all
};

LazyLock for regex

The IN_LIST_RE regex is compiled once via std::sync::LazyLock (stable since Rust 1.80):

static IN_LIST_RE: LazyLock<Regex> = LazyLock::new(|| {
    Regex::new(r"(?i)IN\s*\(\s*\?(?:\s*,\s*\?)*\s*\)").unwrap()
});

LazyLock is preferred over the lazy_static! macro because it is in std, no external dependency needed.

Other micro-optimizations

• String::with_capacity(query.len()): pre-allocates the template to avoid reallocation in the common case where the template is slightly shorter than the input.
• std::mem::take(&mut current_value): moves the accumulated literal value into params without cloning, replacing current_value with an empty String in place. This is a zero-cost ownership transfer.
• is_identifier_byte_before(): checks whether the byte before a digit is alphanumeric or underscore, preventing digits within identifiers (player2, col_1) from being misinterpreted as numeric literals.

HTTP normalizer

Hand-coded UUID check

The HTTP normalizer replaces UUID path segments with {uuid}. Instead of using a regex, the check is hand-coded:

fn is_uuid(s: &str) -> bool {
    if s.len() != 36 { return false; }
    let b = s.as_bytes();
    b[8] == b'-' && b[13] == b'-' && b[18] == b'-' && b[23] == b'-'
        && b.iter().enumerate().all(|(i, &c)| {
            matches!(i, 8 | 13 | 18 | 23) || c.is_ascii_hexdigit()
        })
}

Why hand-coded? This function is called on every path segment of every HTTP URL in the pipeline. A compiled regex (Regex::is_match) runs the regex engine on every call. The hand-coded check is a length check (fast rejection for >99% of segments), four byte comparisons for dash positions and a single pass for hex digits.

split_origin and host-aware grouping

fn split_origin(target: &str) -> (Option<&str>, &str) {
    match target
        .strip_prefix("http://")
        .or_else(|| target.strip_prefix("https://"))
    {
        Some(rest) => match rest.find('/') {
            Some(idx) => (Some(&rest[..idx]), &rest[idx..]),
            None => (Some(rest), "/"),
        },
        None => (None, target),
    }
}

This splits the authority from the path without pulling in the url crate (~50KB binary overhead). It handles http://, https:// and bare paths (/api/foo, which have no authority).

The callee host is then kept in the template for DNS-addressed calls (GET user-svc/api/foo) and dropped for IP-literal authorities. host_group_prefix classifies the authority. An IPv4 dotted-decimal or a bracketed IPv6 literal is a load-balanced replica address and is dropped, so pods behind one service keep deduping into one template. A DNS hostname is lowercased, stripped of RFC 3986 userinfo and port, and prepended to the path.

Keeping the host stops two calls to the same path on different backends (http://ms-a/x, http://ms-b/x) from collapsing into one template and raising a false redundant_http. Because the finding signature hashes the template (acknowledgments::compute_signature), an outbound-HTTP finding's ack signature depends on the callee host.

Query parameter limit

Query parameters are stripped from the URL template and collected into params. The collection is capped at 100 parameters via .take(100) to prevent unbounded memory allocation from URLs with adversarially large query strings. Since query parameters are not part of the normalized template, excess parameters beyond 100 are not extracted.

Pre-allocation

let mut result = String::with_capacity(path.len() + 8);

The + 8 accounts for the longest replacement ({uuid} = 6 chars, replacing a 36-char UUID). This avoids reallocation in the common case where replacements make the path shorter.

Normalization dispatcher

The normalize() function dispatches on event_type:

pub fn normalize(event: SpanEvent) -> NormalizedEvent {
    match event.event_type {
        EventType::Sql => { /* sql::normalize_sql(...) */ }
        EventType::HttpOut => { /* http::normalize_http(...) */ }
        EventType::Messaging => { /* "{operation} {target}", no params */ }
    }
}

The messaging arm normalizes nothing: a topic or queue name is already a template, with no variable part to extract. Routing it through normalize_http would mask numeric segments as {id} and strip the account id out of an SQS queue ARN, merging two AWS accounts into one template. The one transformation it does apply lives in sanitize_span_event, not here. A destination carrying a scheme (amqp://user:pass@host/q) has its userinfo and query stripped, because unlike SQL and HTTP this arm has no parser between an operator-supplied string and the finding template. A scheme-less destination is left byte-for-byte alone, since @ and # are legal in queue names (ORDERS@QM1, logs.#) and rewriting them would merge distinct destinations.

normalize_all() is a simple events.into_iter().map(normalize).collect(). The into_iter() consumes the input vector and each SpanEvent is moved (not cloned) into the normalizer.

Defense-in-depth

Query truncation. normalize_sql truncates input at MAX_QUERY_LEN (64 KB) before processing to prevent the state-machine tokenizer from running on adversarially large inputs. Truncation uses floor_char_boundary to avoid splitting multi-byte UTF-8 characters. This is a second layer after the sanitize_span_event field caps applied at the ingestion boundary.