Advanced Regular Expressions

Concept

Module 9 introduced regexes. This module covers the features used in real log analysis and validation:

Capture accessed by name: m.Groups["ecu"] Group without creating a capture \1, \k<name> Match the same text again All values of a repeated group Assert what follows, without consuming Assert what precedes Match as little as possible Between a word and a non-word character

IgnoreCase, Multiline, Singleline, ExplicitCapture, IgnorePatternWhitespace Change matching behaviour Regex.Replace(s, p, m =>...)

MatchEvaluator computes each replacement Engines and Performance Interpreted (default)

Pattern parsed at run time and interpreted Compiled to IL at run time — faster matching, slower start-up

Source generator writes the matcher at compile time — fast, AOT friendly, readable generated code (Module 39)

Automata-based engine with linear-time guarantee; no backreferences or lookarounds

Catastrophic backtracking: nested quantifiers like ^(a+)+$ take exponential time on a non-matching input. It is a denial-of-service risk (ReDoS) when patterns meet untrusted input. Defences: always pass a timeout (RegexMatchTimeoutException), rewrite the pattern without nested quantifiers, or use NonBacktracking.

Why It Matters

Diagnostic logs, trace files and test reports are text. Regexes validate VINs and DTCs, extract versions, and mask sensitive data — and a single bad pattern on untrusted input can hang a server.

Syntax

[GeneratedRegex(@"(?<ecu>\w+): sw=(?<ver>[\d.]+)")] private static partial Regex Version();
var m = Version().Match(text); string ecu = m.Groups["ecu"].Value;
Regex.Replace(s, pattern, m => m.Value.ToUpper());
Regex.IsMatch(input, pattern, RegexOptions.None, TimeSpan.FromMilliseconds(100));

Example

43.1 Groups, lookarounds, lazy matching, replacement, engines and backtracking

using System.Diagnostics;
using System.Text.RegularExpressions;

// Character classes, quantifiers, anchors
Show(@"^\d{3}-\d{4}$", "555-1234", "5551234");
Show(@"^0x[0-9A-Fa-f]{1,8}$", "0x7E0", "0xZZ");

// Groups, named groups, captures and backreferences
Match m = Regex.Match("VCU: sw=2.4.1 hw=B2", @"(?<ecu>\w+): sw=(?<major>\d+)\.(?<minor>\d+)\.(?<patch>\d+)");
Console.WriteLine($"ecu {m.Groups["ecu"].Value}, version {m.Groups["major"]}.{m.Groups["minor"]}.{m.Groups["patch"]}, groups: {m.Groups.Count}");
Console.WriteLine($"repeated word (backreference \\1): {Regex.Match("the the CAN bus", @"\b(\w+) \1\b").Value}");
Match many = Regex.Match("IDs: 100,200,300", @"IDs: (?:(\d+),?)+");
Console.WriteLine($"one group, several captures: {string.Join(" ", many.Groups[1].Captures.Select(c => c.Value))}");

// Lookahead and lookbehind (zero-width assertions)
Console.WriteLine($"lookahead - numbers followed by ' V': {string.Join(",", Regex.Matches("12.6 V, 3 A, 13.1 V", @"\d+(\.\d+)?(?= V)").Select(x => x.Value))}");
Console.WriteLine($"lookbehind - values after 'id=': {string.Join(",", Regex.Matches("id=7E0 id=7E8 x=1", @"(?<=id=)[0-9A-F]+").Select(x => x.Value))}");
Console.WriteLine($"
…

Expected Result

The greedy .* swallowed both <dtc> elements while the lazy .*? stopped at the first closing tag. The nested pattern ^(a+)+$ hit the 200 ms timeout on only 31 characters, while the equivalent ^a+$ and the NonBacktracking engine answered immediately.

^\d{3}-\d{4}$            555-1234:True  5551234:False
^0x[0-9A-Fa-f]{1,8}$     0x7E0:True  0xZZ:False
ecu VCU, version 2.4.1, groups: 5
repeated word (backreference \1): the the
one group, several captures: 100 200 300
lookahead - numbers followed by ' V': 12.6,13.1
lookbehind - values after 'id=': 7E0,7E8
negative lookahead - DTCs not starting with U: P0A80,C1234
greedy .*  : <dtc>P0A80</dtc><dtc>U0100</dtc>
lazy   .*? : <dtc>P0A80</dtc>
VIN EHB************** and EHB**************
interpreted       found 20000 matches
compiled          found 20000 matches
source-generated  found 20000 matches
catastrophic backtracking: timed out after ~200 ms: True
rewritten pattern ^a+$ returns False immediately: True
RegexOptions.NonBacktracking on the evil pattern: False (linear time)

Common Mistakes

Nested quantifiers on untrusted input without a timeout (ReDoS).

Greedy .* where lazy .*? or a negated class [^<]* is intended.

Forgetting anchors (^...$) in validation — \d{3} matches inside 12345.

Constructing new Regex(...) inside loops.

Parsing HTML/XML/JSON with regexes instead of a parser.

Unescaped user input inside a pattern — use Regex.Escape.

Best Practices

Use [GeneratedRegex] for fixed patterns.

Name groups; use RegexOptions.ExplicitCapture when you only need named groups.

Always specify a timeout (or NonBacktracking) for patterns on external input.

Keep patterns readable (IgnorePatternWhitespace with comments) and unit-test them.

Interview Relevance

Medium: "greedy vs lazy?", "what is a lookahead?", "what is catastrophic backtracking / ReDoS?", "Compiled vs GeneratedRegex?".

Practice Questions

Extract (timestamp, id, data) from Vector ASC-style log lines using named groups.

Validate IPv4 addresses (0–255 per octet).

Replace every hex byte in a string with its decimal value using a MatchEvaluator.

Write a pattern that matches DTCs *not* followed by (inactive).

Quick Quiz

Q1. (Multiple choice) Which construct asserts that text is followed by V without including it in the match? A) (?: V) B) (?= V) C) (?<= V) D) \b V Answer: B (verified: 12.6,13.1).

Q2. (Predict the output) Regex.Match("<dtc>P0A80</dtc><dtc>U0100</dtc>", "<dtc>.*?</dtc>").Value

Answer: <dtc>P0A80</dtc> (verified).

Q3. (True/False) RegexOptions.NonBacktracking supports backreferences.

Answer: False. It trades those features for guaranteed linear-time matching.

Q4. (Find the bug) Regex.IsMatch(userInput, @"^(\w+\s?)+$") hangs for some inputs.

Answer: Nested quantifiers cause catastrophic backtracking; add a timeout and rewrite the pattern (or use NonBacktracking).

Coding Challenge

Using [GeneratedRegex] with named groups, extract ecu and version from "VCU sw=2.4.1; BMS sw=1.9.0; GW sw=3.2" and print them.

using System.Text.RegularExpressions;

string text = "VCU sw=2.4.1; BMS sw=1.9.0; GW sw=3.2";
foreach (Match m in Rx.Firmware().Matches(text))
    Console.WriteLine($"{m.Groups["ecu"].Value,-4} {m.Groups["version"].Value}");

static partial class Rx
{
    [GeneratedRegex(@"(?<ecu>[A-Z]+) sw=(?<version>\d+(?:\.\d+)+)")]
    public static partial Regex Firmware();
}