Parallel Programming and Concurrent Collections

Parallelism helps CPU-bound work (checksums, signal processing, compression) on multi-core machines.

Concept

Data and Task Parallelism Parallel.For(0, n, i =>...) Loop iterations in parallel (data parallelism) Parallel.ForEach(items, item =>...) Parallel loop over a collection

Parallel.ForEachAsync(items, async (item, ct) =>...) Async bodies with a concurrency limit Parallel.Invoke(a, b, c)

Run several independent actions (task parallelism) PLINQ: items.AsParallel().Select(...)

Parallel LINQ; AsOrdered() keeps input order MaxDegreeOfParallelism, CancellationToken

Parallelism helps CPU-bound work (checksums, signal processing, compression) on multi-core machines. For I/O-bound work, use async (Module 31) instead of more threads. Exceptions from parallel loops are collected into an AggregateException; cancellation throws OperationCanceledException.

Concurrent Collections (System.Collections.Concurrent) Thread-safe map; AddOrUpdate, GetOrAdd, TryUpdate Lock-free FIFO; Enqueue, TryDequeue Lock-free LIFO; Push, TryPop

Unordered, optimized for the same thread adding and taking Wraps one of the above with blocking and bounding (producer/consumer)

Use them instead of locking ordinary collections when many threads share one collection; prefer ordinary collections plus a lock when you need multi-step atomic operations across several collections.

Why It Matters

Processing gigabytes of test logs, simulations, image data or calibration sweeps finishes in a fraction of the time when all cores are used — if done safely.

Syntax

Parallel.For(0, n, new ParallelOptions { MaxDegreeOfParallelism = 4 }, i => Work(i));
var results = data.AsParallel().AsOrdered().Select(Compute).ToArray();
dict.AddOrUpdate(key, 1, (_, old) => old + 1);

Example

using System.Collections.Concurrent;
using System.Diagnostics;

// CPU-bound work: checksum of many firmware blocks
byte[][] blocks = Enumerable.Range(0, 64).Select(i => Enumerable.Range(0, 200_000).Select(j => (byte)(i * j)).ToArray()).ToArray();
static uint Checksum(byte[] data)
{
    uint sum = 0;
    for (int r = 0; r < 20; r++)
        foreach (byte b in data) sum = (sum * 31) ^ b;
    return sum;
}

var sw = Stopwatch.StartNew();
uint[] sequential = blocks.Select(Checksum).ToArray();
long seqMs = sw.ElapsedMilliseconds;

sw.Restart();
uint[] parallelResults = new uint[blocks.Length];
Parallel.For(0, blocks.Length, i => parallelResults[i] = Checksum(blocks[i]));     // data parallelism
long parMs = sw.ElapsedMilliseconds;
Console.WriteLine($"Parallel.For results identical: {sequential.SequenceEqual(parallelResults)}; faster on {Environment.ProcessorCount} cores: {parMs < seqMs}");

// PLINQ
uint[] plinq = blocks.AsParallel().AsOrdered().Select(Checksum).ToArray();
Console.WriteLine($"PLINQ (AsOrdered) identical: {sequential.SequenceEqual(plinq)}");

// Parallel.ForEach with a thread-safe collection and MaxDegreeOfParallelism
var perEcu = new ConcurrentDictionary<string, int>()
…

Expected Result

(Whether the parallel version is faster depends on your core count; this machine had 8.)

Parallel.For results identical: True; faster on 8 cores: True
PLINQ (AsOrdered) identical: True
ConcurrentDictionary counts: ABS=3333, BMS=3333, VCU=3334
Parallel.Invoke: flash VCU, flash BMS, flash ABS
Parallel.For cancelled -> OperationCanceledException
AggregateException with 2 errors: block 1 corrupt, block 3 corrupt
ConcurrentQueue 1000, ConcurrentStack 1000, ConcurrentBag 1000, TryDequeue ok: True
BlockingCollection consumed sum = 210

Common Mistakes

Parallelizing tiny loop bodies — overhead exceeds the gain.

Writing to shared non-thread-safe collections from parallel loops.

Using Parallel.For for I/O (use Parallel.ForEachAsync or Task.WhenAll with throttling).

Assuming PLINQ preserves order (use AsOrdered()).

Calling GetOrAdd with a factory that has side effects (it may run more than once).

Best Practices

Measure first; parallelize coarse-grained CPU work.

Write results into per-index slots or use thread-local accumulation and combine at the end.

Set MaxDegreeOfParallelism when calling external resources.

Handle AggregateException and support cancellation.

Interview Relevance

High: "Parallel.For vs Task.WhenAll", "CPU-bound vs I/O-bound", "PLINQ ordering", "ConcurrentDictionary.GetOrAdd pitfalls", "BlockingCollection producer/consumer".

Practice Questions

Compute SHA-256 hashes of 100 files in parallel with a degree of parallelism of 4.

Count word frequencies in a large text with ConcurrentDictionary and compare with a sequential version.

Implement a producer/consumer pipeline with BlockingCollection and two consumers.

Use Parallel.ForEachAsync to download (simulate) 20 files with max 5 at a time.

Quick Quiz

Q1. (Multiple choice) Which is best for CPU-bound work over a large array? A) Task.Delay B) Parallel.For C) async void D) Thread.Sleep Answer: B.

Q2. (True/False) PLINQ always preserves the input order.

Answer: False. Use AsOrdered() (the example uses it and the results matched).

Q3. (Predict the output) What exception type reports that two iterations of Parallel.For threw?

Answer: AggregateException with 2 inner exceptions (verified).

Q4. (Predict the output) The sum consumed from the BlockingCollection producing 1..20.

Answer: 210 (verified).

Coding Challenge

Use Parallel.ForEachAsync with MaxDegreeOfParallelism = 3 to "flash" 9 ECUs (each a 30 ms delay) and record the maximum number of simultaneous flashes.

int active = 0, max = 0;
object gate = new();
string[] ecus = Enumerable.Range(1, 9).Select(i => $"ECU{i}").ToArray();
await Parallel.ForEachAsync(ecus, new ParallelOptions { MaxDegreeOfParallelism = 3 }, async (ecu, ct) =>
{
    lock (gate) { active++; max = Math.Max(max, active); }
    await Task.Delay(30, ct);
    lock (gate) active--;
});
Console.WriteLine($"flashed {ecus.Length} ECUs, max simultaneous = {max}");