Linear Algebra for Engineers

The condition number measures how much errors in the data are amplified in the solution; cond ≈ 1 is ideal, cond ≈ 1e12 means about 12 of the ~16 significant digits may be lost.

Concept

Solve a square system Least squares (more equations than unknowns) x = A \ b (QR-based) or pinv(A) * b LU, QR, Cholesky factorizations

[L, U, P] = lu(A), [Q, R] = qr(A), chol(A) eig(A); [V, D] = eig(A) svd(A); [U, S, V] = svd(A) cond(A), rcond(A) Rank, null space, range rank, null, orth

sparse, speye, spdiags (large FEM/circuit problems) The condition number measures how much errors in the data are amplified in the solution; cond ≈ 1 is ideal, cond ≈ 1e12 means about 12 of the ~16 significant digits may be lost. The Hilbert matrix is the classic ill-conditioned example.

Engineering interpretations: eigenvalues of a stiffness/mass problem are natural frequencies; eigenvalues of a state matrix determine stability; the SVD gives principal directions (PCA) and the rank of noisy data.

Why It Matters

Mesh/nodal circuit analysis, truss and FEM structural analysis, state-space control and data fitting all end in linear systems — solving them reliably is core engineering computing.

Syntax

I = R \ V;                       % circuit currents
coef = [x ones(size(x))] \ y;    % least-squares line
[V, D] = eig(K, M);              % generalized eigenproblem (vibration)

Example

22.1 Mesh analysis, factorizations, conditioning, least squares, SVD and eigenvalues

% Circuit: three loop currents (mesh analysis) -> R * I = V
R = [ 15  -5   0;
      -5  20 -10;
       0 -10  25];
V = [10; 0; 5];
I = R \ V;
fprintf('mesh currents: %s A\n', mat2str(I', 5))
fprintf('residual norm(R*I - V) = %.2e\n', norm(R * I - V))

[L, U, Pm] = lu(R);
fprintf('LU: norm(P*R - L*U) = %.2e\n', norm(Pm * R - L * U))
[Q, Rq] = qr(R);
fprintf('QR: Q orthogonal: %d\n', norm(Q' * Q - eye(3)) < 1e-12)

fprintf('cond(R) = %.3f (well conditioned), cond(hilb(8)) = %.3e (ill conditioned)\n', cond(R), cond(hilb(8)))

% overdetermined system: least squares with backslash
x = (0:4)';
y = [0.9; 3.1; 5.0; 7.2; 8.9];
A = [x ones(size(x))];
coef = A \ y;
fprintf('least-squares line: slope %.4f, intercept %.4f\n', coef)

[U2, S, V2] = svd([3 2 2; 2 3 -2]);
fprintf('singular values: %s\n', mat2str(diag(S)', 5))
fprintf('null space dimension of [1 2; 2 4]: %d\n', size(null([1 2; 2 4]), 2))

K = [2 -1 0; -1 2 -1; 0 -1 2];              % stiffness-like symmetric matrix
e = eig(K);
fprintf('eigenvalues of K: %s (all positive -> positive definite: %d)\n', mat2str(e', 5), all(e > 0))

Expected Result

The residual of the circuit solution is at machine-precision level. The resistance matrix is well conditioned, while hilb(8) has a condition number around 10¹⁰ — solutions with it lose about 10 digits. The overdetermined line fit came straight from A \ y. All eigenvalues of the symmetric matrix K are positive, so K is positive definite (as a stiffness matrix of a stable structure should be).

mesh currents: [0.7907 0.37209 0.34884] A
residual norm(R*I - V) = 3.20e-15
LU: norm(P*R - L*U) = 0.00e+00
QR: Q orthogonal: 1
cond(R) = 3.595 (well conditioned), cond(hilb(8)) = 1.526e+10 (ill conditioned)
least-squares line: slope 2.0100, intercept 1.0000
singular values: [5 3]
null space dimension of [1 2; 2 4]: 1
eigenvalues of K: [0.58579 2 3.4142] (all positive -> positive definite: 1)

Common Mistakes

Using inv to solve systems.

Ignoring warnings like "Matrix is close to singular or badly scaled".

Assuming eig returns eigenvalues sorted (sort them if order matters).

Using dense matrices for huge sparse problems.

Best Practices

Check cond (or the residual) when results look wrong.

Scale equations with very different units before solving.

Use sparse storage and iterative solvers (pcg, gmres) for large systems.

Interview Relevance

High for engineering roles: "how do you solve an overdetermined system?", "what is the condition number?", "what do eigenvalues tell you about a system?", "what is SVD used for?".

Practice Questions

Solve the nodal equations of a 3-node resistor network.

Compute the natural frequencies of a 2-mass, 3-spring system with eig(K, M).

Show how solving hilb(10) * x = b loses accuracy.

Use SVD to find the best rank-1 approximation of a 3×3 matrix.

Quick Quiz

Q1. (Multiple choice) A large condition number means: A) The matrix is large B) Solutions are sensitive to small errors in the data C) The matrix is sparse D) The determinant is large Answer: B.

Q2. (Predict the output) Are all eigenvalues of [2 -1 0; -1 2 -1; 0 -1 2] positive?

Answer: Yes (verified).

Q3. (True/False) A \ b can solve an overdetermined system in the least-squares sense.

Answer: True (verified).

Q4. (Find the bug) x = inv(A) * b; with A = hilb(12) gives very inaccurate x.

Answer: The matrix is severely ill-conditioned; no method can recover all digits, but A \ b with a residual check (and reformulating/scaling the problem) is the right approach.

Coding Challenge

For the resistor network equations [3 -1 -1; -1 3 -1; -1 -1 3] * v = [10; 0; 0] (node voltages), solve for v, print the condition number and verify the residual.

G = [3 -1 -1; -1 3 -1; -1 -1 3];
I = [10; 0; 0];
v = G \ I;
fprintf('node voltages: %s V\n', mat2str(v', 5))
fprintf('cond(G) = %.3f, residual %.2e\n', cond(G), norm(G * v - I))