Machine Vision for Automation

Giving machines eyes. Learn vision-system components (cameras, sensors, lenses, lighting), resolution and field of view, the pinhole camera model and calibration (intrinsics, extrinsics, distortion), image processing (thresholding, filtering, edges, morphology, blobs, template matching), measurement and inspection, 2-D/3-D vision (stereo, structured light, time-of-flight), deep-learning inspection, and vision-guided robotics — with Python for FOV/resolution, blob centroids and pixel-to-world mapping.

Machine vision inspects products, reads codes, measures dimensions and guides robots — tirelessly and fast. A camera, the right lens and (most importantly) the right lighting turn a scene into images; algorithms turn images into decisions: pass/fail, position, identity.

Vision is where optics, electronics, software and mechanics meet — a core mechatronic subsystem in modern factories, cars (ADAS) and autonomous robots.

An analogy

A vision system is like a photographer plus a judge: the photographer (camera, lens, lighting) must capture the feature clearly; the judge (software) can only decide well from a good photo — bad lighting cannot be fixed by clever code.

System components

Every vision system needs:

Classic image processing

Pre-processing (filters to reduce noise, contrast normalisation), segmentation (thresholding, Otsu, adaptive), morphology (erode/dilate/open/close), edge detection (Sobel, Canny), blob analysis (area, centroid, orientation), template/pattern matching (normalised cross-correlation, geometric matching), Hough transforms (lines, circles), OCR and barcode/2-D code reading. OpenCV and HALCON are common libraries.

Calibration and measurement

Camera calibration with a checkerboard estimates intrinsics (focal length, principal point) and lens distortion (radial, tangential); hand–eye calibration relates camera and robot frames. Sub-pixel edge detection enables measurement accuracy of 1/10 pixel; telecentric lenses remove perspective error for gauging. Gauge R&R validates measurement capability.

3-D and deep learning

3-D sensing: stereo (depth from disparity), structured light (projected patterns), laser triangulation (profile scanners), time-of-flight cameras; point clouds feed bin picking and dimensioning. Deep learning (CNNs for classification, YOLO-style detection, segmentation, anomaly detection trained on good parts only) handles variable, cosmetic defects where rules fail — with needs for labelled data, validation and explainability.

Formulas

Worked example: Required resolution

Given: Detect 0.1 mm scratches over a 50 mm FOV, 3 px per feature

Find: Pixels across.

Answer: 1500 px → 2 MP camera (1600 px) or better

Worked example: Stereo depth

Given: f = 800 px, baseline 0.1 m, disparity 40 px

Find: Z.

Answer: 2.0 m

Worked example: Motion blur

Given: Conveyor 0.5 m/s, 86 µm/px, max 1 px blur

Find: Exposure time.

Answer: ≤ 172 µs (use strobed lighting)

FOV, resolution, blob centroid and pixel-to-world

import math
# Camera selection: field of view, spatial resolution, pinhole projection
sensor_w_px, sensor_w_mm, f_mm = 2448, 8.45, 16.0
wd = 400.0                                     # working distance mm
fov = sensor_w_mm * wd / f_mm
print(f"field of view {fov:.0f} mm, {fov/sensor_w_px*1000:.1f} um per pixel")
# Smallest defect detectable needs ~3 px across
print(f"smallest reliable feature ~ {3*fov/sensor_w_px:.2f} mm")
# Threshold + centroid of a tiny binary image (a blob)
img = ["..........", "...###....", "..#####...", "..#####...", "...###....", ".........."]
pts = [(x, y) for y, row in enumerate(img) for x, c in enumerate(row) if c == "#"]
cx = sum(p[0] for p in pts) / len(pts); cy = sum(p[1] for p in pts) / len(pts)
print(f"blob area {len(pts)} px, centroid ({cx:.2f}, {cy:.2f})")
# Pixel -> world with a pinhole model (camera looking straight down)
u0, v0, fx = 1224, 1024, f_mm / (sensor_w_mm / sensor_w_px)
u, v = 1500, 1100
X = (u - u0) * wd / fx; Y = (v - v0) * wd / fx
print(f"pixel ({u},{v}) -> ({X:.1f}, {Y:.1f}) mm from optical axis")
# field of view 211 mm, 86.3 um per pixel
# smallest reliable feature ~ 0.26 mm
# blob area 16 px, centroid (4.00, 2.50)
# pixel (1500,1100) -> (23.8, 6.6) mm from optical axis

A 5 MP camera with a 16 mm lens at 400 mm sees a 211 mm field at 86 µm/pixel, so features below ~0.26 mm are unreliable — choose a longer lens or higher resolution for finer inspection. The centroid and pinhole back-projection are the core of vision-guided pick-and-place (after proper calibration including lens distortion).

Common mistakes

Applications

Key takeaways

Going further