AI & ML / Computer vision / 02_opencv_and_preprocessing.md

OpenCV and classical preprocessing

Updated 4 interview angles 5 min read source
On this page7
  1. The operations worth knowing cold
  2. The preprocessing order that matters
  3. Resizing is a decision, not a detail
  4. Augmentation, and what not to augment
  5. Where it breaks in production
  6. Related
  7. Interview angle

OpenCV and classical preprocessing

Deep learning did not replace classical CV; it sits on top of it. Every production pipeline still resizes, denoises, deskews and thresholds before a model sees anything, and for structured inputs like scanned drawings the classical operations often do more work than the network.

The interview signal here is knowing when a morphological operation is the right answer and a model is overkill.

The operations worth knowing cold

Operation What it does Typical use
Threshold grey to binary ink vs paper
Morphology grow or shrink regions close gaps, drop specks
Blur suppress high frequency denoise before edges
Canny edge map line and contour finding
Hough lines or circles from edges walls, rulings, gridlines
Warp resample under a matrix deskew, rectify
python 
import cv2

grey = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Adaptive beats global on anything unevenly lit or scanned.
binary = cv2.adaptiveThreshold(
    grey, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
    cv2.THRESH_BINARY_INV, blockSize=31, C=8,
)

# Close 1-2px breaks in lines so contours are continuous.
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
closed = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel, iterations=2)

Otsu vs adaptive is a common probe. Otsu picks one global threshold by maximising between-class variance — correct for a cleanly scanned page with even lighting. Adaptive computes a threshold per neighbourhood, which is what you need for a photographed document, a shadow across the page, or a drawing with heavy and light linework in different regions.

Morphology in one sentence each: erosion shrinks white regions, dilation grows them, opening (erode then dilate) removes small specks, closing (dilate then erode) fills small holes and joins near-touching lines. Closing before contour finding is the standard fix for a dashed or broken line.

The preprocessing order that matters

text 
load -> greyscale -> denoise -> deskew -> threshold -> morphology
     -> contours / model

Two ordering rules are worth defending:

Denoise before edges, never after. Canny amplifies high-frequency content, so noise becomes edges and no amount of downstream filtering separates them again. A light Gaussian blur first costs nothing.

Deskew before threshold on photographs, after on scans. Warping a greyscale image resamples smoothly; warping an already-binarised image introduces jagged, aliased edges that morphology then has to clean up.

Resizing is a decision, not a detail

python 
# Down: INTER_AREA. Up: INTER_CUBIC or INTER_LINEAR.
small = cv2.resize(img, (640, 640), interpolation=cv2.INTER_AREA)

# Preserve aspect ratio, pad the remainder - what detectors expect.
def letterbox(img, size=640):
    h, w = img.shape[:2]
    s = size / max(h, w)
    nh, nw = int(round(h * s)), int(round(w * s))
    out = cv2.resize(img, (nw, nh), interpolation=cv2.INTER_AREA)
    top, left = (size - nh) // 2, (size - nw) // 2
    return cv2.copyMakeBorder(out, top, size - nh - top,
                              left, size - nw - left,
                              cv2.BORDER_CONSTANT, value=(114, 114, 114))

INTER_LINEAR on a large downscale aliases: it samples a few pixels and ignores the rest, so thin lines disappear entirely. INTER_AREA averages over the source region, which is what you want for line drawings where a wall may be two pixels wide.

Squashing to a square instead of letterboxing distorts aspect ratio, and any geometry you recover afterwards is wrong by that factor. Keep the transform so you can invert it — see Geometry: homography, rectification, vanishing lines.

Augmentation, and what not to augment

Albumentations is the usual library because it transforms masks, boxes and keypoints alongside the image, which hand-rolled augmentation forgets.

The rule is that augmentation should model variation you actually expect. For photographs: flips, rotation, brightness, blur, JPEG artefacts. For scanned drawings: small rotations, scan noise, contrast, line-thickness jitter.

What not to do is the more interesting half. Do not vertically flip a floor plan or a document — text and symbols have a canonical orientation, and teaching the model that upside-down is normal costs accuracy on the real distribution. Do not hue-shift when colour carries meaning, as it does in layered CAD exports where colour encodes the layer.

Where it breaks in production

  • Silent BGR/RGB mismatch. No error, just worse numbers.
  • Hard-coded kernel sizes. A (3, 3) kernel tuned on 300 DPI scans does nothing at 72 DPI and destroys detail at 1200. Scale kernels with resolution.
  • Threshold tuned on one batch. A global threshold that worked on ten clean files fails on the first photographed one. Prefer adaptive plus a fallback.
  • EXIF rotation ignored. cv2.imread drops the EXIF orientation tag, so phone photos load sideways while a viewer shows them upright.
  • Preprocessing that differs between training and inference. The most common cause of “it worked in the notebook”. Share one function; never reimplement it in the serving path.

Interview angle 4

  • “Otsu or adaptive threshold?” - Otsu picks one global value and is right for an evenly lit scan. Adaptive works per neighbourhood and is what you need for photographs, shadows, or drawings with varying line weight. Follow-up: what if the page is half shadow? Adaptive, or normalise illumination first by dividing by a heavily blurred copy.
  • “How do you join a broken line before contour finding?” - morphological closing, dilate then erode, with a kernel sized to the gap. Say why the order matters: opening would delete the thin line instead.
  • “You downscale a 4000px drawing to 640 and the doors vanish. Why?” - bilinear interpolation samples rather than averages, so a two-pixel line falls between samples. Use INTER_AREA, or tile the image and run at native resolution instead of downscaling.
  • “Which augmentations would you not use here?” - vertical flips on documents or plans, because orientation is canonical, and hue shifts when colour encodes CAD layer.
Previous
The computer vision task taxonomy