Synthetic data and image normalization

Vision projects are usually annotation-bound, not model-bound. Synthetic data and input normalization are the two levers that move a project when you cannot buy more labels — one manufactures labelled examples, the other shrinks the variation the model has to cope with.

Synthetic data, from cheapest to dearest

Cut-and-paste composition is underrated. Take real annotated objects, paste them onto real backgrounds at plausible positions and scales, and you get new images with exact labels for almost no cost. It works because detectors learn local appearance more than global scene coherence — the pasted result looks wrong to a human and trains fine.

The details that decide whether it helps: blend the edges (a hard cut teaches the model to find cut edges), vary scale and rotation, respect placement rules (a door belongs on a wall), and keep the ratio of synthetic to real modest.

Procedural rendering

Procedural rendering is the strong option for drawings specifically, because a floor plan is generated from structured data in the first place. You can synthesise a layout, render it through a drawing library with varied line weights, hatching, fonts and symbol sets, and emit pixel-perfect masks and polygons alongside it. There is no annotation step at all, and the ground truth is exact rather than human-approximate.

def render_plan(layout, style):
    """layout: rooms, walls, openings. Returns image plus exact masks."""
    img = blank(style.size, style.paper)
    masks = {k: zeros(style.size) for k in ("wall", "door", "window")}
    for wall in layout.walls:
        draw_line(img, wall, width=style.wall_px, colour=style.ink)
        draw_line(masks["wall"], wall, width=style.wall_px, colour=1)
    for op in layout.openings:
        draw_symbol(img, op, style.symbols[op.kind])
        draw_poly(masks[op.kind], op.footprint, colour=1)
    return apply_scan_effects(img, style), masks

apply_scan_effects is the part that matters — noise, slight rotation, JPEG artefacts, blur, uneven illumination, paper texture. Without it you train on impossibly clean input and the model fails on the first real scan.

The domain gap is the whole problem

A model trained purely on synthetic data usually underperforms on real data, sometimes badly. The mitigations, in order of how well they work:

• Fine-tune on real data. Pretrain on plentiful synthetic, then fine-tune on the small real set. This is the reliable answer and the one to give first.
• Domain randomisation. Vary the synthetic parameters so aggressively that reality looks like just another sample from the distribution. Randomise line weight, font, symbol style, paper tint, noise — more than looks reasonable.
• Mix the batches. Train on both at once with real oversampled, rather than a clean two-stage split.
• Validate on real only. A validation set containing synthetic data will tell you the domain gap does not exist.

That last point is the one that catches people. Synthetic data belongs in train, never in val or test. If it appears in validation, every metric you have measures the wrong distribution.

Image-to-image normalization

The complement: instead of widening the training distribution, narrow the input distribution. Map every incoming image toward one canonical appearance before the model sees it.

photo of printout ─┐
scanned PDF       ─┼─> normalize ─> canonical form ─> model
vector export     ─┘   (deskew, illumination,
                        binarise, line weight, DPI)

For drawings the canonical form is well defined — upright, flat, black ink on white paper, known DPI, uniform line weight. Classical operations get you most of the way, and they are deterministic, inspectable and free at inference. See OpenCV and classical preprocessing and Geometry: homography, rectification, vanishing lines.

Learned normalization — a small image-to-image network that maps photograph to clean scan — is the option when classical steps cannot close the gap. It buys quality and costs you a second model to train, monitor and explain, plus a new failure mode: a normalizer that hallucinates plausible linework that was never in the original. In a measurement product that is unacceptable in a way that a blurry input is not.

The trade to state: normalization concentrates the variation into one inspectable stage, which makes the downstream model simpler and its failures easier to attribute. Augmentation spreads robustness through the model, where it cannot be inspected. For a small team with a narrow input domain, normalization is usually the better investment.

Where it breaks in production

• Synthetic in the validation set. Every number becomes optimistic.
• Too-clean synthetics. No noise, no skew, perfect fonts. The model learns properties that no real input has.
• One synthetic generator, one style. The model overfits to your renderer’s conventions and fails on a different draughtsman’s.
• A normalizer that invents. A learned image-to-image model completing a broken line changes a measurement. Prefer classical steps where the product is a number.
• Normalization drift. The canonical form is defined by code that changes. Version it with the model, or old and new inputs are no longer comparable.

Related

• OpenCV and classical preprocessing - the classical normalization operations.
• CV pipelines and designing for imperfect input - where normalization sits, and testing against bad inputs.
• Evaluating vision models and pipelines - why val must be real.
• Floor plan recognition and takeoff - procedural generation applied.
• Distillation and synthetic data
  • synthetic data for text models.