Geometry: homography, rectification, vanishing lines

The task: someone photographs a printed floor plan at an angle, and you need a top-down image where measurements are proportional to reality. That is a homography, and the geometry behind it is one of the few CV areas where the classical answer is still strictly better than a learned one.

The transform hierarchy

Each row adds freedom and preserves less.

Affine keeps parallel lines parallel. It handles rotation, scale, shear and translation — enough to deskew a flatbed scan, where the page was flat and only rotated. Three point correspondences determine it.

Homography does not preserve parallelism, which is exactly what you need for a photograph: parallel walls converge toward a vanishing point. Four point correspondences determine it, no three of them collinear.

The distinction is the most common probe in this area. Use affine for scans, homography for photographs. Using affine on a perspective image leaves a residual distortion that quietly corrupts every measurement you take.

The four-point rectification

If you can find the corners of a known rectangle — the page, the drawing border, a rectangular room — you can rectify.

import cv2
import numpy as np

def rectify(img, corners, out_w, out_h):
    """corners: 4 points (x, y) in order tl, tr, br, bl."""
    src = np.asarray(corners, dtype=np.float32)
    dst = np.float32([[0, 0], [out_w, 0], [out_w, out_h], [0, out_h]])
    H = cv2.getPerspectiveTransform(src, dst)
    warped = cv2.warpPerspective(img, H, (out_w, out_h))
    return warped, H

Keep H. Every detection you make in the warped image needs mapping back to source coordinates, or vice versa, and recomputing it later from remembered corners is where the off-by-one errors live.

# Map points between the two frames. Note the reshape - cv2 wants (N, 1, 2).
pts = np.array([[x, y]], dtype=np.float32).reshape(-1, 1, 2)
in_warped = cv2.perspectiveTransform(pts, H)
back = cv2.perspectiveTransform(in_warped, np.linalg.inv(H))

Choosing out_w and out_h decides your resolution budget. Setting them from the longest edge of the source quadrilateral avoids throwing away detail; a fixed square output squashes a landscape drawing.

When you cannot find four corners

This is the real case. The page edge is cropped, the border is occluded, the drawing is a photograph of a screen.

Vanishing points. In a perspective image, sets of parallel lines in the world meet at a vanishing point in the image. Two vanishing points from two perpendicular directions are enough to recover the rectifying transform.

        ·  vanishing point (horizontal lines meet here)
   ────────
  ─────────
 ──────────

The pipeline: detect edges, extract line segments with a Hough transform, cluster them by orientation, and intersect each cluster to estimate its vanishing point. Architectural drawings and building interiors are the ideal case for this because they are full of long, genuinely parallel lines.

Robust estimation is not optional. Line detection produces outliers, and a least-squares fit through outliers gives a confidently wrong answer. RANSAC repeatedly fits to a minimal random sample and keeps the fit with the most inliers.

H, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,
                             ransacReprojThreshold=3.0)
inlier_ratio = mask.mean()      # below ~0.5, distrust the result

inlier_ratio is your confidence signal, and it is the natural trigger for fallback logic — see CV pipelines and designing for imperfect input.

Scale: from pixels to metres

A homography gives you a top-down image. It does not give you units. You need one known real-world length to fix the scale, and drawings usually supply one:

• A scale bar, if present — detect it, measure it in pixels, divide.
• A stated ratio such as 1:100, combined with the page size and DPI.
• A dimension annotation — OCR a labelled measurement and use it.
• A known object, such as a standard door width, as a last resort.

Say out loud that a single scalar is only valid after rectification. Applying a pixels-per-metre factor to an unrectified photograph is wrong by a factor that varies across the image, and that error is invisible until someone checks a quantity against reality.

Where it breaks in production

• Lens distortion first. Homography assumes a pinhole camera. A wide-angle phone lens bends straight lines, so straighten with cv2.undistort before estimating, or the fit absorbs the barrel distortion as skew.
• Near-degenerate correspondences. Four points that are nearly collinear, or clustered in one corner, give a numerically unstable H. Check the spread, not just the count.
• Warping a binarised image. Resampling a binary mask aliases the edges. Warp the greyscale, threshold after.
• Interpolation on masks. Use INTER_NEAREST for label masks, or warping invents class ids that never existed between 3 and 5.
• Assuming the plan is planar. It is, but the paper may not be. A curled page is not a homography, and no four points fix it; that needs a mesh warp.

Related

• OpenCV and classical preprocessing - edges and Hough lines feed this.
• Monocular depth estimation - when the scene is not planar and one transform cannot fix it.
• Contours, polygons and vector geometry - measuring in rectified coordinates.
• Floor plan recognition and takeoff - the whole pipeline this sits in.
• CV pipelines and designing for imperfect input - using the inlier ratio as a confidence gate.