Open-weight VLMs: the landscape

The question behind “which VLMs have you used” is whether you can name the families, say what each was built for, and admit where each one breaks. Model names date fast; the shapes and the trade-offs do not. Dates below are release months, so you can place a model relative to whatever came out last week.

The families

Qwen and InternVL

Qwen2.5-VL (Alibaba, January 2025; 3B, 7B, 32B, 72B). Native dynamic resolution, window attention in the encoder, M-RoPE, grounding with coordinates, document parsing to HTML-like output, hour-long video. The 7B is the workhorse of open document pipelines: strong OCR, tables and forms at a size that fits one mid-range GPU. Licences differ by size — 7B and 32B are Apache-2.0, the 3B and 72B carry Qwen’s own terms — so check the card.

Qwen3-VL (Alibaba, September–October 2025; dense 2B, 4B, 8B, 32B and MoE 30B-A3B, 235B-A22B; Instruct and Thinking variants; Apache-2.0). Interleaved M-RoPE, DeepStack (multi-level encoder features fed into the LLM), 256K native context, OCR across 32 languages, stronger grounding and GUI-agent use. Alibaba’s claim that the 8B matches the previous 72B on document tasks is roughly right on benchmarks; verify on your own scans.

InternVL (OpenGVLab, InternVL3 April 2025 at 1B–78B; InternVL3.5 August 2025 up to 241B-A28B). InternViT encoder, 448² tiles plus a thumbnail, 256 tokens per tile; InternVL3 added native multimodal pretraining and variable visual positions; 3.5 added a resolution router and a decoupled vision/language deployment mode. Consistently strong on general benchmarks (MMMU) and competitive on OCR. MIT weights over Qwen or InternLM backbones.

Llama, Pixtral and Gemma

Llama 3.2 Vision (Meta, September 2024; 11B, 90B). Cross-attention adapter over a frozen Llama 3.1 text model, up to four 560² tiles, built for one image per turn. Keeps Llama’s text quality and instruction following, grounds poorly, and reads dense documents worse than Qwen at the same size.

The licence matters: the multimodal Llama 3.2 models are explicitly not licensed for companies domiciled in the EU, and Llama 4 (April 2025, natively multimodal MoE) shipped with a similar clause. Read it before a European client sees a demo.

Pixtral 12B (Mistral, September 2024; Apache-2.0). Mistral Nemo plus a 400M encoder trained from scratch, native variable resolution with explicit row-break tokens, 128K context, interleaved multi-image. Good at charts and instruction following; OCR a step behind Qwen.

Pixtral Large (124B) is research-licensed; Mistral Small 3.1 / 3.2 (24B, 2025, Apache-2.0) is the Mistral you can ship commercially and the better document model of the three.

Gemma 3 (Google, March 2025; 4B, 12B, 27B). SigLIP encoder at a fixed 896² with a pan-and-scan crop for odd aspect ratios, 128K context, strong multilingual text. Fixed resolution is the limitation for scans. Gemma’s licence is permissive with a use policy attached.

The small and the specialised

Then the 2025 wave of document-specialist OCR VLMs — olmOCR (AI2, a Qwen-VL fine-tune), Nanonets-OCR-s, dots.ocr, PaddleOCR-VL, DeepSeek-OCR, MinerU 2.5, SmolDocling and Granite-Docling. They are 0.3B–3B models that turn a page into Markdown, HTML or a document markup with reading order, tables and formulas, at a fraction of a general VLM’s cost.

They answer no questions; they are the parsing stage in front of a text model, and for pure “PDF to text” they are now the default over Tesseract-era pipelines.

Reading the benchmarks

Two rules. DocVQA is saturated — the top open models score in the mid-90s and the gaps are noise, so it no longer separates candidates for a document workload. And every public benchmark is in the training data of something; the number that matters is field-level accuracy on 200 of your own documents, which is a day’s work to build (Evaluating document extraction).

Choosing one

# The decision, as code you would actually write in a design doc.
def pick(workload):
    if workload.needs_boxes or workload.counts_objects:
        return "detector first (YOLO / Florence-2), VLM for semantics"
    if workload.kind == "pdf_to_text":
        return "document OCR VLM (olmOCR / dots.ocr / PaddleOCR-VL)"
    if workload.eu_customer and workload.model.startswith("llama"):
        return "not this one - licence"
    if workload.pages_per_day > 100_000:
        return "Qwen3-VL-8B or Qwen2.5-VL-7B, AWQ/FP8, batched"
    if workload.needs_reasoning_over_page:
        return "Qwen3-VL-32B / InternVL3.5-38B, or the MoE if you have HBM"
    return "Qwen3-VL-8B; measure before going bigger"

Licences change between sizes and versions of the same family, so the check is a script run in CI against the pinned revision, not a memory:

from huggingface_hub import model_info

for repo in ["Qwen/Qwen3-VL-8B-Instruct", "OpenGVLab/InternVL3_5-8B",
             "meta-llama/Llama-3.2-11B-Vision-Instruct",
             "mistralai/Mistral-Small-3.2-24B-Instruct-2506"]:
    card = model_info(repo).card_data or {}
    print(f"{repo:50} {card.get('license')}")

The honest summary for 2026: Qwen is the default for documents, InternVL is the credible second with a stronger research cadence, Mistral Small is the European-licence-safe pick, Llama Vision is the one you choose for its text lineage rather than its eyes, and the document-OCR specialists replace the general model wherever the task is “read this page” rather than “answer about this page”.

Related

• How a VLM sees an image - the architectures named above.
• VLM failure modes in production - what “strengths and failure modes” means in practice.
• Layout detection and OCR engines - the non-VLM parsing stack the specialists compete with.
• Which model for which task - the same choice for text models.