Elasticsearch

Article.

Elasticsearch is a distributed search and analytics engine built on Lucene. Interviews reach it three ways: full-text search for a product, log and metrics analytics, and lately hybrid and vector retrieval for RAG. The junior answer describes the query DSL. The senior answer explains that it is a derived, eventually consistent index made of shards, and every operational surprise follows from that sentence.

Verified 2026-10

Verified 2026-10-15 against Elasticsearch 9.5.5 (Lucene 10.5.1) running locally; the outputs below come from that run. Licensing and the OpenSearch fork are in OpenSearch and the fork.

The model: index, shards, segments

Three levels

index "orders"            what you query
 ├─ shard 0  primary + replica    a full Lucene index each
 │   ├─ segment  (immutable)
 │   └─ segment  (immutable)
 ├─ shard 1  primary + replica
 └─ shard 2  primary + replica

• An index is a logical name. Applications should only ever see an alias over it (06).
• A shard is a complete Lucene index holding a slice of the documents. A document goes to shard hash(_routing) % primaries, where _routing is the id unless you pass one. That formula is why the primary count is fixed at creation: change it and every document is on the wrong shard. The split and shrink APIs exist; in practice you reindex.
• A segment is an immutable inverted index inside a shard (01). Updates write a new copy and mark the old one deleted; merges reclaim the space later.

The defaults, read from a running 9.5 node

index.number_of_shards     1
index.number_of_replicas   1
index.refresh_interval     1s
index.search.idle.after    30s
index.translog.durability  REQUEST
index.max_result_window    10000
index.gc_deletes           60s
cluster.max_shards_per_node  1000

One primary and one replica means a single-node cluster is yellow from the first index: the replica has nowhere to go. Yellow is “no redundancy”, red is “a primary is missing, some data is unsearchable”.

The write path

What happens on a write

1. The client sends the document to any node, the coordinating node, which routes it to the primary shard.
2. The primary writes it to an in-memory buffer and appends it to the translog, fsynced before the request returns (durability: REQUEST).
3. The primary forwards it to the in-sync replicas, then acknowledges.
4. Refresh, every second by default, turns the buffer into a new searchable segment. Until then the document is durable but invisible to search.
5. Flush commits segments to disk and trims the translog. Merge combines small segments in the background.

Near-real-time, measured

An index with refresh_interval: 30s, to make the gap visible:

es.index(index="orders", id="o1", document={"status": "paid"})
es.get(index="orders", id="o1")["found"]                   # True
es.count(index="orders", query={"term": {"status": "paid"}})  # 0

es.index(index="orders", id="o2", document={"status": "paid"},
         refresh="wait_for")                               # blocked 29.8 s
es.count(index="orders", query={"term": {"status": "paid"}})  # 2

A get by id is realtime: if the document changed since the last refresh, the get refreshes that shard first. A search sees only refreshed segments. wait_for blocks until the next scheduled refresh, which at 30 s outlived the client’s default 10 s timeout on the first run. refresh=true forces a refresh of the shards the write touched, and on a hot path it produces many tiny segments.

Two consequences: “write then immediately search” is a race in tests and in product flows, and a shard nobody searched for 30 s goes search-idle and skips scheduled refreshes, so the first search after a quiet spell waits for one.

The read path: query then fetch

Two phases

• Query phase. The coordinating node sends the query to one copy of every shard. Each scores its documents locally and returns the ids and scores of its top from + size.
• Fetch phase. The coordinator merges those lists, keeps the global top size, and fetches only those documents from their shards.

Deep pagination follows directly: from=9990, size=20 asks every shard for 10,010 hits. The default max_result_window of 10,000 rejects it:

400 Result window is too large, from + size must be less
than or equal to: [10000]

Use search_after, with a point in time for a consistent export (06).

Scores are per shard

BM25 uses term statistics: how rare a word is across the documents. Each shard computes them from its own documents. The same 41 documents, one of them “rust engineer”, searched for rust:

one shard    query_then_fetch      score 4.169
one shard    dfs_query_then_fetch  score 4.169
five shards  query_then_fetch      score 2.471
five shards  dfs_query_then_fetch  score 4.169

The five-shard index scored the same document differently because its shard held 10 documents, not 41. On large indexes the statistics even out; on small indexes, many shards, or routed tenants they do not, and relevance looks random. dfs_query_then_fetch collects global statistics first at the cost of an extra round trip. The better fix is fewer, larger shards.

Concurrency and keeping it in sync

Optimistic concurrency

Every write gets a _seq_no and _primary_term. Read them, and make the write conditional:

doc = es.get(index="orders", id="o1")
seq, term = doc["_seq_no"], doc["_primary_term"]
es.index(index="orders", id="o1", document={"status": "shipped"},
         if_seq_no=seq, if_primary_term=term)          # writer A: ok
es.index(index="orders", id="o1", document={"status": "refunded"},
         if_seq_no=seq, if_primary_term=term)          # writer B: 409

Writer B gets 409 version_conflict_engine_exception and must re-read. There are no multi-document transactions.

The source of truth lives elsewhere

Elasticsearch is a projection of a database. The pattern that holds up is outbox or CDC to an indexer: the database commits the row and an event in one transaction, and the indexer applies events to Elasticsearch (The transactional outbox). Events arrive late and out of order, so the indexer writes with the source’s version, and Elasticsearch rejects anything older:

def apply(event):
    if event.op == "delete":
        es.delete(index="products", id=event.id,
                  version=event.version, version_type="external")
    else:
        es.index(index="products", id=event.id, document=event.doc,
                 version=event.version, version_type="external")
# version 5 stored, then a late version 3 arrives -> 409, version 5 kept

version must increase per row: a version column, or the CDC log position. A 409 here is success, not an error: the newer state is already in place.

Deletes are forgotten after 60 seconds

The version of a deleted document is kept only for index.gc_deletes. A late event after that recreates the document. Run with gc_deletes set to 2 s:

index v5, delete at v6, wait 3 s, late event at v4
after a late v4 event: {'name': 'v4, late'}

The deleted product came back. Replays, backfills and lagging consumers make this real. The fix is a soft delete: index {"deleted": true} at the new version, filter it out of queries, and purge old tombstones on a schedule with delete_by_query.

Sizing a cluster

Shards

Elastic’s current guidance: shards of 10-50 GB, under 200 million documents each, and at most 1,000 non-frozen shards per node (cluster.max_shards_per_node). Every shard costs heap, file handles and cluster-state entries, so 5,000 shards of 200 MB is a slower, less stable cluster than 50 shards of 20 GB.

For time-series data, write to a data stream with rollover by size and age, and let ILM move old indexes to warm, cold and frozen tiers (06). Shard count then follows from data volume instead of a guess at creation time.

Nodes

• Three master-eligible nodes, so elections have a quorum. Since 7.0 the voting configuration is managed automatically; the old minimum_master_nodes split-brain setting is gone.
• Heap at no more than half the RAM, and below the compressed-pointer limit of roughly 31 GB. The other half is the filesystem cache, which is where Lucene’s segments are actually read from.
• Data tiers (hot, warm, cold, frozen) put recent data on fast disks and old data on cheap storage or snapshots.

When to choose it, and when not

Where it breaks in production

• Disk watermarks. At 85% no new shards are placed on the node, at 90% shards move away, and at 95% (flood stage) indexes on it become read-only. Writes fail cluster-wide for those indexes until disk is freed; the block lifts automatically below the high watermark.
• Mapping explosion. Dynamic mapping of user-supplied keys creates thousands of fields and bloats cluster state (03).
• Oversharding. One index per tenant per day, five shards each, and the cluster spends its heap on bookkeeping.
• refresh=true on every request. Tiny segments, constant merging, falling indexing throughput.
• Deep pagination and huge terms aggregations. Both make every shard return far more than the client asked for (04).
• Treating it as the database. No transactions, a mapping that cannot change in place, and deletes forgotten after 60 s. Keep the ability to rebuild the index from the source with a reindex and an alias swap.

The trade not taken

The common over-reach is adding Elasticsearch for search in an application whose data already lives in Postgres. That buys a second system to run, a sync pipeline with the ordering problems above, and eventual consistency between the two.

Postgres full-text search (tsvector, GIN, ts_rank) handles millions of rows and simple ranking inside the transaction that wrote the row. Move to Elasticsearch when the requirements say so: tunable relevance, faceted navigation, many languages, typo tolerance at scale, or log volumes Postgres was never meant to hold.

The folder

Related

• NoSQL Decision Guide — Which Store, and When - where a search engine fits among the NoSQL stores
• Hybrid search and reranking - BM25 plus vectors for RAG
• RAG on large data volumes - sharding and fan-out for retrieval