How Elasticsearch and OpenSearch actually work: Lucene, shards, and the write path
A search cluster is a distributed layer over Lucene, and most performance problems trace back to how documents are written, refreshed, merged, and sharded. Here's the mental model that makes it tunable.
Elasticsearch and OpenSearch (a fork of the same lineage) are distributed search engines wrapped around Apache Lucene. Almost every tuning question — why writes are slow, why the heap is full, why a query is expensive — becomes obvious once you understand what Lucene does underneath and how the cluster layers sharding on top. Here's that model.
Lucene: the inverted index and immutable segments
At the core is an inverted index: for every term, a list of the documents that contain it — which is what makes full-text search fast. Lucene stores this in segments, and the crucial property is that segments are immutable. You never edit a segment; you write new ones and delete old ones. That immutability is the source of both Elasticsearch's speed and its quirks.
The write path: buffer, translog, refresh, merge
- A new document goes into an in-memory buffer and is appended to the translog, which gives durability — if the node crashes before the buffer is flushed, the translog replays it.
- A refresh (by default every 1 second, controlled by refresh_interval) turns the buffer into a new, searchable segment. This is why Elasticsearch is 'near real-time': documents aren't searchable the instant you index them, only after a refresh.
- Because refreshes create many small segments, Lucene merges them in the background into fewer, larger ones. Merging is I/O-heavy; force-merging down to one segment is powerful for read-only indices and dangerous on active ones.
Two levers fall straight out of this: raise refresh_interval (or disable it) during a big bulk load to avoid churning tiny segments, and size your bulk API requests to a few MB — too small wastes round-trips, too large pressures the heap.
Shards, replicas, and routing
An index is split into shards, each a self-contained Lucene index, and shards are distributed across nodes — that's the horizontal scaling. Primaries take writes; replicas serve reads and provide failover. The common mistake is too many small shards (each carries fixed overhead) or too few huge ones (hard to move and recover); aim for shards in the tens-of-GB range and size the count to your data, not by reflex. Custom routing sends related documents to the same shard, which speeds queries that filter on the routing key.
Query vs filter context, and caching
How you phrase a query changes both its cost and its cacheability. Query context computes a relevance score (expensive); filter context answers a yes/no (cheaper) and its results are cached in the node's filter cache. Put anything that doesn't need scoring — date ranges, term matches, tenant filters — in filter context. The request cache stores whole aggregation results for identical queries, and the profile API tells you exactly where a slow query spends its time.
Analyzers: matching how text is actually written
Text is run through an analyzer at both index and query time — a tokenizer plus filters for lowercasing, stemming, and synonyms. For technical or legal vocabulary this is where matches are won or lost: the right synonym set and stemming decide whether a query for a term finds the document that phrases it differently. Phrase and proximity queries then match terms in order or near each other — essential for matching claim language, where word order carries meaning.
Reindexing without downtime, and cluster ops
Because mappings are mostly immutable once set, changing them means reindexing into a new index and switching an alias over atomically — which is how you get zero-downtime schema changes. Ingest pipelines transform documents on the way in. On the operations side, the recurring themes are the JVM heap (keep it at ~50% of RAM and under ~32GB so pointers stay compressed), circuit breakers that reject queries before they OOM the node, hot-warm-cold tiering to move ageing data to cheaper hardware, and snapshots for backup and restore.
Elasticsearch isn't magic — it's Lucene's immutable segments, distributed across shards, refreshed and merged on a schedule. Tune what you can see in that path and the mysteries mostly disappear.