Shortening embedding vectors cuts storage and search cost. Whether embedding dimensionality retrieval precision survives the cut depends on the model, the domain and the target size. The evidence is benchmark-specific, largely preprint, and in places secondhand. This reference separates what has been measured from what has been argued.
Definitions and what the numbers measure
An embedding is the fixed-length vector a model produces for one text. Its dimension, written d, is that length; 768 floats per text means d=768. Stored as float32, each dimension occupies 4 bytes. Cosine similarity scores two vectors by the cosine of the angle between them. Top-k retrieval returns the k highest-scoring documents. nDCG@10 is normalised discounted cumulative gain over the top ten results. A retention ratio is the score at reduced dimension divided by the score at full dimension.
The figures below are benchmark results under the conditions stated with each. They are not production measurements, and they are not directly comparable, because they use different metrics.
The theoretical ceiling: LIMIT and the critical-size claim
Weller et al. argue that the number of distinct top-k document subsets a single embedding vector can return is limited by the embedding dimension. According to the paper, the limit holds even when free embeddings are optimised directly on the test set. To test this, the authors built the LIMIT dataset, on which even state-of-the-art embedding models fail despite simple queries. The paper, by authors from Google DeepMind and JHU, is listed as an ICLR 2026 publication.
This article draws on the arXiv abstract and a mirrored excerpt. The full PDF has not been checked against these statements, so quote the paper only after reading it.
A mirrored excerpt reports a critical corpus size of about 1.7 million documents at d=768, derived from the paper's fitted extrapolation. Treat this as an estimate, not a finding, until the PDF confirms it.
In this article's reading, the result concerns what a single vector can represent under the paper's construction. It does not show that embedding retrieval fails on ordinary corpora, and the sources reviewed do not show the ceiling binding at typical RAG corpus sizes.
Measured retention when vectors are shortened
For a Granite multilingual embedding model, MTEB retrieval scores were 52.6 at 768 dimensions, 51.6 at 256 and 50.4 at 128 (arXiv 2605.13521, preprint). Across a sixfold reduction in dimension, the score fell by 2.2 points on that benchmark.
In a patent-domain benchmark (arXiv 2605.24297, preprint), all five models kept above 88% of their full-dimension nDCG@10 at 256 dimensions. At 64 dimensions, one model fell to 62.5%. The result is model-dependent, so a single safe size does not follow.
A separate ESG-domain preprint (arXiv 2609.15434) reports that PCA reduction did not degrade NDCG for nearly all models tested, down to 16 dimensions. That is one domain. The authors suggest the leading components carry the signal and low-variance dimensions add noise. This is their interpretation.
Matryoshka training versus post-hoc truncation
Matryoshka Representation Learning (arXiv 2205.13147) trains a model so that prefixes of its vector remain usable. The paper reports up to 14x smaller embeddings at the same ImageNet-1K accuracy, and up to 14x real-world speed-ups for large-scale retrieval. These are vision and retrieval results from that paper, not text-retrieval benchmarks. The method is described as a minimal change to existing pipelines with no added inference cost (Semantic Scholar entry for the paper).
One truncation comparison comes from a Hugging Face STS experiment, relayed through a GitHub pull request and a secondary blog. It reports that a Matryoshka model retained 98.37% of performance at 8.3% of embedding size, against 96.46% for a standard model under naive truncation. The original Hugging Face post was not located, so these figures are secondhand and rest on one experiment.
A blog argues that truncating a standard model degrades quality silently. That is an argument, not a measured benchmark.
Where shortening breaks down
Three conditions in the sources reviewed limit the rule that shorter is safe.
- Model dependence. In the patent benchmark, one model fell to 62.5% at 64 dimensions while others held above 88% at 256.
- Domain and model-family dependence. The ESG result covers one domain. The Granite result covers one model family.
- Exact-match tokens. A Medium post says dense embeddings struggle with SKUs, error codes and citation numbers. No primary benchmark was found, and this is a different failure mode from the dimension ceiling.
Quantization and hybrid lexical-plus-dense search are not covered by the sources found, so their interaction with truncation is unknown here.
Cost arithmetic and two-stage retrieval
Storage is linear in dimension. One guide gives 8 KB per 2048-dimension float32 vector and 8 GB per million documents. By the same arithmetic, 768 dimensions is 3,072 bytes per vector, or about 3 GB per million documents. A blog estimates that cosine similarity between 1536-dimension vectors is roughly 4x slower than between 384-dimension vectors, all else equal. That is an arithmetic estimate, not a measurement. Another blog describes returns as diminishing, without a measured curve.
Two-stage retrieval, a small-dimension candidate search followed by full-dimension rescoring, is recommended in several practice sources. Production stacks often add a cross-encoder reranker to the first pass. One aggregator reports 99% accuracy for two-pass search against 89.2% for single-pass 1536-dimension search. The original was not verified, so treat that figure as unconfirmed. No measured cost data for rescoring was found. Its cost depends on keeping full-dimension vectors and on the reranker chosen.
Measuring the tradeoff on your own data
Build a fixed query set with relevance labels from your own corpus, and record a full-dimension baseline using the same index and scoring. Evaluate truncation and PCA at each candidate size. If PCA is used, fit it on data that excludes the evaluation queries. Report the retention ratio with confidence intervals across queries, and treat differences smaller than the sampling noise as unresolved. For that judgment, see statistical power when comparing models. Public benchmark numbers such as those above also carry a contamination caveat, covered in benchmark contamination detection and its limits.
What remains open
- Whether LIMIT-type failures occur at RAG scales. The 1.7 million figure is unconfirmed.
- Whether the Matryoshka advantage over truncation replicates under a primary protocol.
- Whether two-stage rescoring recovers full precision.
- How truncation interacts with quantization and hybrid search.
Several sources were read only from abstracts, search excerpts or mirrors, and most preprint dates are unconfirmed. Before shortening a production index, run the protocol above on your own labelled queries, and treat published retention figures as priors rather than guarantees.