Engineering blog of Vespa, the open source search and vector engine spun out of Yahoo, covering search, ranking and ML serving.
HN commenters praise its approach to hybrid vector and faceted search and link it for approximate nearest neighbor and multi-vector indexing.
Hey everyone, I also work at Weaviate.
Weaviate has implemented Hybrid Search because it helps with search performance in a few ways (Zero-Shot, Out-of-Domain, Continual Learning). Members of the Qdrant team are arguing against implementing Hybrid Search in Vector Databases with 3 main points that I believe are incorrect: 1. That there are not comparative benchmarks on Hybrid Search. 2. “Multi-Tool” systems are generally flawed by design in favor of specialization. 3. Cross-Encoder inference does not need additional processing modules - this one is also more so related to the particular design differences of Weaviate and Qdrant, where the Qdrant team is again arguing that you don’t need to implement the thing you implemented.
TLDR
1. Such benchmarks exist as in trengrj's initial response and we are working on them as well.
2. There are a few arguments why adding sparse search doesn't require too much extra specialization, how it is already used in filtered vector search to begin with, and why it makes sense to apply the rank fusion in the database where each scoring method happens.
3. Cross Encoder inference generally doesn't happen in the database itself thus it makes sense to use modules to process the additional ranking logic. There are several other examples of inferences in search pipelines that this kind of design enables.
More detail:
1. “You don’t publish comparative benchmarks”
Firstly, jtrengrj has responded with exactly this request. I think it’s actually…
https://twitter.com/eugeneyan/status/1678060204943097863
>When Deepmind needs semantic retrieval, they just use the largest index on the planet.
Fun fact: Query-doc similarity was done via simple TF-IDF instead of vectors. It performed better than vector retrieval when retrieve docs > 45 (they used 50).
https://blog.vespa.ai/improving-zero-shot-ranking-with-vespa/
>This case illustrates that in-domain effectiveness does not necessarily transfer to an out-of-domain zero-shot application of the model. Generally, as observed on the BEIR dense leaderboard, dense embeddings models trained on NQ labels underperform the BM25 baseline across almost all BEIR datasets.
Not OP but I am working on a search engine with vector ranking. Why do you say that vector search would help with narrow queries? In my experience, semantic search helps broaden the query to search for adjacent ideas without exact term marches.
Hybrid approaches that use vector search for broad matches and rerank using BM25 could be what you’re looking for. See https://blog.vespa.ai/efficient-open-domain-question-answering-on-vespa/
For use in retrieval/RAG, an emerging paradigm is to not parse the PDF at all.
By using a multi-modal foundation model, you convert visual representations ("screenshots") of the pdf directly into searchable vector representations.
Paper: Efficient Document Retrieval with Vision Language Models - https://arxiv.org/abs/2407.01449
Vespa.ai blog post https://blog.vespa.ai/retrieval-with-vision-language-models-colpali/ (my day job)
Yes, Vespa has a generic Tensor framework that allows you to index multiple vectors for a single field, see https://blog.vespa.ai/semantic-search-with-multi-vector-indexing/ for details.
field embeddings type tensor<float>(p
{}, x[384]) to represent a multi-vector field
{ "0": [0.1....], "1": [0.2,..] }
Thanks for sharing. How does this compare with DiskANN (https://zilliz.com/blog/diskann-a-disk-based-anns-solution-with-high-recall-and-high-qps-on-billion-scale-dataset) or HNSW-IF (https://blog.vespa.ai/vespa-hybrid-billion-scale-vector-search/)?