Top Qs
Timeline
Chat
Perspective
Milvus (vector database)
Open-source distributed vector database From Wikipedia, the free encyclopedia
Remove ads
Milvus is a distributed vector database developed by Zilliz. It is available as both open-source software and a cloud service.
 | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
Remove ads
Milvus is an open-source project under LF AI & Data Foundation[2] distributed under the Apache License 2.0.
Remove ads
History
Milvus has been developed by Zilliz since 2017.[3]
Milvus joined Linux Foundation as an incubation project in January 2020 and became a graduate in June 2021.[2] The details about its architecture and possible applications were presented on ACM SIGMOD Conference in 2021[4]
Milvus 2.0, a major redesign of the whole product with a new architecture,[5] was released in January 2022.
Features
Summarize
Perspective
Similarity search
Major similarity search related features that are available in the active 2.4.x Milvus branch:[6]
- In-memory, on-disk and GPU indices,
- Single query, batch query and range query search,
- Support of sparse vectors, binary vectors, JSON and arrays,
- FP32, FP16 and BF16 data types,
- Euclidean distance, inner product distance and cosine distance support for floating-point data,
- Hamming distance and jaccard distance for binary data,
- Support of graph indices (including HNSW), Inverted-lists based indices and a brute-force search.
- Support of vector quantization for lossy input data compression, including product quantization (PQ) and scalar quantization (SQ), that trades stored data size for accuracy,
- Re-ranking.
Milvus similarity search engine relies on heavily-modified forks of third-party open-source similarity search libraries, such as Faiss,[7][8] DiskANN[9][10] and hnswlib.[11]
Milvus includes optimizations for I/O data layout, specific to graph search indices.[12]
Database
As a database, Milvus provides the following features:[6]
- Column-oriented database
- Four supported data consistency levels, including strong consistency and eventual consistency.[13]
- Data sharding
- Streaming data ingestion, which allows to process and ingest data in real-time as it arrives
- A dynamic schema, which allows inserting the data without a predefined schema
- Independent storage and compute layers
- Multi-tenancy scenarios (database-oriented, collection-oriented, partition-oriented)[14]
- Memory-mapped data storage
- Role-based access control
- Multi-vector and hybrid search[15]
Deployment options
Milvus can be deployed as an embedded database, standalone server, or distributed cluster. Zillis Cloud offers a fully managed version.[16]
GPU support
Milvus provides GPU accelerated index building and search using Nvidia CUDA technology[17][18] via Nvidia RAFT library,[19] including a recent GPU-based graph indexing algorithm Nvidia CAGRA[20]
Integration
Milvus provides official SDK clients for Java, NodeJS, Python and Go.[21] An additional C# SDK client was contributed by Microsoft.[6][22] The database can integrate with Prometheus and Grafana for monitoring and alerts, frameworks Haystack[23] and LangChain,[24] IBM Watsonx,[25] and OpenAI models.[26][27]
Remove ads
See also
References
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads