Top Qs
Timeline
Chat
Perspective
Milvus (vector database)
Open-source distributed vector database From Wikipedia, the free encyclopedia
Remove ads
Milvus is a distributed vector database developed by Zilliz. It is available as both open-source software and a cloud service called Zilliz Cloud.
 | A major contributor to this article appears to have a close connection with its subject. (November 2024) |
Milvus is an open-source project under the LF AI & Data Foundation[2] and is distributed under the Apache License 2.0.
Remove ads
History
Milvus has been developed by Zilliz since 2017.[3]
Milvus joined Linux Foundation as an incubation project in January 2020 and became a graduate in June 2021.[2] The details about its architecture and possible applications were presented at ACM SIGMOD Conference in 2021.[4]
Milvus 2.0, a major redesign of the whole product with a new architecture,[5] was released in January 2022.
Features
Summarize
Perspective
Similarity search
Various similarity search-related features are available in Milvus:[6]
- In-memory, on-disk and GPU indices,
- Single query, batch query and range query search,
- Support of sparse vectors, binary vectors, JSON and arrays,
- FP32, FP16 and BF16 data types,
- Euclidean distance, inner product distance and cosine distance support for floating-point data,
- Hamming distance and jaccard distance for binary data,
- Support of graph indices (including HNSW), Inverted-lists based indices and a brute-force search.
- Support of vector quantization for lossy input data compression, including product quantization (PQ) and scalar quantization (SQ), that trades stored data size for accuracy,
- Re-ranking.
Milvus' similarity search engine relies on heavily-modified forks of third-party open-source similarity search libraries, such as Faiss,[7][8] DiskANN[9][10] and hnswlib.[11]
Milvus includes optimizations for I/O data layout, specific to graph search indices.[12]
Database
As a database, Milvus provides the following features:[6]
- Support for column-oriented databases
- Four supported data consistency levels, including strong consistency and eventual consistency[13]
- Data sharding
- Streaming data ingestion, which allows processing and ingestion of data in real-time as it arrives
- A dynamic schema, which allows insertion of data without a predefined schema
- Independent storage and compute layers
- Support for multi-tenancy scenarios (database-oriented, collection-oriented, partition-oriented)[14]
- Memory-mapped data storage
- Role-based access control
- Multi-vector and hybrid search[15]
Deployment options
Milvus can be deployed as an embedded database, standalone server, or distributed cluster. Zilliz Cloud offers a fully managed version.[16]
GPU support
Milvus provides GPU accelerated index building and search using Nvidia CUDA technology[17][18] via the Nvidia RAFT library,[19] including a recent GPU-based graph indexing algorithm known as CAGRA.[20]
Integration
Milvus provides official SDK clients for Java, NodeJS, Python and Go.[21] An additional C# SDK client was contributed by Microsoft.[6][22] The database can integrate with DataDog,[23] Prometheus and Grafana for monitoring and alerts, as well as generative AI frameworks Haystack,[24] LangChain,[25] IBM Watsonx,[26] and those provided by OpenAI.[27][28]
Several storage providers have built integrations with Milvus to support AI workloads and large-scale vector search. These integrations aim to optimize performance, simplify inferencing workflows, and enhance data management capabilities:
Remove ads
See also
References
External links
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads