Comparison of distributed file systems

From Wikipedia, the free encyclopedia

In computing, a distributed file system (DFS) or network file system is any file system that allows access from multiple hosts to files shared via a computer network. This makes it possible for multiple users on multiple machines to share files and storage resources.

Distributed file systems differ in their performance, mutability of content, handling of concurrent writes, handling of permanent or temporary loss of nodes or storage, and their policy of storing content.

Locally managed

Summarize
Perspective

FOSS

More information Client, Written in ...
Client Written in License Access API High availability Shards Efficient Redundancy Redundancy Granularity Initial release year Memory requirements (GB)
Alluxio (Virtual Distributed File System) Java Apache License 2.0 HDFS, FUSE, HTTP/REST, S3 hot standby No Replication[1] File[2] 2013
Ceph C++ LGPL librados (C, C++, Python, Ruby), S3, Swift, FUSE Yes Yes Pluggable erasure codes[3] Pool[4] 2010 1 per TB of storage
Coda C GPL C Yes Yes Replication Volume[5] 1987
GlusterFS C GPLv3 libglusterfs, FUSE, NFS, SMB, Swift, libgfapi mirror Yes Reed-Solomon[6] Volume[7] 2005
HDFS Java Apache License 2.0 Java and C client, HTTP, FUSE[8] transparent master failover No Reed-Solomon[9] File[10] 2005
IPFS Go Apache 2.0 or MIT HTTP gateway, FUSE, Go client, Javascript client, command line tool Yes with IPFS Cluster Replication[11] Block[12] 2015[13]
LizardFS[14] C++ GPLv3 POSIX, FUSE, NFS-Ganesha, Ceph FSAL (via libcephfs) master No Reed-Solomon[15] File[16] 2013
Lustre C GPLv2 POSIX, NFS-Ganesha, NFS, SMB Yes Yes No redundancy[17][18] No redundancy[19][20] 2003
MinIO Go AGPL3.0 AWS S3 API, FTP, SFTP Yes Yes Reed-Solomon[21] Object[22] 2014
MooseFS C GPLv2 POSIX, FUSE master No Replication[23] File[24] 2008
OpenAFS C IBM Public License Virtual file system, Installable File System Replication Volume[25] 2000[26]
OpenIO[27] C AGPLv3 / LGPLv3 Native (Python, C, Java), HTTP/REST, S3, Swift, FUSE (POSIX, NFS, SMB, FTP) Yes Pluggable erasure codes[28] Object[29] 2015 0.5
Quantcast File System C Apache License 2.0 C++ client, FUSE (C++ server: MetaServer and ChunkServer are both in C++) master No Reed-Solomon[30] File[31] 2012
RozoFS C, Python GPLv2 FUSE, SMB, NFS, key/value Yes Mojette[32] Volume[33] 2011[34]
Tahoe-LAFS Python GNU GPL[35] HTTP (browser or CLI), SFTP, FTP, FUSE via SSHFS, pyfilesystem Reed-Solomon[36] File[37] 2007
XtreemFS Java, C++ BSD License libxtreemfs (Java, C++), FUSE Replication[38] File[39] 2009
Close

Proprietary

More information Client, Written in ...
Client Written in License Access API
BeeGFS C / C++ FRAUNHOFER FS (FhGFS) EULA,[40]

GPLv2 client

POSIX
ObjectiveFS[41] C Proprietary POSIX, FUSE
Spectrum Scale (GPFS) C, C++ Proprietary POSIX, NFS, SMB, Swift, S3, HDFS
MapR-FS C, C++ Proprietary POSIX, NFS, FUSE, S3, HDFS, CLI
Isilon OneFS C/C++ Proprietary POSIX, NFS, SMB/CIFS, HDFS, HTTP, FTP, SWIFT Object, CLI, Rest API
Qumulo C/C++ Proprietary POSIX, NFS, SMB/CIFS, CLI, S3, Rest API
Scality C Proprietary FUSE, NFS, REST, AWS S3
Close

Remote access

More information Name, Run by ...
Close

Comparison

Some researchers have made a functional and experimental analysis of several distributed file systems including HDFS, Ceph, Gluster, Lustre and old (1.6.x) version of MooseFS, although this document is from 2013 and a lot of information are outdated (e.g. MooseFS had no HA for Metadata Server at that time).[43]

The cloud based remote distributed storage from major vendors have different APIs and different consistency models.[44]

See also

References

Loading related searches...

Wikiwand - on

Seamless Wikipedia browsing. On steroids.