Top Qs
Timeline
Chat
Perspective
BeeGFS
Distributed file system From Wikipedia, the free encyclopedia
Remove ads
BeeGFS (formerly FhGFS) is a parallel file system developed for high-performance computing. BeeGFS includes a distributed metadata architecture for scalability and flexibility reasons. It specializes in data throughput.
 | This article contains promotional content. (August 2018) |
BeeGFS was originally developed at the Fraunhofer Center for High Performance Computing in Germany by a team led by Sven Breuner.[2] Breuner later became the CEO of ThinkParQ (2014–2018), the spin-off company that was founded in 2014 to maintain BeeGFS and offer professional services.
While the Community Edition of BeeGFS can be downloaded and used free of charge, the Enterprise Edition must be used under a professional support subscription contract.[3]
Remove ads
History and usage
Summarize
Perspective
BeeGFS started in 2005 as an in-house development at Fraunhofer Center for HPC to replace the existing file system on the institute's new compute cluster and to be used in a production environment.
In 2007, the first beta version of the software was announced during ISC07 in Dresden, Germany and introduced to the public during SC07 in Reno, Nevada. One year later the first stable major release became available.
In 2014, Fraunhofer started its spin-off, the new company called ThinkParQ[4] for BeeGFS. In this process, FhGFS was renamed and became BeeGFS.[5] While ThinkParQ maintains the software and offers professional services, further feature development will continue in cooperation of ThinkParQ and Fraunhofer.
Due to the nature of BeeGFS being free of charge, it is unknown how many active installations there are. However, in 2014 there were already around 100 customers worldwide that used BeeGFS with commercial support by ThinkParQ and Fraunhofer. Among those are academic users such as universities and research facilities[6] as well as commercial companies in fields like the finance or the oil & gas industry.
Notable installations include several TOP500 computers such as the Loewe-CSC[7] cluster at the Goethe University Frankfurt, Germany (No. 22 on installation), the Vienna Scientific Cluster[8] at the University of Vienna, Austria (No. 56 on installation), and the Abel[9] cluster at the University of Oslo, Norway (No. 96 on installation).
BeeGFS and containers
An open-source container storage interface (CSI) driver enables BeeGFS to be used with container orchestrators like Kubernetes.[10] The driver is designed to support environments where containers running in Kubernetes and jobs running in traditional HPC workload managers need to share access to the same BeeGFS file system. The driver enables two main workflows:
- Static provisioning allows administrators to grant containers access to existing directories in BeeGFS.
- Dynamic provisioning allows containers to request BeeGFS storage on-demand (represented as a new directory).
Container access and visibility into the file system is restricted to the intended directory. Dynamic provisioning takes into account BeeGFS features including storage pools and striping when creating the corresponding directory in BeeGFS. General features of a POSIX file system such as the ability to specify permissions on new directories are also exposed, easing integration of global shared storage and containers. This notably simplifies tracking and limiting container consumption of the shared file system using BeeGFS quotas.[11]
Remove ads
Benchmarks
The following benchmarks have been performed on Fraunhofer Seislab,[12] a test and experimental cluster at Fraunhofer ITWM with 25 nodes (20 compute plus 5 storage) and a three-tier memory: 1 TB RAM, 20 TB SSD, 120 TB HDD. Single node performance on the local file system without BeeGFS is 1,332 MB/s (write) and 1,317 MB/s (read).
BeeGFS and exascale
Fraunhofer ITWM is participating in the Dynamic-Exascale Entry Platform – Extended Reach (DEEP-ER) project of the European Union.[13]
See also
References
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads