CuPy

CuPy is an open source library for GPU-accelerated computing with Python programming language, providing support for multi-dimensional arrays, sparse matrices, and a variety of numerical algorithms implemented on top of them.^[3] CuPy shares the same API set as NumPy and SciPy, allowing it to be a drop-in replacement to run NumPy/SciPy code on GPU. CuPy supports Nvidia CUDA GPU platform, and AMD ROCm GPU platform starting in v9.0.^[4][5]

CuPy
Original author(s)	Seiya Tokui
Developer(s)	Community, Preferred Networks, Inc.
Initial release	September 2, 2015; 9 years ago (2015-09-02).[1]
Stable release	v13.3.0[2] / August 22, 2024; 10 months ago (2024-08-22)[2]
Repository	github.com/cupy/cupy
Written in	Python, Cython, CUDA
Operating system	Linux, Windows
Platform	Cross-platform
Type	Numerical analysis
License	MIT
Website	cupy.dev

CuPy has been initially developed as a backend of Chainer deep learning framework, and later established as an independent project in 2017.^[6]

CuPy is a part of the NumPy ecosystem array libraries^[7] and is widely adopted to utilize GPU with Python,^[8] especially in high-performance computing environments such as Summit,^[9] Perlmutter,^[10] EULER,^[11] and ABCI.^[12]

CuPy is a NumFOCUS sponsored project.^[13]

Features

CuPy implements NumPy/SciPy-compatible APIs, as well as features to write user-defined GPU kernels or access low-level APIs.^[14][15]

NumPy-compatible APIs

The same set of APIs defined in the NumPy package (numpy.*) are available under cupy.* package.

• Multi-dimensional array (cupy.ndarray) for boolean, integer, float, and complex data types
• Module-level functions
• Linear algebra functions
• Fast Fourier transform
• Random number generator

SciPy-compatible APIs

The same set of APIs defined in the SciPy package (scipy.*) are available under cupyx.scipy.* package.

• Sparse matrices (cupyx.scipy.sparse.*_matrix) of CSR, COO, CSC, and DIA format
• Discrete Fourier transform
• Advanced linear algebra
• Multidimensional image processing
• Sparse linear algebra
• Special functions
• Signal processing
• Statistical functions

User-defined GPU kernels

• Kernel templates for element-wise and reduction operations
• Raw kernel (CUDA C/C++)
• Just-in-time transpiler (JIT)
• Kernel fusion

Distributed computing

• Distributed communication package (cupyx.distributed), providing collective and peer-to-peer primitives

Low-level CUDA features

• Stream and event
• Memory pool
• Profiler
• Host API binding
• CUDA Python support^[16]

Interoperability

• DLPack^[17]
• CUDA Array Interface^[18]
• NEP 13 (__array_ufunc__)^[19]
• NEP 18 (__array_function__)^[20][21]
• Array API Standard^[22][23]

Examples

Array creation

>>> import cupy as cp
>>> x = cp.array([1, 2, 3])
>>> x
array([1, 2, 3])
>>> y = cp.arange(10)
>>> y
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

Basic operations

>>> import cupy as cp
>>> x = cp.arange(12).reshape(3, 4).astype(cp.float32)
>>> x
array([[ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  6.,  7.],
       [ 8.,  9., 10., 11.]], dtype=float32)
>>> x.sum(axis=1)
array([ 6., 22., 38.], dtype=float32)

Raw CUDA C/C++ kernel

>>> import cupy as cp
>>> kern = cp.RawKernel(r'''
... extern "C" __global__
... void multiply_elemwise(const float* in1, const float* in2, float* out) {
...     int tid = blockDim.x * blockIdx.x + threadIdx.x;
...     out[tid] = in1[tid] * in2[tid];
... }
... ''', 'multiply_elemwise')
>>> in1 = cp.arange(16, dtype=cp.float32).reshape(4, 4)
>>> in2 = cp.arange(16, dtype=cp.float32).reshape(4, 4)
>>> out = cp.zeros((4, 4), dtype=cp.float32)
>>> kern((4,), (4,), (in1, in2, out))  # grid, block and arguments
>>> out
array([[  0.,   1.,   4.,   9.],
       [ 16.,  25.,  36.,  49.],
       [ 64.,  81., 100., 121.],
       [144., 169., 196., 225.]], dtype=float32)

Applications

• spaCy^[24][25]
• XGBoost^[26]
• turboSETI (Berkeley SETI)^[27]
• NVIDIA RAPIDS^{[28][29][30][31]}
• einops^[32][33]
• scikit-learn^[34]
• MONAI
• Chainer^[35]

See also

• Free software portal

• Array programming
• List of numerical-analysis software
• Dask