Welcome to nvitop’s documentation!

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

https://user-images.githubusercontent.com/16078332/171005261-1aad126e-dc27-4ed3-a89b-7f9c1c998bf7.png — The CLI from `nvitop`.

Installation

It is highly recommended to install nvitop in an isolated virtual environment. Simple installation and run via pipx:

pipx run nvitop

Install from PyPI ():

pip3 install --upgrade nvitop

Note

Python 3.7+ is required, and Python versions lower than 3.7 is not supported.

Install from conda-forge ():

conda install -c conda-forge nvitop

Install the latest version from GitHub ( Commit Count ):

pip3 install --upgrade pip setuptools
pip3 install git+https://github.com/XuehaiPan/nvitop.git#egg=nvitop

Or, clone this repo and install manually:

git clone --depth=1 https://github.com/XuehaiPan/nvitop.git
cd nvitop
pip3 install .

If this repo is useful to you, please star ⭐️ it to let more people know 🤗.

Quick Start

A minimal script to monitor the GPU devices based on APIs from nvitop:

from nvitop import Device

devices = Device.all()  # or Device.cuda.all()
for device in devices:
    processes = device.processes()  # type: Dict[int, GpuProcess]
    sorted_pids = sorted(processes)

    print(device)
    print(f'  - Fan speed:       {device.fan_speed()}%')
    print(f'  - Temperature:     {device.temperature()}C')
    print(f'  - GPU utilization: {device.gpu_utilization()}%')
    print(f'  - Total memory:    {device.memory_total_human()}')
    print(f'  - Used memory:     {device.memory_used_human()}')
    print(f'  - Free memory:     {device.memory_free_human()}')
    print(f'  - Processes ({len(processes)}): {sorted_pids}')
    for pid in sorted_pids:
        print(f'    - {processes[pid]}')
    print('-' * 120)

Another more advanced approach with coloring:

import time

from nvitop import Device, GpuProcess, NA, colored

print(colored(time.strftime('%a %b %d %H:%M:%S %Y'), color='red', attrs=('bold',)))

devices = Device.cuda.all()  # or `Device.all()` to use NVML ordinal instead
separator = False
for device in devices:
    processes = device.processes()  # type: Dict[int, GpuProcess]

    print(colored(str(device), color='green', attrs=('bold',)))
    print(colored('  - Fan speed:       ', color='blue', attrs=('bold',)) + f'{device.fan_speed()}%')
    print(colored('  - Temperature:     ', color='blue', attrs=('bold',)) + f'{device.temperature()}C')
    print(colored('  - GPU utilization: ', color='blue', attrs=('bold',)) + f'{device.gpu_utilization()}%')
    print(colored('  - Total memory:    ', color='blue', attrs=('bold',)) + f'{device.memory_total_human()}')
    print(colored('  - Used memory:     ', color='blue', attrs=('bold',)) + f'{device.memory_used_human()}')
    print(colored('  - Free memory:     ', color='blue', attrs=('bold',)) + f'{device.memory_free_human()}')
    if len(processes) > 0:
        processes = GpuProcess.take_snapshots(processes.values(), failsafe=True)
        processes.sort(key=lambda process: (process.username, process.pid))

        print(colored(f'  - Processes ({len(processes)}):', color='blue', attrs=('bold',)))
        fmt = '    {pid:<5}  {username:<8} {cpu:>5}  {host_memory:>8} {time:>8}  {gpu_memory:>8}  {sm:>3}  {command:<}'.format
        print(colored(fmt(pid='PID', username='USERNAME',
                          cpu='CPU%', host_memory='HOST-MEM', time='TIME',
                          gpu_memory='GPU-MEM', sm='SM%',
                          command='COMMAND'),
                      attrs=('bold',)))
        for snapshot in processes:
            print(fmt(pid=snapshot.pid,
                      username=snapshot.username[:7] + ('+' if len(snapshot.username) > 8 else snapshot.username[7:8]),
                      cpu=snapshot.cpu_percent, host_memory=snapshot.host_memory_human,
                      time=snapshot.running_time_human,
                      gpu_memory=(snapshot.gpu_memory_human if snapshot.gpu_memory_human is not NA else 'WDDM:N/A'),
                      sm=snapshot.gpu_sm_utilization,
                      command=snapshot.command))
    else:
        print(colored('  - No Running Processes', attrs=('bold',)))

    if separator:
        print('-' * 120)
    separator = True

https://user-images.githubusercontent.com/16078332/177041142-fe988d58-6a97-4559-84fd-b51204cf9231.png — An example monitoring script built with APIs from `nvitop`.

Please refer to section More than a Monitor in README for more examples.

API Reference

Module Contents

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

nvitop.version.PYNVML_VERSION_CANDIDATES = ('11.450.51', '11.450.129', '11.460.79', '11.470.66', '11.495.46', '11.510.69', '11.515.48', '11.515.75', '11.525.84', '11.525.112', '11.525.131', '12.535.77', '12.535.108', '12.535.133')

The list of supported nvidia-ml-py versions. See also: nvidia-ml-py’s Release History.

To install nvitop with a specific version of nvidia-ml-py, use nvitop[pynvml-xx.yyy.zzz], for example:

pip3 install 'nvitop[pynvml-11.450.51]'

or

pip3 install nvitop nvidia-ml-py==11.450.51

Note

The package nvidia-ml-py is not backward compatible over releases. This may cause problems such as “Function Not Found” errors with old versions of NVIDIA drivers (e.g. the NVIDIA R430 driver on Ubuntu 16.04 LTS). The ideal solution is to let the user install the best-fit version of nvidia-ml-py. See also: nvidia-ml-py’s Release History.

nvidia-ml-py==11.450.51 is the last version supports the NVIDIA R430 driver (CUDA 10.x). Since nvidia-ml-py>=11.450.129, the definition of struct nvmlProcessInfo_t has introduced two new fields gpuInstanceId and computeInstanceId (GI ID and CI ID in newer nvidia-smi) which are incompatible with some old NVIDIA drivers. nvitop may not display the processes correctly due to this incompatibility.

class nvitop.NaType[source]

Bases: str

A singleton (str: 'N/A') class represents a not applicable value.

The NA instance behaves like a str instance ('N/A') when doing string manipulation (e.g. concatenation). For arithmetic operations, for example NA / 1024 / 1024, it acts like the math.nan.

Examples

>>> NA
'N/A'

>>> 'memory usage: {}'.format(NA)  # NA is an instance of `str`
'memory usage: N/A'
>>> NA.lower()                     # NA is an instance of `str`
'n/a'
>>> NA.ljust(5)                    # NA is an instance of `str`
'N/A  '
>>> NA + ' str'                    # string contamination if the operand is a string
'N/A str'

>>> float(NA)                      # explicit conversion to float (`math.nan`)
nan
>>> NA + 1                         # auto-casting to float if the operand is a number
nan
>>> NA * 1024                      # auto-casting to float if the operand is a number
nan
>>> NA / (1024 * 1024)             # auto-casting to float if the operand is a number
nan

static __new__(cls) → NaType[source]: Get the singleton instance (nvitop.NA).

__bool__() → bool[source]

Convert NA to bool and return False.

>>> bool(NA)
False

__int__() → int[source]

Convert NA to int and return 0.

>>> int(NA)
0

__float__() → float[source]

Convert NA to float and return math.nan.

>>> float(NA)
nan
>>> float(NA) is math.nan
True

__add__(other: object) → str | float[source]

Return math.nan if the operand is a number or uses string concatenation if the operand is a string (NA + other).

A special case is when the operand is nvitop.NA itself, the result is math.nan instead of 'N/AN/A'.

>>> NA + ' str'
'N/A str'
>>> NA + NA
nan
>>> NA + 1
nan
>>> NA + 1.0
nan

__radd__(other: object) → str | float[source]

Return math.nan if the operand is a number or uses string concatenation if the operand is a string (other + NA).

>>> 'str' + NA
'strN/A'
>>> 1 + NA
nan
>>> 1.0 + NA
nan

__sub__(other: object) → float[source]

Return math.nan if the operand is a number (NA - other).

>>> NA - 'str'
TypeError: unsupported operand type(s) for -: 'NaType' and 'str'
>>> NA - NA
'N/AN/A'
>>> NA + 1
nan
>>> NA + 1.0
nan

__rsub__(other: object) → float[source]

Return math.nan if the operand is a number (other - NA).

>>> 'str' - NA
TypeError: unsupported operand type(s) for -: 'str' and 'NaType'
>>> 1 - NA
nan
>>> 1.0 - NA
nan

__mul__(other: object) → float[source]

Return math.nan if the operand is a number (NA * other).

A special case is when the operand is nvitop.NA itself, the result is also math.nan.

>>> NA * 1024
nan
>>> NA * 1024.0
nan
>>> NA * NA
nan

__rmul__(other: object) → float[source]

Return math.nan if the operand is a number (other * NA).

>>> 1024 * NA
nan
>>> 1024.0 * NA
nan

__truediv__(other: object) → float[source]

Return math.nan if the operand is a number (NA / other).

>>> NA / 1024
nan
>>> NA / 1024.0
nan
>>> NA / 0
ZeroDivisionError: float division by zero
>>> NA / 0.0
ZeroDivisionError: float division by zero
>>> NA / NA
nan

__rtruediv__(other: object) → float[source]

Return math.nan if the operand is a number (other / NA).

>>> 1024 / NA
nan
>>> 1024.0 / NA
nan

__floordiv__(other: object) → float[source]

Return math.nan if the operand is a number (NA // other).

>>> NA // 1024
nan
>>> NA // 1024.0
nan
>>> NA / 0
ZeroDivisionError: float division by zero
>>> NA / 0.0
ZeroDivisionError: float division by zero
>>> NA // NA
nan

__rfloordiv__(other: object) → float[source]

Return math.nan if the operand is a number (other // NA).

>>> 1024 // NA
nan
>>> 1024.0 // NA
nan

__mod__(other: object) → float[source]

Return math.nan if the operand is a number (NA % other).

>>> NA % 1024
nan
>>> NA % 1024.0
nan
>>> NA % 0
ZeroDivisionError: float modulo
>>> NA % 0.0
ZeroDivisionError: float modulo

__rmod__(other: object) → float[source]

Return math.nan if the operand is a number (other % NA).

>>> 1024 % NA
nan
>>> 1024.0 % NA
nan

__divmod__(other: object) → tuple[float, float][source]

The pair (NA // other, NA % other) (divmod(NA, other)).

>>> divmod(NA, 1024)
(nan, nan)
>>> divmod(NA, 1024.0)
(nan, nan)
>>> divmod(NA, 0)
ZeroDivisionError: float floor division by zero
>>> divmod(NA, 0.0)
ZeroDivisionError: float floor division by zero

__rdivmod__(other: object) → tuple[float, float][source]

The pair (other // NA, other % NA) (divmod(other, NA)).

>>> divmod(1024, NA)
(nan, nan)
>>> divmod(1024.0, NA)
(nan, nan)

__pos__() → float[source]

Return math.nan (+NA).

>>> +NA
nan

__neg__() → float[source]

Return math.nan (-NA).

>>> -NA
nan

__abs__() → float[source]

Return math.nan (abs(NA)).

>>> abs(NA)
nan

__round__(ndigits: int | None = None) → int | float[source]

Round nvitop.NA to ndigits decimal places, defaulting to 0.

If ndigits is omitted or None, returns 0, otherwise returns math.nan.

>>> round(NA)
0
>>> round(NA, 0)
nan
>>> round(NA, 1)
nan

__lt__(x: object) → bool[source]: The nvitop.NA is always greater than any number, or uses the dictionary order for string.

__le__(x: object) → bool[source]: The nvitop.NA is always greater than any number, or uses the dictionary order for string.

__gt__(x: object) → bool[source]: The nvitop.NA is always greater than any number, or uses the dictionary order for string.

__ge__(x: object) → bool[source]: The nvitop.NA is always greater than any number, or uses the dictionary order for string.

__format__(format_spec: str) → str[source]: Format nvitop.NA according to format_spec.

nvitop.NA = 'N/A': The singleton instance of NaType. The actual value is str: 'N/A'.

nvitop.NotApplicableType: alias of NaType

nvitop.NotApplicable = 'N/A': The singleton instance of NaType. The actual value is str: 'N/A'.

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

exception nvitop.NVMLError(value)[source]

Bases: Exception

Base exception class for NVML query errors.

static __new__(typ, value)[source]: Map value to a proper subclass of NVMLError.

nvitop.nvmlCheckReturn(retval: _Any, types: type | tuple[type, ...] | None = None) → bool[source]: Check whether the return value is not nvitop.NA and is one of the given types.

Bases: object

Live class of the GPU devices, different from the device snapshots.

Device.__new__() returns different types depending on the given arguments.

- (index: int)        -> PhysicalDevice
- (index: (int, int)) -> MigDevice
- (uuid: str)         -> Union[PhysicalDevice, MigDevice]  # depending on the UUID value
- (bus_id: str)       -> PhysicalDevice

Examples

>>> Device.driver_version()              # version of the installed NVIDIA display driver
'470.129.06'

>>> Device.count()                       # number of NVIDIA GPUs in the system
10

>>> Device.all()                         # all physical devices in the system
[
    PhysicalDevice(index=0, ...),
    PhysicalDevice(index=1, ...),
    ...
]

>>> nvidia0 = Device(index=0)            # -> PhysicalDevice
>>> mig10   = Device(index=(1, 0))       # -> MigDevice
>>> nvidia2 = Device(uuid='GPU-xxxxxx')  # -> PhysicalDevice
>>> mig30   = Device(uuid='MIG-xxxxxx')  # -> MigDevice

>>> nvidia0.memory_free()                # total free memory in bytes
11550654464
>>> nvidia0.memory_free_human()          # total free memory in human readable format
'11016MiB'

>>> nvidia2.as_snapshot()                # takes an onetime snapshot of the device
PhysicalDeviceSnapshot(
    real=PhysicalDevice(index=2, ...),
    ...
)

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.
libnvml.NVMLError_NotFound – If the device is not found for the given NVML identifier.
libnvml.NVMLError_InvalidArgument – If the device index is out of range.
TypeError – If the number of non-None arguments is not exactly 1.
TypeError – If the given index is a tuple but is not consist of two integers.

UUID_PATTERN: re.Pattern = re.compile('^ # full match\n (?:(?P<MigMode>MIG)-)? # prefix for MIG UUID\n (?:(?P<GpuUuid>GPU)-)? # prefix for GPU UUID\n (?, re.VERBOSE)

GPU_PROCESS_CLASS: alias of GpuProcess

cuda: alias of CudaDevice

classmethod is_available() → bool[source]: Test whether there are any devices and the NVML library is successfully loaded.

static driver_version() → str | NaType[source]

The version of the installed NVIDIA display driver. This is an alphanumeric string.

Command line equivalent:

nvidia-smi --id=0 --format=csv,noheader,nounits --query-gpu=driver_version

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.

static cuda_driver_version() → str | NaType[source]

The maximum CUDA version supported by the NVIDIA display driver. This is an alphanumeric string.

This can be different from the version of the CUDA Runtime. See also cuda_runtime_version().

Returns: Union[str, NaType]: The maximum CUDA version supported by the NVIDIA display driver.

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.

static max_cuda_version() → str | NaType

The maximum CUDA version supported by the NVIDIA display driver. This is an alphanumeric string.

This can be different from the version of the CUDA Runtime. See also cuda_runtime_version().

Returns: Union[str, NaType]: The maximum CUDA version supported by the NVIDIA display driver.

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.

static cuda_runtime_version() → str | NaType[source]

The CUDA Runtime version. This is an alphanumeric string.

This can be different from the CUDA driver version. See also cuda_driver_version().

Returns: Union[str, NaType]: The CUDA Runtime version, or nvitop.NA when no CUDA Runtime is available or no CUDA-capable devices are present.

static cudart_version() → str | NaType

The CUDA Runtime version. This is an alphanumeric string.

This can be different from the CUDA driver version. See also cuda_driver_version().

Returns: Union[str, NaType]: The CUDA Runtime version, or nvitop.NA when no CUDA Runtime is available or no CUDA-capable devices are present.

classmethod count() → int[source]

The number of NVIDIA GPUs in the system.

Command line equivalent:

nvidia-smi --id=0 --format=csv,noheader,nounits --query-gpu=count

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.

classmethod all() → list[PhysicalDevice][source]: Return a list of all physical devices in the system.

classmethod from_indices(indices: int | Iterable[int | tuple[int, int]] | None = None) → list[PhysicalDevice | MigDevice][source]

Return a list of devices of the given indices.

Parameters:: indices (Iterable[Union[int, Tuple[int, int]]]) – Indices of the devices. For each index, get PhysicalDevice for single int and MigDevice for tuple (int, int). That is: - (int) -> PhysicalDevice - ((int, int)) -> MigDevice

Returns: List[Union[PhysicalDevice, MigDevice]]: A list of PhysicalDevice and/or MigDevice instances of the given indices.

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.
libnvml.NVMLError_NotFound – If the device is not found for the given NVML identifier.
libnvml.NVMLError_InvalidArgument – If the device index is out of range.

static from_cuda_visible_devices() → list[CudaDevice][source]

Return a list of all CUDA visible devices.

The CUDA ordinal will be enumerate from the CUDA_VISIBLE_DEVICES environment variable.

Note

The result could be empty if the CUDA_VISIBLE_DEVICES environment variable is invalid.

See also for CUDA Device Enumeration:

Returns: List[CudaDevice]

A list of CudaDevice instances.

static from_cuda_indices(cuda_indices: int | Iterable[int] | None = None) → list[CudaDevice][source]

Return a list of CUDA devices of the given CUDA indices.

The CUDA ordinal will be enumerate from the CUDA_VISIBLE_DEVICES environment variable.

See also for CUDA Device Enumeration:

Parameters:: cuda_indices (Iterable[int]) – The indices of the GPU in CUDA ordinal, if not given, returns all visible CUDA devices.

Returns: List[CudaDevice]: A list of CudaDevice of the given CUDA indices.

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.
RuntimeError – If the index is out of range for the given CUDA_VISIBLE_DEVICES environment variable.

static parse_cuda_visible_devices(cuda_visible_devices: str | None = <VALUE OMITTED>) → list[int] | list[tuple[int, int]][source]

Parse the given CUDA_VISIBLE_DEVICES value into a list of NVML device indices.

This is a alias of parse_cuda_visible_devices().

Note

The result could be empty if the CUDA_VISIBLE_DEVICES environment variable is invalid.

See also for CUDA Device Enumeration:

Parameters:: cuda_visible_devices (Optional[str]) – The value of the CUDA_VISIBLE_DEVICES variable. If not given, the value from the environment will be used. If explicitly given by None, the CUDA_VISIBLE_DEVICES environment variable will be unset before parsing.

Returns: Union[List[int], List[Tuple[int, int]]]: A list of int (physical device) or a list of tuple of two integers (MIG device) for the corresponding real device indices.

static normalize_cuda_visible_devices(cuda_visible_devices: str | None = <VALUE OMITTED>) → str[source]

Parse the given CUDA_VISIBLE_DEVICES value and convert it into a comma-separated string of UUIDs.

This is an alias of normalize_cuda_visible_devices().

Note

The result could be empty string if the CUDA_VISIBLE_DEVICES environment variable is invalid.

See also for CUDA Device Enumeration:

Parameters:: cuda_visible_devices (Optional[str]) – The value of the CUDA_VISIBLE_DEVICES variable. If not given, the value from the environment will be used. If explicitly given by None, the CUDA_VISIBLE_DEVICES environment variable will be unset before parsing.

Returns: str: The comma-separated string (GPU UUIDs) of the CUDA_VISIBLE_DEVICES environment variable.

Create a new instance of Device.

The type of the result is determined by the given argument.

- (index: int)        -> PhysicalDevice
- (index: (int, int)) -> MigDevice
- (uuid: str)         -> Union[PhysicalDevice, MigDevice]  # depending on the UUID value
- (bus_id: str)       -> PhysicalDevice

Note: This method takes exact 1 non-None argument.

Returns: Union[PhysicalDevice, MigDevice]: A PhysicalDevice instance or a MigDevice instance.

Raises:

TypeError – If the number of non-None arguments is not exactly 1.
TypeError – If the given index is a tuple but is not consist of two integers.

__init__(index: int | str | None = None, *, uuid: str | None = None, bus_id: str | None = None) → None[source]

Initialize the instance created by __new__().

Raises:

libnvml.NVMLError_LibraryNotFound – If cannot find the NVML library, usually the NVIDIA driver is not installed.
libnvml.NVMLError_DriverNotLoaded – If NVIDIA driver is not loaded.
libnvml.NVMLError_LibRmVersionMismatch – If RM detects a driver/library version mismatch, usually after an upgrade for NVIDIA driver without reloading the kernel module.
libnvml.NVMLError_NotFound – If the device is not found for the given NVML identifier.
libnvml.NVMLError_InvalidArgument – If the device index is out of range.

__repr__() → str[source]: Return a string representation of the device.

__eq__(other: object) → bool[source]: Test equality to other object.

__hash__() → int[source]: Return a hash value of the device.

__getattr__(name: str) → Any | Callable[..., Any][source]

Get the object attribute.

If the attribute is not defined, make a method from pynvml.nvmlDeviceGet<AttributeName>(handle). The attribute name will be converted to PascalCase string.

Raises:: AttributeError – If the attribute is not defined in pynvml.py.

Examples

>>> device = Device(0)

>>> # Method `cuda_compute_capability` is not implemented in the class definition
>>> PhysicalDevice.cuda_compute_capability
AttributeError: type object 'Device' has no attribute 'cuda_compute_capability'

>>> # Dynamically create a new method from `pynvml.nvmlDeviceGetCudaComputeCapability(device.handle, *args, **kwargs)`
>>> device.cuda_compute_capability
<function PhysicalDevice.cuda_compute_capability at 0x7fbfddf5d9d0>

>>> device.cuda_compute_capability()
(8, 6)

__reduce__() → tuple[type[Device], tuple[int | tuple[int, int]]][source]: Return state information for pickling.

property index: int | tuple[int, int]

The NVML index of the device.

Returns: Union[int, Tuple[int, int]]: Returns an int for physical device and tuple of two integers for MIG device.

property nvml_index: int | tuple[int, int]

The NVML index of the device.

Returns: Union[int, Tuple[int, int]]: Returns an int for physical device and tuple of two integers for MIG device.

property physical_index: int

The index of the physical device.

Returns: int: An int for the physical device index. For MIG devices, returns the index of the parent physical device.

property handle: LP_struct_c_nvmlDevice_t: The NVML device handle.

property cuda_index: int

The CUDA device index.

The value will be evaluated on the first call.

Raises:: RuntimeError – If the current device is not visible to CUDA applications (i.e. not listed in the CUDA_VISIBLE_DEVICES environment variable or the environment variable is invalid).

name() → str | NaType[source]

The official product name of the GPU. This is an alphanumeric string. For all products.

Returns: Union[str, NaType]: The official product name, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=name

uuid() → str | NaType[source]

This value is the globally unique immutable alphanumeric identifier of the GPU.

It does not correspond to any physical label on the board.

Returns: Union[str, NaType]: The UUID of the device, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=name

bus_id() → str | NaType[source]

PCI bus ID as “domain:bus:device.function”, in hex.

Returns: Union[str, NaType]: The PCI bus ID of the device, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=pci.bus_id

serial() → str | NaType[source]

This number matches the serial number physically printed on each board.

It is a globally unique immutable alphanumeric value.

Returns: Union[str, NaType]: The serial number of the device, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=serial

memory_info() → MemoryInfo[source]

Return a named tuple with memory information (in bytes) for the device.

Returns: MemoryInfo(total, free, used): A named tuple with memory information, the item could be nvitop.NA when not applicable.

memory_total() → int | NaType[source]

Total installed GPU memory in bytes.

Returns: Union[int, NaType]: Total installed GPU memory in bytes, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=memory.total

memory_used() → int | NaType[source]

Total memory allocated by active contexts in bytes.

Returns: Union[int, NaType]: Total memory allocated by active contexts in bytes, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=memory.used

memory_free() → int | NaType[source]

Total free memory in bytes.

Returns: Union[int, NaType]: Total free memory in bytes, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=memory.free

memory_total_human() → str | NaType[source]

Total installed GPU memory in human readable format.

Returns: Union[str, NaType]: Total installed GPU memory in human readable format, or nvitop.NA when not applicable.

memory_used_human() → str | NaType[source]

Total memory allocated by active contexts in human readable format.

Returns: Union[int, NaType]: Total memory allocated by active contexts in human readable format, or nvitop.NA when not applicable.

memory_free_human() → str | NaType[source]

Total free memory in human readable format.

Returns: Union[int, NaType]: Total free memory in human readable format, or nvitop.NA when not applicable.

memory_percent() → float | NaType[source]

The percentage of used memory over total memory (0 <= p <= 100).

Returns: Union[float, NaType]: The percentage of used memory over total memory, or nvitop.NA when not applicable.

memory_usage() → str[source]

The used memory over total memory in human readable format.

Returns: str: The used memory over total memory in human readable format, or 'N/A / N/A' when not applicable.

bar1_memory_info() → MemoryInfo[source]

Return a named tuple with BAR1 memory information (in bytes) for the device.

Returns: MemoryInfo(total, free, used): A named tuple with BAR1 memory information, the item could be nvitop.NA when not applicable.

bar1_memory_total() → int | NaType[source]

Total BAR1 memory in bytes.

Returns: Union[int, NaType]: Total BAR1 memory in bytes, or nvitop.NA when not applicable.

bar1_memory_used() → int | NaType[source]

Total used BAR1 memory in bytes.

Returns: Union[int, NaType]: Total used BAR1 memory in bytes, or nvitop.NA when not applicable.

bar1_memory_free() → int | NaType[source]

Total free BAR1 memory in bytes.

Returns: Union[int, NaType]: Total free BAR1 memory in bytes, or nvitop.NA when not applicable.

bar1_memory_total_human() → str | NaType[source]

Total BAR1 memory in human readable format.

Returns: Union[int, NaType]: Total BAR1 memory in human readable format, or nvitop.NA when not applicable.

bar1_memory_used_human() → str | NaType[source]

Total used BAR1 memory in human readable format.

Returns: Union[int, NaType]: Total used BAR1 memory in human readable format, or nvitop.NA when not applicable.

bar1_memory_free_human() → str | NaType[source]

Total free BAR1 memory in human readable format.

Returns: Union[int, NaType]: Total free BAR1 memory in human readable format, or nvitop.NA when not applicable.

bar1_memory_percent() → float | NaType[source]

The percentage of used BAR1 memory over total BAR1 memory (0 <= p <= 100).

Returns: Union[float, NaType]: The percentage of used BAR1 memory over total BAR1 memory, or nvitop.NA when not applicable.

bar1_memory_usage() → str[source]

The used BAR1 memory over total BAR1 memory in human readable format.

Returns: str: The used BAR1 memory over total BAR1 memory in human readable format, or 'N/A / N/A' when not applicable.

utilization_rates() → UtilizationRates[source]

Return a named tuple with GPU utilization rates (in percentage) for the device.

Returns: UtilizationRates(gpu, memory, encoder, decoder): A named tuple with GPU utilization rates (in percentage) for the device, the item could be nvitop.NA when not applicable.

gpu_utilization() → int | NaType[source]

Percent of time over the past sample period during which one or more kernels was executing on the GPU.

The sample period may be between 1 second and 1/6 second depending on the product.

Returns: Union[int, NaType]: The GPU utilization rate in percentage, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=utilization.gpu

gpu_percent() → int | NaType

Percent of time over the past sample period during which one or more kernels was executing on the GPU.

The sample period may be between 1 second and 1/6 second depending on the product.

Returns: Union[int, NaType]: The GPU utilization rate in percentage, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=utilization.gpu

memory_utilization() → int | NaType[source]

Percent of time over the past sample period during which global (device) memory was being read or written.

The sample period may be between 1 second and 1/6 second depending on the product.

Returns: Union[int, NaType]: The memory bandwidth utilization rate of the GPU in percentage, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=utilization.memory

encoder_utilization() → int | NaType[source]

The encoder utilization rate in percentage.

Returns: Union[int, NaType]: The encoder utilization rate in percentage, or nvitop.NA when not applicable.

decoder_utilization() → int | NaType[source]

The decoder utilization rate in percentage.

Returns: Union[int, NaType]: The decoder utilization rate in percentage, or nvitop.NA when not applicable.

clock_infos() → ClockInfos[source]

Return a named tuple with current clock speeds (in MHz) for the device.

Returns: ClockInfos(graphics, sm, memory, video): A named tuple with current clock speeds (in MHz) for the device, the item could be nvitop.NA when not applicable.

clocks() → ClockInfos

Return a named tuple with current clock speeds (in MHz) for the device.

Returns: ClockInfos(graphics, sm, memory, video): A named tuple with current clock speeds (in MHz) for the device, the item could be nvitop.NA when not applicable.

max_clock_infos() → ClockInfos[source]

Return a named tuple with maximum clock speeds (in MHz) for the device.

Returns: ClockInfos(graphics, sm, memory, video): A named tuple with maximum clock speeds (in MHz) for the device, the item could be nvitop.NA when not applicable.

max_clocks() → ClockInfos

Return a named tuple with maximum clock speeds (in MHz) for the device.

Returns: ClockInfos(graphics, sm, memory, video): A named tuple with maximum clock speeds (in MHz) for the device, the item could be nvitop.NA when not applicable.

clock_speed_infos() → ClockSpeedInfos[source]

Return a named tuple with the current and the maximum clock speeds (in MHz) for the device.

Returns: ClockSpeedInfos(current, max): A named tuple with the current and the maximum clock speeds (in MHz) for the device.

graphics_clock() → int | NaType[source]

Current frequency of graphics (shader) clock in MHz.

Returns: Union[int, NaType]: The current frequency of graphics (shader) clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.current.graphics

sm_clock() → int | NaType[source]

Current frequency of SM (Streaming Multiprocessor) clock in MHz.

Returns: Union[int, NaType]: The current frequency of SM (Streaming Multiprocessor) clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.current.sm

memory_clock() → int | NaType[source]

Current frequency of memory clock in MHz.

Returns: Union[int, NaType]: The current frequency of memory clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.current.memory

video_clock() → int | NaType[source]

Current frequency of video encoder/decoder clock in MHz.

Returns: Union[int, NaType]: The current frequency of video encoder/decoder clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.current.video

max_graphics_clock() → int | NaType[source]

Maximum frequency of graphics (shader) clock in MHz.

Returns: Union[int, NaType]: The maximum frequency of graphics (shader) clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.max.graphics

max_sm_clock() → int | NaType[source]

Maximum frequency of SM (Streaming Multiprocessor) clock in MHz.

Returns: Union[int, NaType]: The maximum frequency of SM (Streaming Multiprocessor) clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.max.sm

max_memory_clock() → int | NaType[source]

Maximum frequency of memory clock in MHz.

Returns: Union[int, NaType]: The maximum frequency of memory clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.max.memory

max_video_clock() → int | NaType[source]

Maximum frequency of video encoder/decoder clock in MHz.

Returns: Union[int, NaType]: The maximum frequency of video encoder/decoder clock in MHz, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=clocks.max.video

fan_speed() → int | NaType[source]

The fan speed value is the percent of the product’s maximum noise tolerance fan speed that the device’s fan is currently intended to run at.

This value may exceed 100% in certain cases. Note: The reported speed is the intended fan speed. If the fan is physically blocked and unable to spin, this output will not match the actual fan speed. Many parts do not report fan speeds because they rely on cooling via fans in the surrounding enclosure.

Returns: Union[int, NaType]: The fan speed value in percentage, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=fan.speed

temperature() → int | NaType[source]

Core GPU temperature in degrees C.

Returns: Union[int, NaType]: The core GPU temperature in Celsius degrees, or nvitop.NA when not applicable.

Command line equivalent:

nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=temperature.gpu

power_usage() → int | NaType[source]

The last measured power draw for the entire board in milliwatts.

Returns: Union[int, NaType]: The power draw for the entire board in milliwatts, or nvitop.NA when not applicable.

Command line equivalent:

$(( "$(nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=power.draw)" * 1000 ))

power_draw() → int | NaType

The last measured power draw for the entire board in milliwatts.

Returns: Union[int, NaType]: The power draw for the entire board in milliwatts, or nvitop.NA when not applicable.

Command line equivalent:

$(( "$(nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=power.draw)" * 1000 ))

power_limit() → int | NaType[source]

The software power limit in milliwatts.

Set by software like nvidia-smi.

Returns: Union[int, NaType]: The software power limit in milliwatts, or nvitop.NA when not applicable.

Command line equivalent:

$(( "$(nvidia-smi --id=<IDENTIFIER> --format=csv,noheader,nounits --query-gpu=power.limit)" * 1000 ))

power_status() → str[source]

The string of power usage over power limit in watts.

Returns: str: The string of power usage over power limit in watts, or 'N/A / N/A' when not applicable.

pcie_throughput() → ThroughputInfo[source]

The current PCIe throughput in KiB/s.

This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval.

Returns: ThroughputInfo(tx, rx): A named tuple with current PCIe throughput in KiB/s, the item could be nvitop.NA when not applicable.

pcie_tx_throughput() → int | NaType[source]

The current PCIe transmit throughput in KiB/s.

This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval.

Returns: Union[int, NaType]: The current PCIe transmit throughput in KiB/s, or nvitop.NA when not applicable.

pcie_rx_throughput() → int | NaType[source]

The current PCIe receive throughput in KiB/s.

This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval.

Returns: Union[int, NaType]: The current PCIe receive throughput in KiB/s, or nvitop.NA when not applicable.

pcie_tx_throughput_human() → str | NaType[source]

The current PCIe transmit throughput in human readable format.

This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval.

Returns: Union[str, NaType]: The current PCIe transmit throughput in human readable format, or nvitop.NA when not applicable.

pcie_rx_throughput_human() → str | NaType[source]

The current PCIe receive throughput in human readable format.

This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval.

Returns: Union[str, NaType]: The current PCIe receive throughput in human readable format, or nvitop.NA when not applicable.

nvlink_link_count() → int[source]

The number of NVLinks that the GPU has.

Returns: Union[int, NaType]: The number of NVLinks that the GPU has.

nvlink_throughput(interval: float | None = None) → list[ThroughputInfo][source]

The current NVLink throughput for each NVLink in KiB/s.