CUDA ---- device管理

device管理

NVIDIA提供了集中凡是来查询和管理GPU device，掌握GPU信息查询很重要，因为这可以帮助你设置kernel的执行配置。

本博文将主要介绍下面两方面内容：

• CUDA runtime API function
• NVIDIA系统管理命令行

使用runtime API来查询GPU信息

你可以使用下面的function来查询所有关于GPU device 的信息：

cudaError_t cudaGetDeviceProperties(cudaDeviceProp *prop, int device);

GPU的信息放在cudaDeviceProp这个结构体中。

代码

#include <cuda_runtime.h>
#include <stdio.h>
int main(int argc, char **argv) {    
　　printf("%s Starting...\n", argv[]);
    int deviceCount = ;
    cudaError_t error_id = cudaGetDeviceCount(&deviceCount);
    if (error_id != cudaSuccess) {
        printf("cudaGetDeviceCount returned %d\n-> %s\n",
        (int)error_id, cudaGetErrorString(error_id));
        printf("Result = FAIL\n");
        exit(EXIT_FAILURE);
    }
    if (deviceCount == ) {
        printf("There are no available device(s) that support CUDA\n");
    } else {
        printf("Detected %d CUDA Capable device(s)\n", deviceCount);
    }
    int dev, driverVersion = , runtimeVersion = ;
    dev =;
    cudaSetDevice(dev);
    cudaDeviceProp deviceProp;
    cudaGetDeviceProperties(&deviceProp, dev);
    printf("Device %d: \"%s\"\n", dev, deviceProp.name);
    cudaDriverGetVersion(&driverVersion);
    cudaRuntimeGetVersion(&runtimeVersion);
    printf(" CUDA Driver Version / Runtime Version %d.%d / %d.%d\n",driverVersion/, (driverVersion%)/,runtimeVersion/, (runtimeVersion%)/);
    printf(" CUDA Capability Major/Minor version number: %d.%d\n",deviceProp.major, deviceProp.minor);
    printf(" Total amount of global memory: %.2f MBytes (%llu bytes)\n",(float)deviceProp.totalGlobalMem/(pow(1024.0,)),(unsigned long long) deviceProp.totalGlobalMem);
    printf(" GPU Clock rate: %.0f MHz (%0.2f GHz)\n",deviceProp.clockRate * 1e-3f, deviceProp.clockRate * 1e-6f);
    printf(" Memory Clock rate: %.0f Mhz\n",deviceProp.memoryClockRate * 1e-3f);
    printf(" Memory Bus Width: %d-bit\n",deviceProp.memoryBusWidth);
    if (deviceProp.l2CacheSize) {
        printf(" L2 Cache Size: %d bytes\n",
        deviceProp.l2CacheSize);
    }
    printf(" Max Texture Dimension Size (x,y,z) 1D=(%d), 2D=(%d,%d), 3D=(%d,%d,%d)\n",
    deviceProp.maxTexture1D , deviceProp.maxTexture2D[],
    deviceProp.maxTexture2D[],
    deviceProp.maxTexture3D[], deviceProp.maxTexture3D[],
    deviceProp.maxTexture3D[]);
    printf(" Max Layered Texture Size (dim) x layers 1D=(%d) x %d, 2D=(%d,%d) x %d\n",
    deviceProp.maxTexture1DLayered[], deviceProp.maxTexture1DLayered[],
    deviceProp.maxTexture2DLayered[], deviceProp.maxTexture2DLayered[],
    deviceProp.maxTexture2DLayered[]);
    printf(" Total amount of constant memory: %lu bytes\n",deviceProp.totalConstMem);
    printf(" Total amount of shared memory per block: %lu bytes\n",deviceProp.sharedMemPerBlock);
    printf(" Total number of registers available per block: %d\n",deviceProp.regsPerBlock);
    printf(" Warp size: %d\n", deviceProp.warpSize);
    printf(" Maximum number of threads per multiprocessor: %d\n",deviceProp.maxThreadsPerMultiProcessor);
    printf(" Maximum number of threads per block: %d\n",deviceProp.maxThreadsPerBlock);
    printf(" Maximum sizes of each dimension of a block: %d x %d x %d\n",
    deviceProp.maxThreadsDim[],
    deviceProp.maxThreadsDim[],
    deviceProp.maxThreadsDim[]);
    printf(" Maximum sizes of each dimension of a grid: %d x %d x %d\n",
    deviceProp.maxGridSize[],
    deviceProp.maxGridSize[],
    deviceProp.maxGridSize[]);
    printf(" Maximum memory pitch: %lu bytes\n", deviceProp.memPitch);
    exit(EXIT_SUCCESS);
}                

编译运行：

$ nvcc checkDeviceInfor.cu -o checkDeviceInfor
$ ./checkDeviceInfor

输出：

./checkDeviceInfor Starting...
Detected  CUDA Capable device(s)
Device : "Tesla M2070"
CUDA Driver Version / Runtime Version 5.5 / 5.5
CUDA Capability Major/Minor version number: 2.0
Total amount of global memory: 5.25 MBytes ( bytes)
GPU Clock rate:  MHz (1.15 GHz)
Memory Clock rate:  Mhz
Memory Bus Width: -bit
L2 Cache Size:  bytes
Max Texture Dimension Size (x,y,z) 1D=(), 2D=(,), 3D=(,,)
Max Layered Texture Size (dim) x layers 1D=() x , 2D=(,) x
Total amount of constant memory:  bytes
Total amount of shared memory per block:  bytes
Total number of registers available per block:
Warp size:
Maximum number of threads per multiprocessor:
Maximum number of threads per block:
Maximum sizes of each dimension of a block:  x  x
Maximum sizes of each dimension of a grid:  x  x
Maximum memory pitch:  bytes

决定最佳GPU

对于支持多GPU的系统，是需要从中选择一个来作为我们的device的，抉择出最佳计算性能GPU的一种方法就是由其拥有的处理器数量决定，可以用下面的代码来选择最佳GPU。

int numDevices = ;
cudaGetDeviceCount(&numDevices);
if (numDevices > ) {
    int maxMultiprocessors = , maxDevice = ;
    for (int device=; device<numDevices; device++) {
        cudaDeviceProp props;
        cudaGetDeviceProperties(&props, device);
        if (maxMultiprocessors < props.multiProcessorCount) {
            maxMultiprocessors = props.multiProcessorCount;
            maxDevice = device;
        }
    }
    cudaSetDevice(maxDevice);
}    

使用nvidia-smi来查询GPU信息

nvidia-smi是一个命令行工具，可以帮助你管理操作GPU device，并且允许你查询和更改device状态。

nvidia-smi用处很多，比如，下面的指令：

$ nvidia-smi -L
GPU : Tesla M2070 (UUID: GPU-68df8aec-e85c--2b81-0c9e689a43a7)
GPU : Tesla M2070 (UUID: GPU-382f23c1--01e2--ff9628930b70)

然后可以使用下面的命令来查询GPU 0 的详细信息：

$nvidia-smi –q –i 

下面是该命令的一些参数，可以精简nvidia-smi的显示信息：

MEMORY

UTILIZATION

ECC

TEMPERATURE

POWER

CLOCK

COMPUTE

PIDS

PERFORMANCE

SUPPORTED_CLOCKS

PAGE_RETIREMENT

ACCOUNTING

比如，显示只device memory的信息：

$nvidia-smi –q –i  –d    MEMORY | tail –n
Memory Usage
Total :  MB
Used :  MB
Free :  MB

设置device

对于多GPU系统，使用nvidia-smi可以查看各GPU属性，每个GPU从0开始依次标注，使用环境变量CUDA_VISIBLE_DEVICES可以指定GPU而不用修改application。

可以设置环境变量CUDA_VISIBLE_DEVICES-2来屏蔽其他GPU，这样只有GPU2能被使用。当然也可以使用CUDA_VISIBLE_DEVICES-2,3来设置多个GPU，他们的device ID分别为0和1.

代码下载：CodeSamples.zip