Allowing GPU memory growth
By default, TensorFlow maps nearly all of the GPU memory of all GPUs (subject to CUDA_VISIBLE_DEVICES
) visible to the process. This is done to more efficiently use the relatively precious GPU memory resources on the devices by reducing memory fragmentation.
In some cases it is desirable for the process to only allocate a subset of the available memory, or to only grow the memory usage as is needed by the process. TensorFlow provides two Config options on the Session to control this.
The first is the allow_growth
option, which attempts to allocate only as much GPU memory based on runtime allocations: it starts out allocating very little memory, and as Sessions get run and more GPU memory is needed, we extend the GPU memory region needed by the TensorFlow process. Note that we do not release memory, since that can lead to even worse memory fragmentation. To turn this option on, set the option in the ConfigProto by:
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.Session(config=config, ...)
The second method is the per_process_gpu_memory_fraction
option, which determines the fraction of the overall amount of memory that each visible GPU should be allocated. For example, you can tell TensorFlow to only allocate 40% of the total memory of each GPU by:
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.4
session = tf.Session(config=config, ...)
This is useful if you want to truly bound the amount of GPU memory available to the TensorFlow process.
Allowing GPU memory growth的更多相关文章
- Reducing and Profiling GPU Memory Usage in Keras with TensorFlow Backend
keras 自适应分配显存 & 清理不用的变量释放 GPU 显存 Intro Are you running out of GPU memory when using keras or ten ...
- GPU Memory Usage占满而GPU-Util却为0的调试
最近使用github上的一个开源项目训练基于CNN的翻译模型,使用THEANO_FLAGS='floatX=float32,device=gpu2,lib.cnmem=1' python run_nn ...
- 重置GPU显存 Reset GPU memory after CUDA errors
Sometimes CUDA program crashed during execution, before memory was flushed. As a result, device memo ...
- tensorflow 运行效率 GPU memory leak 问题解决
问题描述: Tensorflow 训练时运行越来越慢,重启后又变好. 用的是Tensorflow-GPU 1.2版本,在GPU上跑,大概就是才开始训练的时候每个batch的时间很低,然后随着训练的推进 ...
- Jupyter notebook Tensorflow GPU Memory 释放
Jupyter notebook 每次运行完tensorflow的程序,占着显存不释放.而又因为tensorflow是默认申请可使用的全部显存,就会使得后续程序难以运行.暂时还没有找到在jupyter ...
- Gradient Boosting, Decision Trees and XGBoost with CUDA ——GPU加速5-6倍
xgboost的可以参考:https://xgboost.readthedocs.io/en/latest/gpu/index.html 整体看加速5-6倍的样子. Gradient Boosting ...
- NVIDIA GPU Pascal架构简述
NVIDIA GPU Pascal架构简述 本文摘抄自英伟达Pascal架构官方白皮书:https://www.nvidia.com/en-us/data-center/resources/pasca ...
- Tensorflow2对GPU内存的分配策略
一.问题源起 从以下的异常堆栈可以看到是BLAS程序集初始化失败,可以看到是执行MatMul的时候发生的异常,基本可以断定可能数据集太大导致memory不够用了. 2021-08-10 16:38:0 ...
- GPU keylogger && GPU Based rootkit(Jellyfish rootkit)
catalog . OpenCL . Linux DMA(Direct Memory Access) . GPU rootkit PoC by Team Jellyfish . GPU keylogg ...
随机推荐
- pycharm中查找替换妙用
1.二行空格变一行(转载https://www.cnblogs.com/dreamfine/p/7760575.html) 网上COPY的代码,经常多出一个空行,不用一行行删除了,用替换功能吧,查找 ...
- kubernetes国内镜像拉取
因国内访问不到goole服务器,只能拉取国内的镜像,这里以阿里云为例. 安装minikube时报failed to pull image "k8s.gcr.io/kube-apiserver ...
- JS 函数(arguments、箭头函数、bind)
参数 函数内部可用的 arguments 对象来访问函数的实参 注意 在函数递归调用的时候(在某一刻同一个函数运行了多次,也就是有多套实参),那么 arguments 属性的值是最近一次该函数调用时传 ...
- 在keil调用Notepad++
先打开keil, 新建一个 取名为notepad 选择notepad++的安装路径 设置参数 保持后可以看多了notepad的选项 运行当前的文件在notepad++打开
- 用JS 和 jQery获取屏幕的高度和宽度
用的时候,网上找了下,放在一起,方便以后查阅 document.body.clientWidth document.body.offsetWidth(包括线宽)//网页可见区域宽 document.b ...
- Django - session 会话跟踪技术
1.session简介 |session 英 /'seʃ(ə)n/ 美 /'sɛʃən/ 基于cookies开发,将值存到服务端 写session 读session Session是服务器端技术,利用 ...
- 我要重新开始学习C++了!
C++实在是博大精深!之前总不想读厚厚的C++ Primer. 然而,现在的水平真的只是初学者!只是因为写的代码太简单,所以没有用到一些特性.可以说还是门外汉! 写笔记!
- jquery接触初级----jquery 选择器
css 选择器主要有:元素选择器,ID选择器,类选择器,群组选择器,后代选择器,普通配符选择器等,通过css选择,我们可以很方便的给元素添加样式,使网页看起来更加好看 jquery 选择器也有相似的功 ...
- linux 2.6.32文件系统的inode
接上文: crash> struct -xo dentry.d_inode ffff8818118002c0 struct dentry { [ffff8818118002d0] struct ...
- Redis Cluster 添加/删除 完整折腾步骤
Redis还是挺好玩的,今天测试了集群的添加.删除节点.重分配slot等.更深入的理解redis的游戏规则.步骤繁多,但是详细. 环境解释: 我是在一台Centos 6.9上测试的,各个redis节点 ...