3D Gaussian 三维视觉重建

论文资料

• 论文 https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/3d_gaussian_splatting_low.pdf
  • 资料网站 https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
  • GitHub https://github.com/graphdeco-inria/gaussian-splatting
• 相关论文
  • https://arxiv.org/html/2406.18533v1
    
    On Scaling Up 3D Gaussian Splatting Training

扩展阅读

• https://zhuanlan.zhihu.com/p/680669616
  
  这篇讲的比较详细易懂
• https://www.magnopus.com/blog/the-rise-of-3d-gaussian-splatting
• https://metaschool.so/articles/gaussian-splatting
• Animatable Gaussian Avatar https://github.com/heawon-yoon/anim-gaussian
  
  使用3D高斯制作3D人物动画头像, 包含安装步骤

视频

合集 https://www.youtube.com/playlist?list=PLAGyKNXhhw2l-OlYOvCxyQ2l29rypwIjW

• 3D高斯介绍 3D Gaussian Splatting for Beginners
• 模型数据说明 Understanding the Gaussian splatting model
• 3D NeRF(另一种实现)和高斯的实际应用 Real World Applications for NeRFs and Gaussian Splatting - Simulation, Real Estate, Cinema, AR, VR!
• 3D NeRF的介绍 NeRF: Neural Radiance Fields for Beginners
• 3D NeRF和高斯的对比 Novel View Rendering and 3D Reconstruction - NeRFs vs Gaussian Splatting
• 在Linux环境下的安装, 训练和渲染 3D Gaussian Splatting in Linux - Setup, Training and Rendering

其它

• Creating 3D Game Models from Video using Photogrammetry https://www.youtube.com/watch?v=bDHJM6nAKtc
  
  7年前的3D重建技术
• 3D Gaussian Splatting Demo https://www.youtube.com/watch?v=c0VNckM21B0
  
  用iPhone XR拍摄后, 使用3D高斯重建的效果演示

B站中文

• https://www.bilibili.com/video/BV11e411n79b
  
  3D高斯介绍
• https://www.bilibili.com/video/BV1bJ4m1b7qW/
  
  3D高斯与其他两种重建方法的效果对比

安装步骤

安装必要的库和依赖

Ubuntu

apt update
apt install build-essential ninja-build

查看显卡

(base) root@ubuntu22:~# nvidia-smi
Sun Mar 30 17:21:08 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.78                 Driver Version: 550.78         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080        Off |   00000000:00:08.0 Off |                  N/A |
| 30%   23C    P8              8W /  320W |      10MiB /  10240MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A       709      G   /usr/lib/xorg/Xorg                              4MiB |
+-----------------------------------------------------------------------------------------+

Cuda

安装 Cuda Toolkit

https://developer.nvidia.com/cuda-toolkit-archive

查看cuda版本

nvcc -V

Cuda 12.4

https://developer.nvidia.com/cuda-12-4-1-download-archive

这个安装遇到问题

wget https://developer.download.nvidia.com/compute/cuda/12.4.1/local_installers/cuda_12.4.1_550.54.15_linux.run
sudo sh cuda_12.4.1_550.54.15_linux.run

这个安装通过

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.4.1/local_installers/cuda-repo-ubuntu2204-12-4-local_12.4.1-550.54.15-1_amd64.deb
dpkg -i cuda-repo-ubuntu2204-12-4-local_12.4.1-550.54.15-1_amd64.deb
cp /var/cuda-repo-ubuntu2204-12-4-local/cuda-*-keyring.gpg /usr/share/keyrings/
apt update
apt install cuda-toolkit-12-4

未尝试

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-4

检查

(base) root@ubuntu22:~/Download# /usr/local/cuda/bin/nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

Cuda 12.1

https://developer.nvidia.com/cuda-12-1-1-download-archive

方式一: 使用run文件, 编译出错

# 这个文件有将近4.5GB, 需要增加带宽, 计划好时间.
wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda_12.1.1_530.30.02_linux.run
sh cuda_12.1.1_530.30.02_linux.run

方式二: apt deb方式安装, 最后提示版本不一致

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-1-local_12.1.1-530.30.02-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

Conda

安装 Miniconda, 安装说明: https://www.cnblogs.com/milton/p/18023969

启用conda

eval "$(/home/milton/miniconda3/bin/conda shell.bash hook)"

创建一个conda环境, python版本为 3.10.12

conda create --name test001 python=3.10.12
conda activate test001

Pytorch

访问 Pytorch 官网 https://pytorch.org/

从 2.6 开始, conda 方式安装不再可用, 并且 2.6 只支持 11.8 和 12.4, 如果要用 12.1, 要换成 2.5

Cuda 12.6 安装 Pytorch 2.6

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

Cuda 12.4 安装 Pytorch 2.6

pip3 install torch torchvision torchaudio

Cuda 12.1 安装 Pytorch 2.5.1

Conda

# CUDA 11.8
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1  pytorch-cuda=11.8 -c pytorch -c nvidia
# CUDA 12.1
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia
# CUDA 12.4
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia

Pip

# CUDA 11.8
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu118
# CUDA 12.1
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu121
# CUDA 12.4
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

其它

pip install plyfile tqdm tensorboard six
pip install opencv-python

导出项目

导出项目仓库

git clone https://github.com/graphdeco-inria/gaussian-splatting --recursive

安装模块

#gaussian
pip install submodules/diff-gaussian-rasterization
pip install submodules/simple-knn

下载训练素材

在项目 GitHub 仓库 的首页找到 Running 部分, 能找到这个下载链接

You can find our SfM data sets for Tanks&Temples and Deep Blending here:

https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/input/tandt_db.zip

然后有说明输出目录 If you do not provide an output model directory (-m), trained models are written to folders with randomized unique names inside the output directory. At this point, the trained models may be viewed with the real-time viewer (see further below).

默认数据集测试

训练: 使用项目自带的数据

python train.py -s [素材路径]
# e.g.
python train.py -s ./data/tandt/truck

执行过程中可以查看GPU的情况

(base) root@ubuntu22:~/WorkPython# nvidia-smi
Sun Mar 30 17:59:38 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.78                 Driver Version: 550.78         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080        Off |   00000000:00:08.0 Off |                  N/A |
| 61%   69C    P2            308W /  320W |    5499MiB /  10240MiB |     97%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A       709      G   /usr/lib/xorg/Xorg                              4MiB |
|    0   N/A  N/A      7040      C   python                                       5484MiB |
+-----------------------------------------------------------------------------------------+

渲染:

python render.py -m [训练结果路径]

python metrics.py -m <path to trained model> # Compute error metrics on renderings

结果查看

下载模型文件后, 通过这个网站查看

https://poly.cam/tools/gaussian-splatting

问题处理

安装 CUDA 失败

(base) root@ubuntu22:~/Download# sudo sh cuda_12.1.1_530.30.02_linux.run
 Installation failed. See log at /var/log/cuda-installer.log for details.
(base) root@ubuntu22:~/Download# more /var/log/cuda-installer.log
(test001) root@ubuntu22:~/Download# more /var/log/nvidia-installer.log
...
Using built-in stream user interface
-> Detected 8 CPUs online; setting concurrency level to 8.
-> Scanning the initramfs with lsinitramfs...
-> Executing: /usr/bin/lsinitramfs   -l /boot/initrd.img-6.5.0-28-generic
-> The file '/tmp/.X0-lock' exists and appears to contain the process ID '737' of a running X server.
-> You appear to be running an X server.  Installing the NVIDIA driver while X is running is not recommended, as doing so may prevent the
 installer from detecting some potential installation problems, and it may not be possible to start new graphics applications after a new
 driver is installed.  If you choose to continue installation, it is highly recommended that you reboot your computer after installation
to use the newly installed driver. (Answer: Abort installation)
ERROR: Installation has failed.  Please see the file '/var/log/nvidia-installer.log' for details.  You may find suggestions on fixing ins
tallation problems in the README available on the Linux driver download page at www.nvidia.com.

停掉X Server后再安装

systemctl stop display-manager

we could not find ninja or g++

sudo apt-get update
sudo apt install build-essential
sudo apt-get install ninja-build

No such file or directory: ‘:/usr/local/cuda-11.8/bin/nvcc

Execute the command directly on the current command line

export CUDA_HOME=/usr/local/cuda

install again

pip install submodules/diff-gaussian-rasterization
pip install submodules/simple-knn