A3C 算法资料收集

A3C 算法资料收集

2019-07-26 21:37:55

Paper: https://arxiv.org/pdf/1602.01783.pdf

Code:

1. 超级马里奥：https://github.com/vietnguyen91/Super-mario-bros-A3C-pytorch

2. https://github.com/ikostrikov/pytorch-a3c

3. https://github.com/vy007vikas/PyTorch-ActorCriticRL

4. https://github.com/higgsfield/RL-Adventure-2

5. https://github.com/haarnoja/sac

6. Deep Reinforcement Learning, Summer 2019 (Samsung) https://github.com/dongminlee94/Samsung-DRL-Code/tree/master/3_A2C_DDPG

7.

Blog:

1. Using Keras and Deep Deterministic Policy Gradient to play TORCS: https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html

2. https://www.cnblogs.com/wangxiaocvpr/p/5681483.html

Slides:

1. https://github.com/dongminlee94/Samsung-DRL-Code/blob/master/3_A2C_DDPG/A2C.pdf

Video Tutorial：

1. Deep RL Bootcamp Lecture 4A: Policy Gradients： https://www.youtube.com/watch?v=S_gwYj1Q-44

 

=============   Super-mario-bros-A3C-pytorch 　https://github.com/vietnguyen91/Super-mario-bros-A3C-pytorch

1. 在尝试运行超级马里奥的时候，提示如下的错误：

# python3 train.py
Traceback (most recent call last):
File "train.py", line 16, in <module>
from env import create_train_env
File "/home/wangxiao/data/Super-mario-bros-A3C-pytorch/src/env.py", line 8, in <module>
from nes_py.wrappers import BinarySpaceToDiscreteSpaceEnv
ImportError: cannot import name 'BinarySpaceToDiscreteSpaceEnv' from 'nes_py.wrappers' (/root/miniconda3/lib/python3.7/site-packages/nes_py/wrappers/__init__.py)

Solution can be found at this Link: http://melonicedlatte.com/programming/2019/06/09/202000.html

2. 遇到的第二个 bug 是：

# python3 train.py
Traceback (most recent call last):
File "train.py", line 92, in <module>
train(opt)
File "train.py", line 81, in train
process.start()
File "/root/miniconda3/lib/python3.7/multiprocessing/process.py", line 112, in start
self._popen = self._Popen(self)
File "/root/miniconda3/lib/python3.7/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/root/miniconda3/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/root/miniconda3/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
self._launch(process_obj)
File "/root/miniconda3/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/root/miniconda3/lib/python3.7/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
File "/root/miniconda3/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 179, in reduce_storage
raise RuntimeError("Cannot pickle CUDA storage; try pickling a CUDA tensor instead")
RuntimeError: Cannot pickle CUDA storage; try pickling a CUDA tensor instead 

Solution: do not use PyTorch 0.4.1 version, please use 0.4.0 version or maybe later version like 1.0.1 with: conda install pytorch=0.4.0 -c pytorch

3. 成功运行：

python3 train.py
Process 0. Episode 0
Process 0. Episode 1
Process 0. Episode 2
Process 0. Episode 3
。 。 。

==