for Qbert sometimes we stay in lives == 0 condtion for a few frames —— baselines中环境包装器EpisodicLifeEnv的分析

相关：

baselines中环境包装器EpisodicLifeEnv的分析

一直不是很理解在reinforcement leanrning算法在atari游戏的observation的交互过程中对lives和episodes的判断，为什么要有lives>0的这个要求，后来发现这个游戏的实战视频，发现这个游戏在某些情况下即使lives=0的时候，也会再允许进行几个step的允许，根据下面的视频可以看到这种情况再lives=0并且同时马上要坐满所有格子的情况下游戏还是给了几个step的操作，因此这就说明了这个操作的用处了。

点击查看代码

    def step(self, action: int) -> GymStepReturn:
        obs, reward, terminated, truncated, info = self.env.step(action)
        self.was_real_done = terminated | truncated
        # check current lives, make loss of life terminal,
        # then update lives to handle bonus lives
        lives = self.env.unwrapped.ale.lives()
        if 0 < lives < self.lives:
            # for Qbert sometimes we stay in lives == 0 condtion for a few frames
            # so its important to keep lives > 0, so that we only reset once
            # the environment advertises done.
            terminated = True
        self.lives = lives
        return obs, reward, terminated, truncated, info

参考视频：

https://www.douyin.com/video/7347860006789844235