单向LSTM笔记, LSTM做minist数据集分类

先介绍下torch.nn.LSTM()这个API

　　1.input_size: 每一个时步(time_step)输入到lstm单元的维度.(实际输入的数据size为[batch_size, input_size])

2. hidden_size: 确定了隐含状态hidden_state的维度. 可以简单的看成: 构造了一个权重 $\W_{input\_size*hidden\_size}$ , 隐含状态

$\h_{batch\_size*hidden\_size}=x_{batch\_size*input\_size}*W_{input\_size*hidden\_size}$

　　3 . num_layers: 叠加的层数。如图所示num_layers为 3

　　4. batch_first: 输入数据的size为[batch_size, time_step, input_size]还是[time_step, batch_size, input_size]

使用单向LSTM对MNIST进行分类,我是在pytorch0.4.1坂本上运行的。

 ########################## pytorch 用LSTM做minist数据分类 ##################

 ##########################################################################

 import torch

 import torch.utils.data as Data

 import torchvision

 import matplotlib.pyplot as plt

 import numpy as np

 BATCH_SIZE = 50

 class RNN(torch.nn.Module):

     def __init__(self):

         super().__init__()

         self.rnn = torch.nn.LSTM(

             input_size=28,

             hidden_size=64,

             num_layers=1,

             batch_first=True

         )

         self.out = torch.nn.Linear(in_features=64, out_features=10)

     def forward(self, x):

         # 一下关于shape的注释只针对单向

         # output: [batch_size, time_step, hidden_size]

         # h_n: [num_layers,batch_size, hidden_size] # 虽然LSTM的batch_first为True,但是h_n/c_n的第一维还是num_layers

         # c_n: 同h_n

         output, (h_n, c_n) = self.rnn(x)

         #print(output.size())

         # output_in_last_timestep=output[:,-1,:] # 也是可以的

         output_in_last_timestep = h_n[-1, :, :]

         # print(output_in_last_timestep.equal(output[:,-1,:])) # ture

         x = self.out(output_in_last_timestep)

         return x

 if __name__ == "__main__":

     # 1. 加载数据

     training_dataset = torchvision.datasets.MNIST("./mnist", train=True,

                                                   transform=torchvision.transforms.ToTensor(), download=True)

     dataloader = Data.DataLoader(dataset=training_dataset,

                                  batch_size=BATCH_SIZE, shuffle=True, num_workers=2)

     # showSample(dataloader)

     test_data = torchvision.datasets.MNIST(root="./mnist", train=False,

                                            transform=torchvision.transforms.ToTensor(), download=False)

     test_dataloader = Data.DataLoader(

         dataset=test_data, batch_size=1000, shuffle=False, num_workers=2)

     testdata_iter = iter(test_dataloader)

     test_x, test_y = testdata_iter.next()

     test_x = test_x.view(-1, 28, 28)

     # 2. 网络搭建

     net = RNN()

     # 3. 训练

     # 3. 网络的训练（和之前CNN训练的代码基本一样）

     optimizer = torch.optim.Adam(net.parameters(), lr=0.001)

     loss_F = torch.nn.CrossEntropyLoss()

     for epoch in range(3):  # 数据集只迭代一次

         for step, input_data in enumerate(dataloader):

             x, y = input_data

             pred = net(x.view(-1, 28, 28))

             loss = loss_F(pred,y)  # 计算loss

             optimizer.zero_grad()

             loss.backward()

             optimizer.step()

             if step % 50 == 49:  # 每50步，计算精度

                 with torch.no_grad():

                     test_pred = net(test_x)

                     prob = torch.nn.functional.softmax(test_pred, dim=1)

                     pred_cls = torch.argmax(prob, dim=1)

                     acc = (pred_cls == test_y).sum().numpy() / pred_cls.size()[0]

                     print(f"{epoch}-{step}: accuracy:{acc}")

由上面代码可以看到输出为：output,(h_n,c_n)=self.rnn(x)，解释下代码中的第28行。

output: 如果num_layer为3，则output只记录最后一层 --------- 第三层的输出。
- 对应图中向上的h_t
- 其size根据batch_first而不同。可能是[batch_size, time_step, hidden_size]或[time_step, batch_size, hidden_size]
h_n: 各个层的最后一个时步的隐含状态h.
- size为[num_layers,batch_size, hidden_size]
- 对应图中向右的h_t. 可以看出对于单层单向的LSTM，其h_n最后一层输出h_n[-1,:,:]，和output最后一个时步的输出output[:,-1,:]相等。在示例代码中print(h_n[-1,:,:].equal(output[:,-1,:]))会打印True
c_n: 各个层的最后一个时步的隐含状态C
- c_n可以看成另一个隐含状态，size和h_n相同

我运行了3个epoch效果如下：

0-49: accuracy:0.3

0-99: accuracy:0.596

0-149: accuracy:0.697

0-199: accuracy:0.734

0-249: accuracy:0.769

0-299: accuracy:0.782

0-349: accuracy:0.751

0-399: accuracy:0.843

0-449: accuracy:0.859

0-499: accuracy:0.87

0-549: accuracy:0.857

0-599: accuracy:0.89

0-649: accuracy:0.88

0-699: accuracy:0.883

0-749: accuracy:0.905

0-799: accuracy:0.905

0-849: accuracy:0.902

0-899: accuracy:0.901

0-949: accuracy:0.908

0-999: accuracy:0.921

0-1049: accuracy:0.917

0-1099: accuracy:0.906

0-1149: accuracy:0.941

0-1199: accuracy:0.935

1-49: accuracy:0.935

1-99: accuracy:0.936

1-149: accuracy:0.941

1-199: accuracy:0.923

1-249: accuracy:0.94

1-299: accuracy:0.936

1-349: accuracy:0.941

1-399: accuracy:0.948

1-449: accuracy:0.937

1-499: accuracy:0.939

1-549: accuracy:0.949

1-599: accuracy:0.949

1-649: accuracy:0.953

1-699: accuracy:0.947

1-749: accuracy:0.918

1-799: accuracy:0.944

1-849: accuracy:0.957

1-899: accuracy:0.959

1-949: accuracy:0.947

1-999: accuracy:0.944

1-1049: accuracy:0.961

1-1099: accuracy:0.964

1-1149: accuracy:0.961

1-1199: accuracy:0.952

2-49: accuracy:0.95

2-99: accuracy:0.952

2-149: accuracy:0.957

2-199: accuracy:0.945

2-249: accuracy:0.957

2-299: accuracy:0.953

2-349: accuracy:0.956

2-399: accuracy:0.942

2-449: accuracy:0.946

2-499: accuracy:0.962

2-549: accuracy:0.956

2-599: accuracy:0.957

2-649: accuracy:0.953

2-699: accuracy:0.958

2-749: accuracy:0.963

2-799: accuracy:0.959

2-849: accuracy:0.954

2-899: accuracy:0.961

2-949: accuracy:0.959

2-999: accuracy:0.961

2-1049: accuracy:0.962

2-1099: accuracy:0.958

2-1149: accuracy:0.955

2-1199: accuracy:0.964

主要参考：https://www.jianshu.com/p/043083d114d4