PyTorch之初级使用

• 使用流程
  ①. 数据准备; ②. 模型确立; ③. 损失函数确立; ④. 优化器确立; ⑤. 模型训练及保存
• 模块介绍
  Part Ⅰ: 数据准备
  torch.utils.data.Dataset 
  torch.utils.data.DataLoader 
  关于Dataset, 作为数据集, 需要实现基本的3个方法, 分别为: __init__、__len__、__getitem__. 示例如下,
  

   1 class TrainingDataset(Dataset):
   2
   3     def __init__(self, X, Y_, transform=None, target_transform=None):
   4         self.__X = X
   5         self.__Y_ = Y_
   6         self.__transform = transform
   7         self.__target_transform = target_transform
   8
   9
  10     def __len__(self):
  11         return len(self.__X)
  12
  13
  14     def __getitem__(self, idx):
  15         x = self.__X[idx]
  16         y_ = self.__Y_[idx]
  17         if self.__transform:
  18             x = self.__transform(x)
  19         if self.__target_transform:
  20             y_ = self.__target_transform(y_)
  21         return x, y_

  关于DataLoader, 作为数据集封装, 将数据集Dataset封装为可迭代对象. 示例如下,

  1 batch_size = 100
  2 trainingLoader = DataLoader(trainingData, batch_size=batch_size, shuffle=True)

  Part Ⅱ: 模型确立
  torch.nn 
  torch.nn.Module 
  网络模型由基类Module派生, 内部所有操作模块均由命名空间nn提供, 需要实现基本的2个方法, 分别为: __init__、forward. 其中, __init__方法定义操作, forward方法运用操作进行正向计算. 示例如下,

   1 class NeuralNetwork(nn.Module):
   2
   3     def __init__(self):
   4         super(NeuralNetwork, self).__init__()
   5         self.__linear_tanh_stack = nn.Sequential(
   6             nn.Linear(3, 5),
   7             nn.Tanh(),
   8             nn.Linear(5, 3)
   9         )
  10
  11
  12     def forward(self, x):
  13         y = self.__linear_tanh_stack(x)
  14         return y
  15
  16
  17 model = NeuralNetwork()

  Part Ⅲ: 损失函数确立
  torch.nn 
  常见损失函数有: nn.MSELoss(回归任务)、nn.CrossEntropyLoss(多分类任务)等. 示例如下,

  1 loss_func = nn.MSELoss(reduction="sum")

  Part Ⅳ: 优化器确立
  torch.optim 
  常见的优化器有: optim.SGD、optim.Adam等. 示例如下,

  1 optimizer = optim.Adam(model.parameters(), lr=0.001)

  Part Ⅴ: 模型训练及保存
  有效整合数据、模型、损失函数及优化器. 注意, 模型参数之梯度默认累积, 每次参数优化需要显式清零. 示例如下,

   1 def train_loop(dataloader, model, loss_func, optimizer):
   2     for batchIdx, (X, Y_) in enumerate(dataloader):
   3         Y = model(X)
   4         loss = loss_func(Y, Y_)
   5
   6         optimizer.zero_grad()
   7         loss.backward()
   8         optimizer.step()
   9
  10
  11 epoch = 50000
  12 for epochIdx in range(epoch):
  13     train_loop(trainingLoader, model, loss_func, optimizer)
  14
  15
  16 torch.save(model.state_dict(), "model_params.pth")

• 代码实现
  本文使用与Back Propagation - Python实现相同的网络架构及数据生成策略, 分别如下所示,
  

  $$
  \begin{equation*}
  \left\{
  \begin{split}
  x &= r + 2g + 3b \\
  y &= r^2 + 2g^2 + 3b^2 \\
  lv &= -3r - 4g - 5b
  \end{split}
  \right.
  \end{equation*} 
  $$
  具体实现如下,

  

    1 import numpy
    2 import torch
    3 from torch import nn
    4 from torch import optim
    5 from torch.utils.data import Dataset, DataLoader
    6 from matplotlib import pyplot as plt
    7
    8
    9 numpy.random.seed(1)
   10 torch.manual_seed(3)
   11
   12
   13 # 生成training数据
   14 def getData(n=100):
   15     rgbRange = (-1, 1)
   16     r = numpy.random.uniform(*rgbRange, (n, 1))
   17     g = numpy.random.uniform(*rgbRange, (n, 1))
   18     b = numpy.random.uniform(*rgbRange, (n, 1))
   19     x_ = r + 2 * g + 3 * b
   20     y_ = r ** 2 + 2 * g ** 2 + 3 * b ** 2
   21     lv_ = -3 * r - 4 * g - 5 * b
   22     RGB = numpy.hstack((r, g, b))
   23     XYLv_ = numpy.hstack((x_, y_, lv_))
   24     return RGB, XYLv_
   25
   26
   27 class TrainingDataset(Dataset):
   28
   29     def __init__(self, X, Y_, transform=None, target_transform=None):
   30         self.__X = X
   31         self.__Y_ = Y_
   32         self.__transform = transform
   33         self.__target_transform = target_transform
   34
   35
   36     def __len__(self):
   37         return len(self.__X)
   38
   39
   40     def __getitem__(self, idx):
   41         x = self.__X[idx]
   42         y_ = self.__Y_[idx]
   43         if self.__transform:
   44             x = self.__transform(x)
   45         if self.__target_transform:
   46             y_ = self.__target_transform(y_)
   47         return x, y_
   48
   49
   50 RGB, XYLv_ = getData(1000)
   51 trainingData = TrainingDataset(RGB, XYLv_, torch.Tensor, torch.Tensor)
   52
   53 batch_size = 100
   54 trainingLoader = DataLoader(trainingData, batch_size=batch_size, shuffle=True)
   55
   56
   57 class NeuralNetwork(nn.Module):
   58
   59     def __init__(self):
   60         super(NeuralNetwork, self).__init__()
   61         self.__linear_tanh_stack = nn.Sequential(
   62             nn.Linear(3, 5),
   63             nn.Tanh(),
   64             nn.Linear(5, 3)
   65         )
   66
   67
   68     def forward(self, x):
   69         y = self.__linear_tanh_stack(x)
   70         return y
   71
   72
   73 model = NeuralNetwork()
   74 loss_func = nn.MSELoss(reduction="sum")
   75 optimizer = optim.Adam(model.parameters(), lr=0.001)
   76
   77
   78 def train_loop(dataloader, model, loss_func, optimizer):
   79     JVal = 0
   80     for batchIdx, (X, Y_) in enumerate(dataloader):
   81         Y = model(X)
   82         loss = loss_func(Y, Y_)
   83
   84         JVal += loss.item()
   85
   86         optimizer.zero_grad()
   87         loss.backward()
   88         optimizer.step()
   89
   90     JVal /= 2
   91     return JVal
   92
   93
   94 JPath = list()
   95 epoch = 50000
   96 for epochIdx in range(epoch):
   97     JVal = train_loop(trainingLoader, model, loss_func, optimizer)
   98     print("epoch: {:5d},   JVal = {:.5f}".format(epochIdx, JVal))
   99     JPath.append(JVal)
  100
  101
  102 torch.save(model.state_dict(), "model_params.pth")
  103
  104
  105 fig = plt.figure(figsize=(6, 4))
  106 ax1 = fig.add_subplot(1, 1, 1)
  107
  108 ax1.plot(numpy.arange(len(JPath)), JPath, "k.", markersize=1)
  109 ax1.plot(0, JPath[0], "go", label="seed")
  110 ax1.plot(len(JPath)-1, JPath[-1], "r*", label="solution")
  111
  112 ax1.legend()
  113 ax1.set(xlabel="$epoch$", ylabel="$JVal$", title="solution-JVal = {:.5f}".format(JPath[-1]))
  114
  115 fig.tight_layout()
  116 fig.savefig("plot_fig.png", dpi=100)

• 结果展示
  可以看到, 在training data上总体loss随epoch增加逐渐降低.

• 使用建议
  ①. 分batch处理训练数据, 可以提升训练初始阶段模型参数收敛速度;
  ②. 常规优化器推荐Adam, 具备自动步长调节的能力.

• 参考文档
  ①. https://pytorch.org/tutorials/beginner/basics/intro.html