NLP（十九） 双向LSTM情感分类模型

原文链接：http://www.one2know.cn/nlp19/

• 使用IMDB情绪数据来比较CNN和RNN两种方法，预处理与上节相同

from __future__ import print_function
import numpy as np
import pandas as pd
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Dense,Dropout,Embedding,LSTM,Bidirectional
from keras.datasets import imdb
from sklearn.metrics import accuracy_score,classification_report

# 限制最大的特征数
max_features = 15000
max_len = 300
batch_size = 64

# 加载数据
(x_train,y_train),(x_test,y_test) = imdb.load_data(num_words=max_features)
print(len(x_train),'train observations')
print(len(x_test),'test observations')

输出：

Using TensorFlow backend.
25000 train observations
25000 test observations

• 如何实现
  
  1.预处理
  
  2.LSTM模型的构建和验证
  
  3.模型评估
• 代码

from __future__ import print_function
import numpy as np
import pandas as pd
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Dense,Dropout,Embedding,LSTM,Bidirectional
from keras.datasets import imdb
from sklearn.metrics import accuracy_score,classification_report

# 限制最大的特征数
max_features = 15000
max_len = 300
batch_size = 64

# 加载数据
(x_train,y_train),(x_test,y_test) = imdb.load_data(num_words=max_features)
# print(len(x_train),'train observations')
# print(len(x_test),'test observations')

# 通过序列填充将所有的数据整合为一个固定维度，提高运行效率
x_train_2 = sequence.pad_sequences(x_train,maxlen=max_len)
x_test_2 = sequence.pad_sequences(x_test,maxlen=max_len)
print('x_train_2.shape:',x_train_2.shape)
print('x_test_2.shape:',x_test_2.shape)
y_train = np.array(y_train)
y_test = np.array(y_test)

# keras框架 => 双向LSTM模型
# 双向LSTM网络有前向和后向连接，使句子中的单词可以同时与左右词汇产生连接
model = Sequential()
model.add(Embedding(max_features,128,input_length=max_len)) # 嵌入层将维数降到128
model.add(Bidirectional(LSTM(64))) # 双向LSTM层
model.add(Dropout(0.5)) # 随机失活
model.add(Dense(1,activation='sigmoid')) # 稠密层 将情感分类0或1
model.compile('adam','binary_crossentropy',metrics=['accuracy']) # 二元交叉熵
print(model.summary())

model.fit(x_train_2,y_train,batch_size=batch_size,epochs=4,validation_split=0.2)

# 预测及结果
y_train_predclass = model.predict_classes(x_train_2,batch_size=1000)
y_test_predclass = model.predict_classes(x_test_2,batch_size=1000)
y_train_predclass.shape = y_train.shape
y_test_predclass.shape = y_test.shape
print('\n\nLSTM Bidirectional Sentiment Classification - Train accuracy:',
      round(accuracy_score(y_train,y_train_predclass),3))
print('\nLSTM Bidirectional Sentiment Classification of Training data\n',
      classification_report(y_train,y_train_predclass))
print('\nLSTM Bidirectional Sentiment Classification - Train Confusion Matrix\n\n',
      pd.crosstab(y_train,y_train_predclass,rownames=['Actuall'],colnames=['Predicted']))
print('\nLSTM Bidirectional Sentiment Classification - Test accuracy:',
      round(accuracy_score(y_test,y_test_predclass),3))
print('\nLSTM Bidirectional Sentiment Classification of Test data\n',
      classification_report(y_test,y_test_predclass))
print('\nLSTM Bidirectional Sentiment Classification - Test Confusion Matrix\n\n',
      pd.crosstab(y_test,y_test_predclass,rownames=['Actuall'],colnames=['Predicted']))

输出：

Using TensorFlow backend.
x_train_2.shape: (25000, 300)
x_test_2.shape: (25000, 300)
WARNING:tensorflow:From D:\Python37\Lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From D:\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
embedding_1 (Embedding)      (None, 300, 128)          1920000
_________________________________________________________________
bidirectional_1 (Bidirection (None, 128)               98816
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 129
=================================================================
Total params: 2,018,945
Trainable params: 2,018,945
Non-trainable params: 0
_________________________________________________________________
None
WARNING:tensorflow:From D:\Python37\Lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Train on 20000 samples, validate on 5000 samples
Epoch 1/4
2019-07-07 20:03:45.649853: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

   64/20000 [..............................] - ETA: 18:21 - loss: 0.6915 - acc: 0.5781
  128/20000 [..............................] - ETA: 13:04 - loss: 0.6918 - acc: 0.5938
  192/20000 [..............................] - ETA: 11:14 - loss: 0.6915 - acc: 0.5729
  256/20000 [..............................] - ETA: 10:19 - loss: 0.6917 - acc: 0.5469
  320/20000 [..............................] - ETA: 9:45 - loss: 0.6915 - acc: 0.5469
  此处省略一堆epoch的一堆操作

LSTM Bidirectional Sentiment Classification - Train accuracy: 0.955

LSTM Bidirectional Sentiment Classification of Training data
               precision    recall  f1-score   support

           0       0.96      0.95      0.95     12500
           1       0.95      0.96      0.95     12500

    accuracy                           0.95     25000
   macro avg       0.95      0.95      0.95     25000
weighted avg       0.95      0.95      0.95     25000

LSTM Bidirectional Sentiment Classification - Train Confusion Matrix

 Predicted      0      1
Actuall
0          11928    572
1            561  11939

LSTM Bidirectional Sentiment Classification - Test accuracy: 0.859

LSTM Bidirectional Sentiment Classification of Test data
               precision    recall  f1-score   support

           0       0.86      0.86      0.86     12500
           1       0.86      0.85      0.86     12500

    accuracy                           0.86     25000
   macro avg       0.86      0.86      0.86     25000
weighted avg       0.86      0.86      0.86     25000

LSTM Bidirectional Sentiment Classification - Test Confusion Matrix

 Predicted      0      1
Actuall
0          10809   1691
1           1829  10671
time============== 2080.618681907654