RNN不像传统的神经网络-它们的输出输出是固定的，而RNN允许我们输入输出向量序列。RNN是为了对序列数据进行建模而产生的。

样本序列性：样本间存在顺序关系，每个样本和它之前的样本存在关联。比如说，在文本中，一个词和它前面的词是有关联的；在气象数据中，一天的气温和前几天的气温是有关联的。

例如本帖要使用RNN生成古诗，你给它输入一堆古诗词，它会学着生成和前面相关联的字词。如果你给它输入一堆姓名，它会学着生成姓名；给它输入一堆古典乐/歌词，它会学着生成古典乐/歌词，甚至可以给它输入源代码。

关于RNN：

本帖代码移植自 char-rnn ，它是基于Torch的洋文模型，稍加修改即可应用于中文。char-rnn使用文本文件做为输入、训练RNN模型，然后使用它生成和训练数据类似的文本。

使用的数据集：全唐诗(43030首)：https://pan.baidu.com/s/1o7QlUhO

训练：

import os

import collections

import numpyas np

import tensorflowas tf

from tensorflow.python.opsimport rnn_cell

from tensorflow.python.opsimport seq2seq

import time

 

#-------------------------------数据预处理---------------------------#

 

poetry_file ='poetry.txt'

 

# 诗集

poetrys = []

with open(poetry_file, "r", encoding='utf-8',) as f:

 for linein f:

 try:

 title, content = line.strip().split(':')

 content = content.replace(' ','')

 if '_' in contentor '(' in contentor '（' in contentor '《' in contentor '[' in content:

 continue

 if len(content) < 5 or len(content) > 79:

 continue

 content = '[' + content + ']'

 poetrys.append(content)

 except Exception as e:

 pass

 

# 按诗的字数排序

poetrys = sorted(poetrys,key=lambda line: len(line))

print('唐诗总数: ', len(poetrys))

 

# 统计每个字出现次数

all_words = []

for poetryin poetrys:

 all_words += [wordfor wordin poetry]

counter = collections.Counter(all_words)

count_pairs = sorted(counter.items(), key=lambda x: -x[1])

words, _ = zip(*count_pairs)

 

# 取前多少个常用字

words = words[:len(words)] + (' ',)

# 每个字映射为一个数字ID

word_num_map = dict(zip(words, range(len(words))))

# 把诗转换为向量形式，参考TensorFlow练习1

to_num = lambda word: word_num_map.get(word, len(words))

poetrys_vector = [ list(map(to_num, poetry)) for poetryin poetrys]

#[[314, 3199, 367, 1556, 26, 179, 680, 0, 3199, 41, 506, 40, 151, 4, 98, 1],

#[339, 3, 133, 31, 302, 653, 512, 0, 37, 148, 294, 25, 54, 833, 3, 1, 965, 1315, 377, 1700, 562, 21, 37, 0, 2, 1253, 21, 36, 264, 877, 809, 1]

#....]

 

# 每次取64首诗进行训练

batch_size = 64

n_chunk = len(poetrys_vector) // batch_size

x_batches = []

y_batches = []

for i in range(n_chunk):

 start_index = i * batch_size

 end_index = start_index + batch_size

 

 batches = poetrys_vector[start_index:end_index]

 length = max(map(len,batches))

 xdata = np.full((batch_size,length), word_num_map[' '], np.int32)

 for rowin range(batch_size):

 xdata[row,:len(batches[row])] = batches[row]

 ydata = np.copy(xdata)

 ydata[:,:-1] = xdata[:,1:]

 """

xdata             ydata

[6,2,4,6,9]       [2,4,6,9,9]

[1,4,2,8,5]       [4,2,8,5,5]

"""

 x_batches.append(xdata)

 y_batches.append(ydata)

 

 

#---------------------------------------RNN--------------------------------------#

 

input_data = tf.placeholder(tf.int32, [batch_size, None])

output_targets = tf.placeholder(tf.int32, [batch_size, None])

# 定义RNN

def neural_network(model='lstm', rnn_size=128, num_layers=2):

 if model == 'rnn':

 cell_fun = rnn_cell.BasicRNNCell

 elif model == 'gru':

 cell_fun = rnn_cell.GRUCell

 elif model == 'lstm':

 cell_fun = rnn_cell.BasicLSTMCell

 

 cell = cell_fun(rnn_size, state_is_tuple=True)

 cell = rnn_cell.MultiRNNCell([cell] * num_layers, state_is_tuple=True)

 

 initial_state = cell.zero_state(batch_size, tf.float32)

 

 with tf.variable_scope('rnnlm'):

 softmax_w = tf.get_variable("softmax_w", [rnn_size, len(words)+1])

 softmax_b = tf.get_variable("softmax_b", [len(words)+1])

 with tf.device("/cpu:0"):

 embedding = tf.get_variable("embedding", [len(words)+1, rnn_size])

 inputs = tf.nn.embedding_lookup(embedding, input_data)

 

 outputs, last_state = tf.nn.dynamic_rnn(cell, inputs, initial_state=initial_state, scope='rnnlm')

 output = tf.reshape(outputs,[-1, rnn_size])

 

 logits = tf.matmul(output, softmax_w) + softmax_b

 probs = tf.nn.softmax(logits)

 return logits, last_state, probs, cell, initial_state

#训练

def train_neural_network():

 logits, last_state, _, _, _ = neural_network()

 targets = tf.reshape(output_targets, [-1])

 loss = seq2seq.sequence_loss_by_example([logits], [targets], [tf.ones_like(targets, dtype=tf.float32)], len(words))

 cost = tf.reduce_mean(loss)

 learning_rate = tf.Variable(0.0, trainable=False)

 tvars = tf.trainable_variables()

 grads, _ = tf.clip_by_global_norm(tf.gradients(cost, tvars), 5)

 optimizer = tf.train.AdamOptimizer(learning_rate)

 train_op = optimizer.apply_gradients(zip(grads, tvars))

 

 with tf.Session() as sess:

 sess.run(tf.initialize_all_variables())

 

 saver = tf.train.Saver(tf.all_variables())

 

 for epochin range(50):

 sess.run(tf.assign(learning_rate, 0.002 * (0.97 ** epoch)))

 n = 0

 for batchein range(n_chunk):

 train_loss, _ , _ = sess.run([cost, last_state, train_op], feed_dict={input_data: x_batches[n], output_targets: y_batches[n]})

 n += 1

 print(epoch, batche, train_loss)

 if epoch % 7 == 0:

 saver.save(sess, 'poetry.module', global_step=epoch)

 

train_neural_network()

使用训练好的模型生成古诗：

import os

import collections

import numpyas np

import tensorflowas tf

from tensorflow.python.opsimport rnn_cell

from tensorflow.python.opsimport seq2seq

import time

 

#-------------------------------数据预处理---------------------------#

 

poetry_file ='poetry.txt'

 

# 诗集

poetrys = []

with open(poetry_file, "r", encoding='utf-8',) as f:

 for linein f:

 try:

 title, content = line.strip().split(':')

 content = content.replace(' ','')

 if '_' in contentor '(' in contentor '（' in contentor '《' in contentor '[' in content:

 continue

 if len(content) < 5 or len(content) > 79:

 continue

 content = '[' + content + ']'

 poetrys.append(content)

 except Exception as e:

 pass

 

# 按诗的字数排序

poetrys = sorted(poetrys,key=lambda line: len(line))

print('唐诗总数: ', len(poetrys))

 

# 统计每个字出现次数

all_words = []

for poetryin poetrys:

 all_words += [wordfor wordin poetry]

counter = collections.Counter(all_words)

count_pairs = sorted(counter.items(), key=lambda x: -x[1])

words, _ = zip(*count_pairs)

 

# 取前多少个常用字

words = words[:len(words)] + (' ',)

# 每个字映射为一个数字ID

word_num_map = dict(zip(words, range(len(words))))

# 把诗转换为向量形式，参考TensorFlow练习1

to_num = lambda word: word_num_map.get(word, len(words))

poetrys_vector = [ list(map(to_num, poetry)) for poetryin poetrys]

#[[314, 3199, 367, 1556, 26, 179, 680, 0, 3199, 41, 506, 40, 151, 4, 98, 1],

#[339, 3, 133, 31, 302, 653, 512, 0, 37, 148, 294, 25, 54, 833, 3, 1, 965, 1315, 377, 1700, 562, 21, 37, 0, 2, 1253, 21, 36, 264, 877, 809, 1]

#....]

 

# 每次取64首诗进行训练

batch_size = 1

n_chunk = len(poetrys_vector) // batch_size

x_batches = []

y_batches = []

for i in range(n_chunk):

 start_index = i * batch_size

 end_index = start_index + batch_size

 

 batches = poetrys_vector[start_index:end_index]

 length = max(map(len,batches))

 xdata = np.full((batch_size,length), word_num_map[' '], np.int32)

 for rowin range(batch_size):

 xdata[row,:len(batches[row])] = batches[row]

 ydata = np.copy(xdata)

 ydata[:,:-1] = xdata[:,1:]

 """

xdata             ydata

[6,2,4,6,9]       [2,4,6,9,9]

[1,4,2,8,5]       [4,2,8,5,5]

"""

 x_batches.append(xdata)

 y_batches.append(ydata)

 

 

#---------------------------------------RNN--------------------------------------#

 

input_data = tf.placeholder(tf.int32, [batch_size, None])

output_targets = tf.placeholder(tf.int32, [batch_size, None])

# 定义RNN

def neural_network(model='lstm', rnn_size=128, num_layers=2):

 if model == 'rnn':

 cell_fun = rnn_cell.BasicRNNCell

 elif model == 'gru':

 cell_fun = rnn_cell.GRUCell

 elif model == 'lstm':

 cell_fun = rnn_cell.BasicLSTMCell

 

 cell = cell_fun(rnn_size, state_is_tuple=True)

 cell = rnn_cell.MultiRNNCell([cell] * num_layers, state_is_tuple=True)

 

 initial_state = cell.zero_state(batch_size, tf.float32)

 

 with tf.variable_scope('rnnlm'):

 softmax_w = tf.get_variable("softmax_w", [rnn_size, len(words)+1])

 softmax_b = tf.get_variable("softmax_b", [len(words)+1])

 with tf.device("/cpu:0"):

 embedding = tf.get_variable("embedding", [len(words)+1, rnn_size])

 inputs = tf.nn.embedding_lookup(embedding, input_data)

 

 outputs, last_state = tf.nn.dynamic_rnn(cell, inputs, initial_state=initial_state, scope='rnnlm')

 output = tf.reshape(outputs,[-1, rnn_size])

 

 logits = tf.matmul(output, softmax_w) + softmax_b

 probs = tf.nn.softmax(logits)

 return logits, last_state, probs, cell, initial_state

 

#-------------------------------生成古诗---------------------------------#

# 使用训练完成的模型

 

def gen_poetry():

 def to_word(weights):

 t = np.cumsum(weights)

 s = np.sum(weights)

 sample = int(np.searchsorted(t, np.random.rand(1)*s))

 return words[sample]

 

 _, last_state, probs, cell, initial_state = neural_network()

 

 with tf.Session() as sess:

 sess.run(tf.initialize_all_variables())

 

 saver = tf.train.Saver(tf.all_variables())

 saver.restore(sess, 'poetry.module-49')

 

 state_ = sess.run(cell.zero_state(1, tf.float32))

 

 x = np.array([list(map(word_num_map.get, '['))])

 [probs_, state_] = sess.run([probs, last_state], feed_dict={input_data: x, initial_state: state_})

 word = to_word(probs_)

 #word = words[np.argmax(probs_)]

 poem = ''

 while word != ']':

 poem += word

 x = np.zeros((1,1))

 x[0,0] = word_num_map[word]

 [probs_, state_] = sess.run([probs, last_state], feed_dict={input_data: x, initial_state: state_})

 word = to_word(probs_)

 #word = words[np.argmax(probs_)]

 return poem

 

print(gen_poetry())

生成的“诗词”（至少格式对上了）：

新犬随风起，一璃迹阵悲。

浅昏罢庄哉，清插去园空。

双叶坐成鉴，王妓水正苑。

鸟声不成影，胙滩朱瓮声。

无斑红芜踏，那期日正闲。

吾燕登无士，无处得赵名。

生成藏头诗：

def gen_poetry_with_head(head):

 def to_word(weights):

 t = np.cumsum(weights)

 s = np.sum(weights)

 sample = int(np.searchsorted(t, np.random.rand(1)*s))

 return words[sample]

 

 _, last_state, probs, cell, initial_state = neural_network()

 

 with tf.Session() as sess:

 sess.run(tf.initialize_all_variables())

 

 saver = tf.train.Saver(tf.all_variables())

 saver.restore(sess, 'poetry.module-7')

 

 state_ = sess.run(cell.zero_state(1, tf.float32))

 poem = ''

 i = 0

 for wordin head:

 while word != '，' and word != '。':

 poem += word

 x = np.array([list(map(word_num_map.get, word))])

 [probs_, state_] = sess.run([probs, last_state], feed_dict={input_data: x, initial_state: state_})

 word = to_word(probs_)

 time.sleep(1)

 if i % 2 == 0:

 poem += '，'

 else:

 poem += '。'

 i += 1

 return poem

 

print(gen_poetry_with_head('一二三四'))

Share the post "TensorFlow练习7: 基于RNN生成古诗词"

TensorFlow练习7: 基于RNN生成古诗词的更多相关文章

  1. TensorFlow教程使用RNN生成唐诗

    本教程转载至:TensorFlow练习7: 基于RNN生成古诗词 使用的数据集是全唐诗,首先提供一下数据集的下载链接:https://pan.baidu.com/s/13pNWfffr5HSN79WN ...

  2. 基于TensorFlow的循环神经网络(RNN)

    RNN适用场景 循环神经网络(Recurrent Neural Network)适合处理和预测时序数据 RNN的特点 RNN的隐藏层之间的节点是有连接的,他的输入是输入层的输出向量.extend(上一 ...

  3. “你什么意思”之基于RNN的语义槽填充(Pytorch实现)

    1. 概况 1.1 任务 口语理解(Spoken Language Understanding, SLU)作为语音识别与自然语言处理之间的一个新兴领域,其目的是为了让计算机从用户的讲话中理解他们的意图 ...

  4. 学习Tensorflow的LSTM的RNN例子

    学习Tensorflow的LSTM的RNN例子 基于TensorFlow一次简单的RNN实现 极客学院-递归神经网络 如何使用TensorFlow构建.训练和改进循环神经网络

  5. 深度学习实战篇-基于RNN的中文分词探索

    深度学习实战篇-基于RNN的中文分词探索 近年来,深度学习在人工智能的多个领域取得了显著成绩.微软使用的152层深度神经网络在ImageNet的比赛上斩获多项第一,同时在图像识别中超过了人类的识别水平 ...

  6. Linux下c基于openssl生成MD5的函数

    Linux下openssl提供了一系列哈希及加密的函数,如果调用openssl提供的MD5函数生成任意字符串的MD5呢?下面提供了一段代码实现Linux下c字符串生成md5的函数. 具体代码: 1 2 ...

  7. 基于RNN的音频降噪算法 (附完整C代码)

    前几天无意间看到一个项目rnnoise. 项目地址: https://github.com/xiph/rnnoise 基于RNN的音频降噪算法. 采用的是 GRU/LSTM 模型. 阅读下训练代码,可 ...

  8. kbmMW基于硬件生成随机数

    按作者的说法,Delphi提供的生成随机数不是真正随机的,因为他是根据种子计算的,即种子+算法生成的随机数,如果被人知道原始种子值和算法的调用次数,则可以重现随机数,因此在安全领域,这是不安全的.同时 ...

  9. 根据wsdl，基于wsimport生成代码的客户端

    根据wsdl,基于wsimport生成代码的客户端 wsimport是jdk自带的命令,可以根据wsdl文档生成客户端中间代码,基于生成的代码编写客户端,可以省很多麻烦. 局限性:wsimport   ...

随机推荐

  1. Maven 项目不打包 *.hbm.xml 映射文件

    使用 Maven 部署 Java Web 项目时,Hibernate 的映射文件 *.hbm.xml 没有被打包部署到目标目录下,解决方法:在 pom.xml 文件中 <build> 节点 ...

  2. 【转】Mapped Statements collection does not contain value for解决

    最近一直在弄springMVC+mybatis的整合,因为接触到这个框架之后发现这个框架确实要比ssh好得多所以我自己也在配置这个框架.但是在配置的过程中我遇到了一些问题,这些问题当我配置完成之后访问 ...

  3. JavaScript基础之运算符及全面的运算符优先级总结

    算数运算符: 加+,减—,乘*,除/,求余%,加加++,减减——, 加减乘除求余运算与数学上的用法完全一样. 不过,加号+还有连接字符串的作用,其他运算符还可以将字符串数字转换成数值型,参见JavaS ...

  4. request.getRequestDispatcher("").forward()中文乱码

    即使jsp页面的编码已设为“UTF-8”,有中文的地方还是会出现乱码,但是用response.sendRedirect不会出现此问题. 解决方案一: 不使用PrintWriter out=respon ...

  5. 【图像处理】基于OpenCV底层实现的直方图匹配

    image processing 系列: [图像处理]图片旋转 [图像处理]高斯滤波.中值滤波.均值滤波 直方图匹配算法.又称直方图规定化.简单说.就是依据某函数.或者另外一张图片的引导,使得原图改变 ...

  6. [Go] sync.Once 的用法

    sync.Once.Do(f func()) 是一个非常有意思的东西,能保证 once 只执行一次,无论你是否更换 once.Do(xx) 这里的方法,这个 sync.Once块 只会执行一次. pa ...

  7. Linux Shell脚本入门－－wc命令

    wc 统计文件里面有多少单词,多少行,多少字符. wc语法 [root@www ~]# wc [-lwm] 选项与参数: -l :仅列出行: -w :仅列出多少字(英文单字): -m :多少字符: 默 ...

  8. linux 内核升级2 转

    linux内核升级 一.Linux内核概览 Linux是一个一体化内核(monolithic kernel)系统. 设备驱动程序可以完全访问硬件. Linux内的设备驱动程序可以方便地以模块化(mod ...

  9. Mustache.js语法

    看了Mustache的github,学学此中的语法,做个笔记 1.简单的变量调换:{{name}} 1 var data = { "name": "Willy" ...

  10. Odoo(OpenERP)配置文件openerp-server.conf详解

    原文地址:http://blog.csdn.net/wangnan537/article/details/42283465 [options] ; addons模块的查找路径 addons_path ...