谣言检测（PLAN）——《Interpretable Rumor Detection in Microblogs by Attending to User Interactions》

论文信息

论文标题：Interpretable Rumor Detection in Microblogs by Attending to User Interactions
论文作者：Ling Min Serena Khoo, Hai Leong Chieu, Zhong Qian, Jing Jiang
论文来源：2020,
论文地址：download 
论文代码：download

Background

　　基于群体智能的谣言检测：Figure 1

　　

　　本文观点：基于树结构的谣言检测模型，往往忽略了 Branch 之间的交互。

1 Introduction

　　Motivation：a user posting a reply might be replying to the entire thread rather than to a specific user.

　　Mehtod：We propose a post-level attention model (PLAN) to model long distance interactions between tweets with the multi-head attention mechanism in a transformer network.

　　We investigated variants of this model:

• • a structure aware self-attention model (StA-PLAN) that incorporates tree structure information in the transformer network;　　
  • a hierarchical token and post-level attention model (StA-HiTPLAN) that learns a sentence representation with token-level self-attention.　　

　　Contributions：

• • We utilize the attention weights from our model to provide both token-level and post-level explanations behind the model’s prediction. To the best of our knowledge, we are the first paper that has done this.　
  • We compare against previous works on two data sets - PHEME 5 events and Twitter15 and Twitter16 . Previous works only evaluated on one of the two data sets.
  • Our proposed models could outperform current state-ofthe-art models for both data sets.

　　目前谣言检测的类型：

　　　　(i) the content of the claim.

　　　　(ii) the bias and social network of the source of the claim.

　　　　(iii) fact checking with trustworthy sources.

　　　　(iv) community response to the claims.

2 Approaches

2.1 Recursive Neural Networks

　　观点：谣言传播树通常是浅层的，一个用户通常只回复一次 source post ，而后进行早期对话。

Dataset	Twitter15	Twitter16	PHEME
Tree-depth	2.80	2.77	3.12

2.2 Transformer Networks

　　Transformer 中的注意机制使有效的远程依赖关系建模成为可能。

　　Transformer 中的注意力机制：

　　　　$\alpha_{i j}=\operatorname{Compatibility}\left(q_{i}, k_{j}\right)=\operatorname{softmax}\left(\frac{q_{i} k_{j}^{T}}{\sqrt{d_{k}}}\right)\quad\quad\quad(1)$

　　　　$z_{i}=\sum_{j=1}^{n} \alpha_{i j} v_{j}\quad\quad\quad(2)$

2.3 Post-Level Attention Network (PLAN)

　　框架如下：

　　

　　首先：将 Post 按时间顺序排列；

　　其次：对每个 Post 使用 Max pool 得到 sentence embedding ；

　　然后：将 sentence embedding $X^{\prime}=\left(x_{1}^{\prime}, x_{2}^{\prime}, \ldots, x_{n}^{\prime}\right)$ 通过 $s$ 个多头注意力模块 MHA 得到 $U=\left(u_{1}, u_{2}, \ldots, u_{n}\right)$；

　　最后：通过 attention 机制聚合这些输出并使用全连接层进行预测 ：

　　　　$\begin{array}{l}\alpha_{k}=\operatorname{softmax}\left(\gamma^{T} u_{k}\right) &\quad\quad\quad(3)\\v=\sum\limits _{k=0}^{m} \alpha_{k} u_{k} &\quad\quad\quad(4)\\p=\operatorname{softmax}\left(W_{p}^{T} v+b_{p}\right) &\quad\quad\quad(5)\end{array}$

　　

　　where $\gamma \in \mathbb{R}^{d_{\text {model }}}, \alpha_{k} \in \mathbb{R}$，$W_{p} \in \mathbb{R}^{d_{\text {model }}, K}$，$b \in \mathbb{R}^{d_{\text {model }}}$，$u_{k}$  is the output after passing through  $s$  number of MHA layers，$v$  and  $p$  are the representation vector and prediction vector for  $X$

　　回顾：

　　

2.4 Structure Aware Post-Level Attention Network (StA-PLAN)

　　上述模型的问题：线性结构组织的推文容易失去结构信息。

　　为了结合显示树结构的优势和自注意力机制，本文扩展了 PLAN 模型，来包含结构信息。

　　　　$\begin{array}{l}\alpha_{i j}=\operatorname{softmax}\left(\frac{q_{i} k_{j}^{T}+a_{i j}^{K}}{\sqrt{d_{k}}}\right)\\z_{i}=\sum\limits _{j=1}^{n} \alpha_{i j}\left(v_{j}+a_{i j}^{V}\right)\end{array}$

　　其中， $a_{i j}^{V}$ 和 $a_{i j}^{K}$  是代表上述五种结构关系(i.e. parent, child, before, after and self) 的向量。

2.5 Structure Aware Hierarchical Token and Post-Level Attention Network (StA-HiTPLAN)

　　本文的PLAN 模型使用 max-pooling 来得到每条推文的句子表示，然而比较理想的方法是允许模型学习单词向量的重要性。因此，本文提出了一个层次注意模型—— attention at a token-level then at a post-level。层次结构模型的概述如 Figure 2b 所示。

　　

2.6 Time Delay Embedding

　　source post 创建的时候，reply 一般是抱持怀疑的状态，而当 source post 发布了一段时间后，reply 有着较高的趋势显示 post 是虚假的。因此，本文研究了 time delay information 对上述三种模型的影响。

　　To include time delay information for each tweet, we bin the tweets based on their latency from the time the source tweet was created. We set the total number of time bins to be 100 and each bin represents a 10 minutes interval. Tweets with latency of more than 1,000 minutes would fall into the last time bin. We used the positional encoding formula introduced in the transformer network to encode each time bin. The time delay embedding would be added to the sentence embedding of tweet. The time delay embedding, TDE, for each tweet is:

　　　　$\begin{array}{l}\mathrm{TDE}_{\text {pos }, 2 i} &=&\sin \frac{\text { pos }}{10000^{2 i / d_{\text {model }}}} \\\mathrm{TDE}_{\text {pos }, 2 i+1} &=&\cos \frac{\text { pos }}{10000^{2 i / d_{\text {model }}}}\end{array}$

　　where pos represents the time bin each tweet fall into and  $p o s \in[0,100)$, $i$  refers to the dimension and  $d_{\text {model }}$  refers to the total number of dimensions of the model.

3 Experiments and Results

dataset

　　

Result 

　　

　　

Explaining the predictions 

Post-Level Explanations

　　首先通过最后的 attention 层获得最重要的推文 $tweet_{impt}$  ，然后从第  $i$  个MHA层获得该层的与 $tweet_{impt}$  最相关的推文 $tweet _{rel,i}$  ，每篇推文可能被识别成最相关的推文多次，最后按照 被识别的次数排序，取前三名作为源推文的解释。举例如下:

　　

Token-Level Explanation

　　可以使用 token-level self-attention 的自注意力权重来进行 token-level 的解释。比如评论 “@inky mark @CP24 as part of a co-op criminal investigation one would URL doesn’t need facts to write stories it appears.”中短语“facts to write stories it appears”表达了对源推文的质疑，下图的自注意力权重图可以看出大量权重集中在这一部分，这说明这个短语就可以作为一个解释：