[异步][jms][activeMq]怎样做到重试机制不会导致一条消息被多次运行.

版权声明：本文为博主原创文章，未经博主同意不得转载。 https://blog.csdn.net/fei33423/article/details/32723571

淘宝海量存储之单机事务面临的问题及解决的方法

http://blog.csdn.net/jiao_fuyou/article/details/15499261

这篇文章讲的比較好.核心关键词: 幂等.

Message Queue ActiveMQ: redeliver,怎样做到重试机制不会导致一条消息被多次运行.

       http://stackoverflow.com/questions/4934386/avoiding-duplicated-messages-on-jms-activemq

这里面提到了两个. 一个是可集成到Mq的camel的组件 Idempotent Consumer( Idempotent
Receiver http://camel.apache.org/idempotent-consumer.html 内含一个概念 Idempotent
Receiver  大致看了下,有 jpa. 老实说没看懂怎样实现 , 邮件组中也有人问,见以下附件),

一个是业务端实现方案(官方文档建议 http://activemq.apache.org/should-i-use-xa.html  ). 

另外一个是XA/jpa分布式事务.(http://activemq.apache.org/should-i-use-xa.html)

    答:   关键词
①消息不丢失 ,②消息不反复运行  是一个问题的两个方面. 不能顾此失彼.

            1.方案.业务端添加一张表存放,消息是否运行.每次业务事务commit后,告知server端已经处理该消息. 即使server
由于接受 timeout 重发,导致多个业务机器收到消息.也不会导致反复处理.

             想到一点,不须要去专门推断是 unique sqlException 还是其它的Exception.这个能够直接当作偶然Exception. 让Mq重试.

下次重试的时候遇到

   业务逻辑检查checkMessageExist(), 再丢弃这条消息,不再重试.

  这个方法做到了①消息不丢失
,②消息不反复运行 依赖消息反复发送.

消息不丢失,消息不反复运行.jpg

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvZmVpMzM0MjM=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="" />

            2,一种思路.让server记录.
当返回acknowledge时,进行推断是否已经运行,假设否,那么运行次数+1, 告知用户commit事务. 假设是就告知用户端,进行事务回滚.

            此方案缺陷,比如acknowledge接受成功,可是业务端事务commit失败(停电,硬盘出错),导致该消息丢失.

这个方法仅仅做到了②消息不反复运行,
没做到①消息不丢失 .某些情况下有可能消息丢失.

 

     

watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvZmVpMzM0MjM=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center" alt="" />

第二个方案里涉及了偶发性故障的问题.

       数据库硬盘坏了,停电了没有持久化成功.和持久化成功了硬盘坏了有差别,前者数据丢失,全然无法恢复,后者也有概率会丢失,可是OP会通过机制尽可能不丢失,假设真的丢失了.那不不是人能控制的.比方未做异地容灾的备份,比方地球毁灭.

       对于前者我们业务端通过"编程自己主动重试"或者"同步返回错误让使用者又一次上传"能够避免前者的消息/数据丢失.  

     编程自己主动重试对异步框架而言更加重要,上面的activeMq就是通过重试避免消息的丢失.
      后者的数据无法恢复并非通过业务端的编程能够控制的.须要Op通过某些机制尽可能的保证.

Pasted from: <http://mail-archives.apache.org/mod_mbox/activemq-users/200711.mbox/%3cec6e67fd0711050843i3bdc758ch8b801f046e96a526@mail.gmail.com%3e>

 

See the background here...

http://activemq.apache.org/should-i-use-xa.html

 

Typically you either need to implement your own duplicate message

detector, use an idempotent consumer... 幂等消费者.  核心关键

http://activemq.apache.org/camel/idempotent-consumer.html

or use XA.

 

Another approach 方法 I've used with customers is that for JPA stuff that

might fail (由于jpa可能失败), use a JPA endpoint in Camel as part of your JMS send (so

it really just does a JPA insert under the covers using your

JpaTemplate object and using the same transaction), then in the

ActiveMQ broker bridge from your entity to a queue.

 

e.g. in your code send to destination

 

new CamelDestination("jpa:com.acme.MyEntity");

 

then in your activemq.xml have a rule...

 

<route>

  <from "jpa:com.acme.MyEntity"/>

  <to "activemq:MyQueue"/>

</route>

 

This then means rather than having the XA issue; you can stick to

simple JDBC operations and transactions; then get the broker to poll

the database and bridge it to a message queue in a separate

transaction (which typically will have less risk of creating

duplicates).

 

If you're not using XA you really should have some kinda duplicate

detection in there just in case though.

 

 

On 05/11/2007, dmd <dmd17@cornell.edu> wrote:

>

> Hi all,

>

> I am using JPA (currently with OpenJPA) to persist the execution state of a

> process.  I sometimes need to use the same transaction to consume a message,

> update the state of the process and persist it to disk, and generate some

> message.

>

> Essentially I have two services which communicate via JMS and both need to

> recover from failures and not lose their state or messages in transit.

>

> Now, I can't yet figure out how to configure the system in order to share

> the transactional context...  Furthermore, I don't know if that requires

> that both the ActiveMQ broker and the JPA must use the same DB.  (Obviously

> there is a requirement that the DB itself be highly available.  We are using

> MS SQL Server with a shared storage configuration.)

>

> Any help would be appreciated, including someone telling me that I'm on

> totally the wrong track and I should use some other technologies.

>

> Cheers,

> Dan

>

> --

> View this message in context: http://www.nabble.com/ActiveMQ-and-JPA-with-global-transactions-tf4752732s2354.html#a13590338

> Sent from the ActiveMQ - User mailing list archive at Nabble.com.

>

>