ActiveMQ学习系列(四)----消息持久化到mysql
前记:目前学习还比较杂乱,还未找到系统化地学习ActiveMq的方法。在网上看到消息持久化的demo,了解了一下,在此记录。
一、目前ActiveMq支持的持久化方法
url:http://activemq.apache.org/persistence.html
3、KahaDB
4、JDBC 配合其自带的 high performance journal;根据官方说法,它内置的高性能journal的工作类似于在缓存层工作,消息会优先写入到journal,后台的定时任务会每隔一段时间间隔去
查看需要写入到jdbc的消息。
二、配置activemq.xml
1、修改persistenceAdapter
<persistenceAdapter>
<!-- <kahaDB directory="${activemq.data}/kahadb"/>-->
<jdbcPersistenceAdapter dataSource="#my-ds"/>
</persistenceAdapter>
上面我们注释了默认的kahadb,添加了jdbc数据源。
2、增加数据源
<bean id="my-ds" class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://192.168.2.140:3306/activemq?characterEncoding=utf-8" />
<property name="username" value="root" />
<property name="password" value="" />
<property name="initialSize" value="" />
<property name="maxTotal" value="" />
<property name="maxIdle" value="" />
<property name="maxWaitMillis" value="" />
<property name="minIdle" value="" />
</bean>
这边有几个注意点:
2.1:上面配置的数据源类对应于dbcp2,如果1.x系列的jar包会报错,classNotFound异常。
2.2:在主安装目录的lib目录下,将mysql的jar包、dbcp的jar包放到该路径下
/usr/local/apache-activemq-5.14./lib
2.3:建议用bin/activemq console方式启动,可以及时查看错误信息。我在启动时,报以下错误:
WARN | Failure Details: Table 'xckk_star_act.ACTIVEMQ_ACKS' doesn't exist
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xckk_star_act.ACTIVEMQ_ACKS' doesn't exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)[:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)[:1.8.0_121]
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.Util.getInstance(Util.java:387)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:939)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2009)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5094)[mysql-connector-java-5.1.38.jar:5.1.38]
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1994)[mysql-connector-java-5.1.38.jar:5.1.38]
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)[commons-dbcp2-2.1.1.jar:2.1.1]
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)[commons-dbcp2-2.1.1.jar:2.1.1]
at org.apache.activemq.store.jdbc.adapter.DefaultJDBCAdapter.doDeleteOldMessages(DefaultJDBCAdapter.java:832)[activemq-jdbc-store-5.14.4.jar:5.14.4]
at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.cleanup(JDBCPersistenceAdapter.java:349)[activemq-jdbc-store-5.14.4.jar:5.14.4]
at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:327)[activemq-jdbc-store-5.14.4.jar:5.14.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_121]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)[:1.8.0_121]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)[:1.8.0_121]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)[:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_121]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
在网上查询结果,原来是数据库需要字符集设置为latin1.
于是,新建了一个数据库,字符集设为Latin1.然后启动ActiveMq,查看数据库,发现多了三张表:
在这边将结构导出来,供手动建表:
/*
SQLyog v10.2
MySQL - 5.1.71 : Database - activemq
*********************************************************************
*/ /*!40101 SET NAMES utf8 */; /*!40101 SET SQL_MODE=''*/; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
CREATE DATABASE /*!32312 IF NOT EXISTS*/`activemq` /*!40100 DEFAULT CHARACTER SET latin1 COLLATE latin1_bin */; USE `activemq`; /*Table structure for table `ACTIVEMQ_ACKS` */ DROP TABLE IF EXISTS `ACTIVEMQ_ACKS`; CREATE TABLE `ACTIVEMQ_ACKS` (
`CONTAINER` varchar(250) COLLATE latin1_bin NOT NULL,
`SUB_DEST` varchar(250) COLLATE latin1_bin DEFAULT NULL,
`CLIENT_ID` varchar(250) COLLATE latin1_bin NOT NULL,
`SUB_NAME` varchar(250) COLLATE latin1_bin NOT NULL,
`SELECTOR` varchar(250) COLLATE latin1_bin DEFAULT NULL,
`LAST_ACKED_ID` bigint(20) DEFAULT NULL,
`PRIORITY` bigint(20) NOT NULL DEFAULT '',
`XID` varchar(250) COLLATE latin1_bin DEFAULT NULL,
PRIMARY KEY (`CONTAINER`,`CLIENT_ID`,`SUB_NAME`,`PRIORITY`),
KEY `ACTIVEMQ_ACKS_XIDX` (`XID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_bin; /*Table structure for table `ACTIVEMQ_LOCK` */ DROP TABLE IF EXISTS `ACTIVEMQ_LOCK`; CREATE TABLE `ACTIVEMQ_LOCK` (
`ID` bigint(20) NOT NULL,
`TIME` bigint(20) DEFAULT NULL,
`BROKER_NAME` varchar(250) COLLATE latin1_bin DEFAULT NULL,
PRIMARY KEY (`ID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_bin; /*Table structure for table `ACTIVEMQ_MSGS` */ DROP TABLE IF EXISTS `ACTIVEMQ_MSGS`; CREATE TABLE `ACTIVEMQ_MSGS` (
`ID` bigint(20) NOT NULL,
`CONTAINER` varchar(250) COLLATE latin1_bin DEFAULT NULL,
`MSGID_PROD` varchar(250) COLLATE latin1_bin DEFAULT NULL,
`MSGID_SEQ` bigint(20) DEFAULT NULL,
`EXPIRATION` bigint(20) DEFAULT NULL,
`MSG` longblob,
`PRIORITY` bigint(20) DEFAULT NULL,
`XID` varchar(250) COLLATE latin1_bin DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `ACTIVEMQ_MSGS_MIDX` (`MSGID_PROD`,`MSGID_SEQ`),
KEY `ACTIVEMQ_MSGS_CIDX` (`CONTAINER`),
KEY `ACTIVEMQ_MSGS_EIDX` (`EXPIRATION`),
KEY `ACTIVEMQ_MSGS_PIDX` (`PRIORITY`),
KEY `ACTIVEMQ_MSGS_XIDX` (`XID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_bin; /*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
三、测试消息持久化
package com.ckl.activemq; import org.apache.activemq.ActiveMQConnectionFactory; import javax.jms.*; public class HelloActiveMQ {
public static void main(String[] args) throws Exception {
HelloWorldProducer producer = new HelloWorldProducer();
HelloWorldConsumer consumer = new HelloWorldConsumer(); Thread threadProducer = new Thread(producer);
threadProducer.start();
//注释掉消费者,不然的话,马上消费者马上把消息消费了,来不及持久化
// Thread threadConsumer = new Thread(consumer);
// threadConsumer.start();
} public static class HelloWorldProducer implements Runnable {
public void run() {
try {
// Create a ConnectionFactory
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://192.168.2.140:61616"); // Create a Connection
Connection connection = connectionFactory.createConnection();
connection.start(); // Create a Session
Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE); // Create the destination (Topic or Queue)
Destination destination = session.createQueue("DemoQueue"); // Create a MessageProducer from the Session to the Topic or Queue
MessageProducer producer = session.createProducer(destination);
// producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);//消息需要持久化 // Create a messages
String text = "Hello world! From: " + Thread.currentThread().getName() + " : " + this.hashCode();
TextMessage message = session.createTextMessage(text); // Tell the producer to send the message
System.out.println("Sent message: " + message.hashCode() + " : " + Thread.currentThread().getName());
producer.send(message); // Clean up
session.close();
connection.close();
} catch (Exception e) {
System.out.println("Caught: " + e);
e.printStackTrace();
}
}
}
public static class HelloWorldConsumer implements Runnable, ExceptionListener {
public void run() {
try {
// Create a ConnectionFactory
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(
"tcp://192.168.2.140:61616");
// Create a Connection
Connection connection = connectionFactory.createConnection();
connection.start();
connection.setExceptionListener(this);
// Create a Session
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Create the destination (Topic or Queue)
Destination destination = session.createQueue("DemoQueue");
// Create a MessageConsumer from the Session to the Topic or
// Queue
MessageConsumer consumer = session.createConsumer(destination);
// Wait for a message
Message message = consumer.receive(1000);
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
String text = textMessage.getText();
System.out.println("Received: " + text);
} else {
System.out.println("Received: " + message);
}
consumer.close();
session.close();
connection.close();
} catch (Exception e) {
System.out.println("Caught: " + e);
e.printStackTrace();
}
} public synchronized void onException(JMSException ex) {
System.out.println("JMS Exception occured. Shutting down client.");
}
}
}
运行后,效果如下:
四、单独启动消费者,查看数据库
注释掉上面的生产者,单独启动消费者后,发现数据库中的消息已被消费。
五、备注
Although the JDBC Store does not offer the best performance, it makes fairly simple to create a simple Master-Slave robust broker setup.
When a group of Active MQ brokers is configured to use a shared database, they’ll all try to connect and grab a lock in the lock table,
but only one will succeed and become the master. The remaining brokers will be slaves, and will be in a wait state, not accepting client
connections until the master fails.
The lock table, is ACTIVEMQ_LOCK , and it is used to ensure that only one Active MQ broker instance can access the database at one time.
If an Active MQ broker can’t grab the database lock, that broker won’t initialize fully, and will wait until the lock becomes free, or it’s shut down.
看到一段资料补充下,涉及到ActiveMq配置为主从模式时,都会去尝试连接该数据源,为了防止资源争用出现的问题,加了ACTIVEMQ_LOCK表。
欢迎留言交流。
ActiveMQ学习系列(四)----消息持久化到mysql的更多相关文章
- ActiveMQ的几种消息持久化机制
为了避免意外宕机以后丢失信息,需要做到重启后可以恢复消息队列,消息系统一般都会采用持久化机制. ActiveMQ的消息持久化机制有JDBC,AMQ,KahaDB和LevelDB,无论使用哪种持久化方式 ...
- scrapy爬虫学习系列四:portia的学习入门
系列文章列表: scrapy爬虫学习系列一:scrapy爬虫环境的准备: http://www.cnblogs.com/zhaojiedi1992/p/zhaojiedi_python_00 ...
- DocX开源WORD操作组件的学习系列四
DocX学习系列 DocX开源WORD操作组件的学习系列一 : http://www.cnblogs.com/zhaojiedi1992/p/zhaojiedi_sharp_001_docx1.htm ...
- .net reactor 学习系列(四)---.net reactor应用场景
原文:.net reactor 学习系列(四)---.net reactor应用场景 前面已经学习了.net reactor一些基础知识,现在准备学习下实际的应用场景,只是简单的保护和 ...
- ActiveMQ消息持久化到Mysql数据库
1.把连接MySQL数据库的jar文件,放到ActiveMQ的lib目录下 2.修改ActiveMQ的conf目录下的activemq.xml文件,修改数据持久化的方式2.1 修改原来的kahadb的 ...
- ActiveMQ(3)---ActiveMQ原理分析之消息持久化
持久化消息和非持久化消息的存储原理 正常情况下,非持久化消息是存储在内存中的,持久化消息是存储在文件中的.能够存储的最大消息数据在${ActiveMQ_HOME}/conf/activemq.xml文 ...
- RabbitMQ学习系列四-EasyNetQ文档跟进式学习与实践
EasyNetQ文档跟进式学习与实践 https://www.cnblogs.com/DjlNet/p/7603554.html 这里可能有人要问了,为什么不使用官方的nuget包呐:RabbitMQ ...
- Identity Server4学习系列四之用户名密码获得访问令牌
1.简介 Identity Server4支持用户名密码模式,允许调用客户端使用用户名密码来获得访问Api资源(遵循Auth 2.0协议)的Access Token,MS可能考虑兼容老的系统,实现了这 ...
- RabbitMQ系列(四)--消息如何保证可靠性传输以及幂等性
一.消息如何保证可靠性传输 1.1.可能出现消息丢失的情况 1.Producer在把Message发送Broker的过程中,因为网络问题等发生丢失,或者Message到了Broker,但是出了问题,没 ...
随机推荐
- WMWARE下/sbin/scsi_id 返回值为空
[root@HE1 ~]# /sbin/scsi_id -g -u /dev/sdb 第一种是从VMWARE层面去解决,用文本编辑器修改vmx文件,在vmx文件中任意位置(通常在最后)添加如下行: d ...
- HTML 表单元素、 输入类型、Input 属性
<input> 元素 最重要的表单元素是 <input> 元素. <input> 元素根据不同的 type 属性,可以变化为多种形态. 注释:下一章讲解所有 HTM ...
- iReport默认参数和变量的含义解析【转】
iReport默认参数和变量的含义解析[转] http://uule.iteye.com/blog/775338 参考:http://blog.csdn.net/terryzero/article/d ...
- Nodejs.sublime-build 在sublime3中的配置
{ "cmd": ["node", "$file"], "file_regex": "^[ ]*File \& ...
- HDU5873
Football Games Time Limit: 2000/1000 MS (Java/Others) Memory Limit: 65536/65536 K (Java/Others)To ...
- js在新页面中返回到上一页浏览的历史位置
在微信浏览器中浏览页面时,在当前页面中当我们将页面往下滚动到某一个位置时,可能我们就会点击某个链接而页面跳转到了另外一个页面,而当我们又返回到上一个页面时我们会发现那个页面还停留在我们之前浏览的位置, ...
- ArcGIS制图表达Representation-符号制作
ArcGIS制图表达Representation-符号制作 by 李远祥 在ArcGIS的符号里面,存在着两种符号体系,一种是传统的标准符号体系,一种是制图表达符号体系.标准符号几乎被绝大部分ArcG ...
- salesforce 零基础学习(六十五)VF页面应善于使用变量和函数(一)常用变量的使用
我们在使用formula或者validation rules等的时候通常会接触到很多function,这些函数很便捷的解决了我们很多问题.其实很多函数也可以应用在VF页面中,VF页面有时候应该善于使用 ...
- excel表格的特殊需求引发的Java思考
前言: 前些天遇到了这样的一个需求,将下图: 将表格中货号-前面部分一致的行合成一行,并且将第二行,第三行的价格添加到第一行中为价格二,价格三.如图: 接到这样的需求,我的第一感觉是直接手动合并(暗暗 ...
- OSS.Social微信项目标准库介绍
经过本周的努力,昨晚终于完成OSS.Social微信项目的标准库支持,当前项目你已经可以同时在.net framework和.net core 中进行调用,调用方法也发生了部分变化,这里我简单分享下, ...