场景:一个上传接口,需要上传几十M的文件,文件中包含10几W的数据,然后对10+W的数据进行同步批量插入,每次批量插入1W。最后返回结果。

项目上线一段时间后,上传接口出现问题,数据库用的MySQL5.7.21,报了如下错误:

2018-07-16 01:30:03.497 ERROR com.alibaba.druid.pool.DruidDataSource Line:1594 - discard connection
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet successfully received from the server was 1,480 milliseconds ago. The last packet sent successfully to the server was 1,480 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:137)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy63.query(Unknown Source)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434)
at com.sun.proxy.$Proxy25.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:231)
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:137)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
at com.sun.proxy.$Proxy51.selectReportListByReportDate(Unknown Source)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl.generateProcurementReport(PrcmtReportServiceImpl.java:97)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$8c585594.generateProcurementReport(<generated>)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$FastClassBySpringCGLIB$$f325780.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.rxwx.service.report.impl.PrcmtReportServiceImpl$$EnhancerBySpringCGLIB$$9a830f32.generateProcurementReport(<generated>)
at com.rxwx.task.sup.GenarateReportJob.selfGenReportTask1(GenarateReportJob.java:37)
at com.rxwx.task.sup.GenarateReportJob$$FastClassBySpringCGLIB$$64b31449.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503)
... 58 common frames omitted

首先百度了一发,按照网上的解决方案,修改了my.cnf中的超时时间:

wait_timeout=31536000
interactive_timeout=31536000

将原来默认的8小时改为1年。这里单位是秒。

修改了德鲁伊的:testOnBorrow = true。

跑了一天,MMP,凌晨又出现这个错误了。

一个电话把睡梦中的我吵醒,扰我周末扰我梦,没得办法,重启下服务,重启下MySQL,先上传吧,睡觉。

周一到公司谷歌了一发,修改my.cnf的net_read_timeout和net_write_timeout 参数,将这两个参数调大,改为6000。由于项目中的数据存在无效数据,所以删除了10天之前的无效数据,减少了表中的数量。明天看效果。

show global variables like "%timeout%";

  1. connect_timeout

    连接超时 mysql连接共有6次握手,3次TCP协议这个跟connect_timeout参数没有关系,另外3次跟connect_timeout参数有关系,该参数主要是为了防止网络不佳时应用重连导致连接数涨太快,一般默认即可。

  2. delayed_insert_timeout

    这是为MyISAM INSERT DELAY设计的超时参数,在INSERT DELAY中止前等待INSERT语句的时间

  3. interactive_timeout

    服务器关闭交互式连接前等待活动的秒数。交互式客户端定义为在mysql_real_connect()中使用CLIENT_INTERACTIVE选项的客户端。参数默认值:28800秒(8小时)

  4. lock_wait_timeout

    锁等待超时时间

  5. net_read_timeout / net_write_timeout

    这个参数只对TCP/IP链接有效,分别是数据库等待接收客户端发送网络包和发送网络包给客户端的超时时间,这是在Activity状态下的线程才有效的参数

  6. slave_net_timeout

    这是Slave判断主机是否挂掉的超时设置,在设定时间内依然没有获取到Master的回应就人为Master挂掉了

  7. wait_timeout

    服务器关闭非交互连接之前等待活动的秒数。

效果很明显。今天没有出现问题。

The last packet successfully received from the server was 1,480 milliseconds ago.的更多相关文章

  1. The last packet successfully received from the server was 20,519 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.

    本地升级了下MySQL的版本,从5.6升为5.7,数据文件直接拷贝的,项目查询数据库报错: Could not retrieve transation read-only status server ...

  2. 【异常】ser class threw exception: java.sql.SQLException: The last packet successfully received from the server was 39,444 milliseconds ago. The last

    1 详细异常 ser class threw exception: java.sql.SQLException: The last packet successfully received from ...

  3. The last packet successfully received from the server was 39,900 milliseconds ago问题解决

    1,之前用Mysql或者mycat的时候都没有这个问题.后来改为haproxy+keepalived+mycat后出现这个问题 2,网上查了很多说法,我按照网上说的改了 datasource: url ...

  4. The last packet successfully received from the server was 2,926,157 milliseconds ago. The last packet sent successfully to the server was 2,926,158 milliseconds ago. is longer than the server configured value of 'wait_timeout'. 解决办法

    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully rec ...

  5. Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 78,050,512 milliseconds ago.

    今天访问已经架上服务器的网站,报错: Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet s ...

  6. Communications link failure,The last packet successfully received from the server was *** millisecon

    使用Connector/J连接MySQL数据库,程序运行较长时间后就会报以下错误: Communications link failure,The last packet successfully r ...

  7. Mysql 的异常:The last packet successfully received from the server was 90 milliseconds ago. The last packet sent successfully to the server was 43,603,303 milliseconds ago. is longer than the server con

    调试一个程序, 调试到一半, 下班回家, 程序卡在了某一行, 第二天早上回来一看, 发现了异常: Wed Sep :: GMT+: WARN: Establishing SSL connection ...

  8. 数据库连接超时:“The last packet successfully received from the server was xxx milliseconds ago”

    产生的原因:应用方的数据库连接有效期时间,大于数据库自己设置的有效期. 解决方案: 一.修改druid配置(如果使用druid的话) spring.datasource.druid.validatio ...

  9. The last packet successfully received from the server was 49,061,696 millise

    解决连接:https://blog.csdn.net/pandajava/article/details/41946251

随机推荐

  1. Diagnostic Viewer 显示空白

    MATLAB R2014a仿真出现错误后,查找错误时,Diagnostic Viewer 显示空白.网上查的一种解决方法.我的电脑是Windows7. 1.点击 所有程序>附件>记事本(以 ...

  2. js动画--缓冲动画

    前面讲述的动画速度都是匀速的,现实生活中的运动速度的变化不一定是恒定的,存在一定的缓冲,就像火车进站一样,速度会越来越慢. 对于改变速度措施,其实只要将动画第一课的程序稍微的改变一下就可以了,我们来看 ...

  3. Nginx——跨域造成的504问题

    前言 前台域名和后台域名是两个不同不同的二级域名,访问的时候造成了跨域,出现了504错误 解决 修改Nginx配置,将超时的时间设置为1200秒 keepalive_timeout 1200; pro ...

  4. Docker Quick Start

    翻译自官方Quick Start: https://hub.docker.com/?overlay=onboarding 以Windows为例 1.下载源码 下载构建第一个容器的所需要的所有的东西 需 ...

  5. OpenCV 学习笔记(11)【OpenCV】光流场方法标出前景(运动)和背景(静止)

    用光流场方法,标出前景(运动)和背景(静止). 环境:VS2017 + OpenCV3.4.1 光流场介绍可以参见英文版学习OpenCV3的第17章Tracking 英文原版学习OpenCV3下载链接 ...

  6. 使用viper 进行golang 应用的配置管理

    viper 是一个强大的golang 配置管理包,支持多种配置格式内容的读取,同时支持读取key/value 存储的数据 而且不只是读取内容 ,同时也包含了,配置的写入操作. 以下是一个简单的demo ...

  7. Xml与Map之间的相互转换

    一.(单层)xml转换为map /** * XML格式字符串转换为Map * * @param xml XML字符串 * @return XML数据转换后的Map * @throws Exceptio ...

  8. linux grep 查询多行的方法

    linux grep 查询多行的方法 经常查询 git log 中使用的方法, 如下 git log |grep "xxxx" 如果想查询多行的话, 就有些尴尬, 如果想查询多行的 ...

  9. Linux expect详解

    随处可见的expect第一次见expect这个命令还是我第一次参加全量上线的时候,那是公司的一个牛人用Shell脚本写的一套自动部署.MD5 比对.发布的全量上线工具,没事的时候,看了下其中的几个脚本 ...

  10. ubuntu笔记2-误删dpkg的/var/lib/dpkg/info文件夹

    由于误删了dpkg的/var/lib/dpkg/info文件夹,导致安装文件的时候报错 错误提示:E: Sub-process /usr/bin/dpkg returned an error code ...