【转】reduce端缓存数据过多出现FGC，导致reduce生成的数据无法写到hdfs
转自 http://blog.csdn.net/bigdatahappy/article/details/41726389
转这个目的，是因为该贴子中调优思路不错，值得学习
搜索推荐有一个job，1000多个map，200个reduce，运行到最后只剩一个reduce（10.39.6.130上）的时候，出现以下异常，导致job失败：
- 2014-12-04 15:49:04,297 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 12 segments left of total size: 11503294914 bytes
- 2014-12-04 15:49:04,314 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
- 2014-12-04 15:49:04,394 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.lzo_deflate]
- 2014-12-04 16:02:26,889 WARN [ResponseProcessor for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086] org.apache.hadoop.hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086
- java.io.IOException: Bad response ERROR for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086 from datanode 10.39.5.193:50010
- at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:819)
- 2014-12-04 16:02:26,889 WARN [ResponseProcessor for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223] org.apache.hadoop.hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223
- java.io.IOException: Bad response ERROR for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223 from datanode 10.39.1.90:50010
- at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:819)
- 2014-12-04 16:02:26,891 WARN [DataStreamer for file /dw_ext/recmd/mds6/mds_filter_relation_10/20141203/_temporary/1/_temporary/attempt_1415948652989_195149_r_000158_3/user-r-00158 block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086] org.apache.hadoop.hdfs.DFSClient: Error Recovery for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086 in pipeline 10.39.6.130:50010, 10.39.5.185:50010, 10.39.5.193:50010: bad datanode 10.39.5.193:50010
- 2014-12-04 16:02:26,891 WARN [DataStreamer for file /dw_ext/recmd/mds6/mds_filter_relation_10/20141203/_temporary/1/_temporary/attempt_1415948652989_195149_r_000158_3/exposure-r-00158 block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223] org.apache.hadoop.hdfs.DFSClient: Error Recovery for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223 in pipeline 10.39.6.130:50010, 10.39.1.89:50010, 10.39.1.90:50010: bad datanode 10.39.1.90:50010
- java.io.EOFException: Premature EOF: no length prefix available
- at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1987)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:176)
- at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:796)
- 2014-12-04 16:05:23,743 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space
- at java.util.Arrays.copyOf(Arrays.java:2734)
- at java.util.Vector.ensureCapacityHelper(Vector.java:226)
- at java.util.Vector.add(Vector.java:728)
- at rec.CommonUtil.pack_Treeset(CommonUtil.java:395)
- at rec.ConvertExposure10$MyReducer.collect_exposure(ConvertExposure10.java:259)
- at rec.ConvertExposure10$MyReducer.reduce(ConvertExposure10.java:329)
- at rec.ConvertExposure10$MyReducer.reduce(ConvertExposure10.java:234)
- at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
- at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
- at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
- at java.security.AccessController.doPrivileged(Native Method)
- at javax.security.auth.Subject.doAs(Subject.java:396)
- at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1550)
- at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
从异常上看，首先是reduce在往hdfs写数据时，发现建pipeline时，没有收到pipeline上最后一个节点的回应：
- 2014-12-04 16:02:26,889 WARN [ResponseProcessor for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086] org.apache.hadoop.hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086
- java.io.IOException: Bad response ERROR for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086 from datanode 10.39.5.193:50010
- at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:819)
- 2014-12-04 16:02:26,889 WARN [ResponseProcessor for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223] org.apache.hadoop.hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223
- java.io.IOException: Bad response ERROR for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223 from datanode 10.39.1.90:50010
- at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:819)
- 2014-12-04 16:02:26,891 WARN [DataStreamer for file /dw_ext/recmd/mds6/mds_filter_relation_10/20141203/_temporary/1/_temporary/attempt_1415948652989_195149_r_000158_3/user-r-00158 block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086] org.apache.hadoop.hdfs.DFSClient: Error Recovery for block BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086 in pipeline 10.39.6.130:50010, 10.39.5.185:50010, 10.39.5.193:50010: bad datanode 10.39.5.193:50010
- 2014-12-04 16:02:26,891 WARN [DataStreamer for file /dw_ext/recmd/mds6/mds_filter_relation_10/20141203/_temporary/1/_temporary/attempt_1415948652989_195149_r_000158_3/exposure-r-00158 block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223] org.apache.hadoop.hdfs.DFSClient: Error Recovery for block BP-1386326728-10.39.2.131-1382089338395:blk_1394153869_320473223 in pipeline 10.39.6.130:50010, 10.39.1.89:50010, 10.39.1.90:50010: bad datanode 10.39.1.90:50010
- java.io.EOFException: Premature EOF: no length prefix available
- at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1987)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:176)
- at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:796)
这里以写block：blk_1394149732_320469086为例，pipeline[10.39.6.130:50010, 10.39.5.185:50010, 10.39.5.193:50010]上面的最后一个DN是10.39.5.193，到10.39.5.193查看该block的日志信息：
2014-12-04 16:00:57,424 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Exception for BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.39.5.193:50010 remote=/10.39.5.185:58225]
- at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
- at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
- at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
- at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
- at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
- at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
- at java.io.DataInputStream.read(DataInputStream.java:132)
- at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:192)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:446)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:702)
- at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:739)
- at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:124)
- at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71)
- at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:232)
- at java.lang.Thread.run(Thread.java:662)
- 2014-12-04 16:00:57,424 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086, type=LAST_IN_PIPELINE, downstreams=0:[]: Thread is interrupted.
10.39.5.193上面日志显示，在读取pipeline上一个节点10.39.5.185的Packet时，一直读取不到，直到10分钟超时：
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.39.5.193:50010 remote=/10.39.5.185:58225]
那我们来看以下pipeline上第二个节点10.39.5.185，dn日志如下：
2014-12-04 16:00:57,988 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086, type=HAS_DOWNSTREAM_IN_PIPELINE
- java.io.EOFException: Premature EOF: no length prefix available
- at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1987)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:176)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1083)
- at java.lang.Thread.run(Thread.java:662)
- 2014-12-04 16:00:58,008 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Exception for BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.39.5.185:50010 remote=/10.39.6.130:59083]
- at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
- at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
- at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
- at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
- at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
- at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
- at java.io.DataInputStream.read(DataInputStream.java:132)
- at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:192)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:446)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:702)
- at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:739)
- at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:124)
- at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71)
- at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:232)
- at java.lang.Thread.run(Thread.java:662)
- 2014-12-04 16:00:58,009 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086, type=HAS_DOWNSTREAM_IN_PIPELINE: Thread is interrupted.
和10.39.5.193日志类似，也是在等待读取pipeline的第一个节点10.39.6.130的Packet时超时：
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.39.5.185:50010 remote=/10.39.6.130:59083]
这样说来，问题出在10.39.6.130上，也即reduce任务运行的节点上，该节点DN日志如下：
- 2014-12-04 16:00:59,987 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Exception for BP-1386326728-10.39.2.131-1382089338395:blk_1394149732_320469086
- java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.39.6.130:50010 remote=/10.39.6.130:45259]
- at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
- at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
- at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
- at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
- at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
- at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
- at java.io.DataInputStream.read(DataInputStream.java:132)
- at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:192)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
- at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:446)
- at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:702)
- at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:739)
- at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:124)
- at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71)
- at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:232)
- at java.lang.Thread.run(Thread.java:662)
但是根据日志信息，10.39.6.130的DN也是在等待Packet，但是一直等到超时也没等到：
java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.39.6.130:50010 remote=/10.39.6.130:45259]
看来不是10.39.6.130节点DN的问题，如果pipeline上面三个DN都没有问题，那问题肯定就出在dfsclient端了，也就是reduce任务在往hdfs写数据的时候根本就没有写得出去，在dfsclient上面就给堵住了，接下来查看dfsclient，也就是reduce任务进程的执行情况：
在10.39.6.130上，根据任务id：attempt_1415948652989_195149_r_000158_3 找到进程id 31050，查看内存使用情况：
- jstat -gcutil 31050 1000：
- S0 S1 E O P YGC YGCT FGC FGCT GCT
- 68.95 0.00 0.00 92.98 66.32 111 16.825 10 25.419 42.244
- 68.95 0.00 0.00 92.98 66.32 111 16.825 10 25.419 42.244
- 68.95 0.00 0.00 92.98 66.32 111 16.825 10 25.419 42.244
- 68.95 0.00 0.00 92.98 66.32 111 16.825 10 25.419 42.244
- 68.95 0.00 0.00 92.98 66.32 111 16.825 10 25.419 42.244
- 68.95 0.00 0.00 92.98 66.32 111 16.825 10 25.419 42.244
- 68.95 0.00 26.75 100.00 66.32 111 16.825 10 25.419 42.244
- 0.00 0.00 31.85 100.00 68.16 111 16.825 10 44.767 61.591
- 0.00 0.00 35.37 100.00 68.16 111 16.825 10 44.767 61.591
- 0.00 0.00 40.64 100.00 68.16 111 16.825 10 44.767 61.591
- 0.00 0.00 45.35 100.00 68.16 111 16.825 10 44.767 61.591
- 0.00 0.00 48.87 100.00 68.16 111 16.825 10 44.767 61.591
- 0.00 0.00 54.14 100.00 68.16 111 16.825 10 44.767 61.591
- 0.00 0.00 58.85 100.00 68.16 111 16.825 10 44.767 61.591
果然，JVM老年代被占满，不断执行FGC，直接stop the world，导致JVM没法对外提供服务，导致dfsclient挂起，一直不能往pipeline上面的节点写Packet，直到socket超时。
根据reduce任务最后的日志，也从侧面验证了这种观点：
Error running child : java.lang.OutOfMemoryError: Java heap space
既然是OOM导致的job失败，那是什么对象导致的内存泄露呢：
执行：
- jmap -histo:live 31050 > jmap.log
- cat jmap.log :
- num #instances #bytes class name
- ----------------------------------------------
- 1: 71824177 2872967080 java.util.TreeMap$Entry
- 2: 71822939 1723750536 java.lang.Long
- 3: 10684 24777776 [B
- 4: 47174 6425152 <methodKlass>
- 5: 47174 6408120 <constMethodKlass>
- 6: 3712 4429776 <constantPoolKlass>
- 7: 66100 3979224 <symbolKlass>
- 8: 3712 2938192 <instanceKlassKlass>
- 9: 3125 2562728 <constantPoolCacheKlass>
- 10: 3477 1267752 [I
- 11: 12923 1180224 [C
- 12: 1794 772488 <methodDataKlass>
- 13: 13379 428128 java.lang.String
- 14: 4034 419536 java.lang.Class
- 15: 6234 410312 [S
- 16: 6409 352576 [[I
- 17: 7567 242144 java.util.HashMap$Entry
- 18: 293 171112 <objArrayKlassKlass>
- 19: 4655 148960 java.util.Hashtable$Entry
- 20: 1535 135080 java.lang.reflect.Method
- 21: 842 121696 [Ljava.util.HashMap$Entry;
果然啊，reduce代码中使用了TreeMap，往里面放置了大量对象，导致出现OOM，TreeMap的Entry就站用了2.8G内存，而我们reduce设置的内存也就1.5G。
总结：对该job出现的异常，一般在以下几种情况下发生：
1、写数据块的DN出现问题，不能写入，就像之前出现的DN由于本地读问题导致xceivers(每个DN用于并发数据传输处理最大线程数)达到4096，耗尽了所有的线程，没法对新发起的输出写入做出相应。
2、网络出现异常，DN节点进或出的带宽被耗尽，导致数据写不出去或者写不进来，这种情况可以观察ganglia看节点带宽使用情况，这种情况一般比较少。当该job出现问题的时候，也怀疑过是带宽被耗尽的问题，查看了一下相关节点ganglia带宽使用情况，最大in/out 85M/s左右，最后排除是带宽问题。
3、dfsclient出现问题，长时间没反应，导致已经发起的socket超时。由于dfsclient情况比较复杂，出现问题的情况比较多，比如本问就是因为reduce出现内存溢出，jvm不断进行FGC，导致dfsclient挂起，最终socket出现超时。
【转】reduce端缓存数据过多出现FGC，导致reduce生成的数据无法写到hdfs的更多相关文章
- 在java中调用mockjs生成模拟数据
一.手写版 在前端有个模拟数据的神器 Mock.js 能生成随机数据,拦截 Ajax 请求,然后我觉得他的这个生成随机数据不错.然后我就到度娘一顿操作,没找到类似的java实现,于是就有了下面的代码: ...
- 由于检索用户的本地应用程序数据路径时出错，导致无法生成 SQL Server 的用户实例
/”应用程序中的服务器错误. 由于检索用户的本地应用程序数据路径时出错,导致无法生成 SQL Server 的用户实例.请确保该用户在此计算机上有本地用户配置文件.该连接将关闭. 堆栈跟踪: [Sql ...
- 查询数据过多页面反应慢引入缓存解决方案（Redis、H2）
问题:原系统查询接口不支持分页也不可能加入分页支持,导致Ajax查询数据过多,返回数据达到2W多条记录时响应已经极慢,查询功能不要求数据实时性,页面反应速度极慢.体验不好:经排查是由于数据量过大导 ...
- hadoop的压缩解压缩,reduce端join,map端join
hadoop的压缩解压缩 hadoop对于常见的几种压缩算法对于我们的mapreduce都是内置支持,不需要我们关心.经过map之后,数据会产生输出经过shuffle,这个时候的shuffle过程特别 ...
- 深入理解Spark 2.1 Core （十一）：Shuffle Reduce 端的原理与源代码分析
http://blog.csdn.net/u011239443/article/details/56843264 在<深入理解Spark 2.1 Core (九):迭代计算和Shuffle的原理 ...
- WEB端缓存机制
WEB端缓存机制 什么是WEB缓存 Web缓存是指一个Web资源(如html页面,图片,js,数据等)存在于Web服务器和客户端(浏览器)之间的副本.缓存会根据进来的请求保存输出内容的副本:当下一个请 ...
- MapReduce在Map端的Combiner和在Reduce端的Partitioner
1.Map端的Combiner. 通过单词计数WordCountApp.java的例子,如何在Map端设置Combiner... 只附录部分代码: /** * 以文本 * hello you * he ...
- Haoop MapReduce 的Partition和reduce端的二次排序
先贴一张原理图(摘自hadoop权威指南第三版) 实际中看了半天还是不太理解其中的Partition,和reduce端的二次排序,最终根据实验来结果来验证自己的理解 1eg 数据如下 20140101 ...
- 简单服务端缓存API设计
Want 我们希望设计一套缓存API,适应不同的缓存产品,并且基于Spring框架完美集成应用开发. 本文旨在针对缓存产品定义一个轻量级的客户端访问框架,目标支持多种缓存产品,面向接口编程,目前支持简 ...
随机推荐
- xpath基础知识
相关链接: http://www.ruanyifeng.com/blog/2009/07/xpath_path_expressions.html 自动生成xpath的工具: http://blog.s ...
- WinAPI【远程注入】三种注入方案【转】
来源:http://www.cnblogs.com/okwary/archive/2008/12/20/1358788.html 导言: 我 们在Code project(www.codeprojec ...
- 初学require.js
引入require.js,可以解决的问题: (1)实现js文件的异步加载,避免网页失去响应: (2)管理模板之间的依赖性,便于代码的编写和维护. 它的模块管理遵循AMD规范(Asynchronous ...
- Ubuntu 16.04 下安装Firefox的Flash插件
在ubuntu系统环境下面打开优酷视频,发现无法播放视频.Adobe Flash Player 是一款轻量级浏览器插件,具有丰富的 Internet 应用运行时间,提供持续的迷人用户体验.绝妙的音频/ ...
- DNS原理及其解析过程【精彩剖析】(转)
2012-03-21 17:23:10 标签:dig wireshark bind nslookup dns 原创作品,允许转载,转载时请务必以超链接形式标明文章 原始出处 .作者信息和本声明.否 ...
- 中位数与第K小元素
算法实际上是模仿快速排序算法设计出来的,其基本思想也是对输入数组进行递归划分,与快速排序不同的是,它只对划分出来的子数组之一进行递归处理: int randompartition(int a[],in ...
- 轻松架设时时监控工具Cacti
Cacti是一套基于PHP,MySQL,SNMP及 RRDTool开发的网络流量监测图形分析工具.通过snmpget来获取数据,使用 RRDtool绘画图形,而且你完全可以不需要了解RRDtool复杂 ...
- 在CI中集成phpmailer，方便使用SMTP发送邮件
直接使用phpmailer的话,有时候不是很方便,特别你的很多功能都是基于CI完成的时候,要相互依赖就不方便了,所以在想,那是否可以将phpmailer集成到CI中呢,像使用email类这样使用他,功 ...
- Android Studio项目引入外部库注意事项(PullToRefresh)
Android Studio开发App项目时引入第三方库是个比较麻烦的事情.之前导入Volley就折腾了好久,导入下拉刷新控件PullToRefresh时又碰到了各种问题.在此记录一下,以便查阅. 一 ...
- 初识lua
转自:http://www.oschina.net/question/12_115993-- 两个横线是单行注释(译者注:这跟 SQL 一样) --[[ 增加两个 [ 和 ] 变成多行注释 我是多行注 ...