HttpClient(4.5.x)正确的使用姿势
前言:
httpclient(4.5.x)默认是启动连接池的, 其降低时耗(避免连接初3次握手, 以及关闭4次握手的消耗), 显著提升高并发处理能力(大量减少time_wait), 确实扮演了重要的角色. 但是封装httpclient, 需要了解不少细节, 还要根据业务合理配置参数.
这里结合这段时间深入httpclient(4.5.x)源码分析, 结合网上的代码案例, 以及线下测试的结果. 尝试写一个可用的的httpclient封装类, 兼顾性能, 接口友好. 感谢cctv, ^_^.
相关文章:
1. HttpClient官方sample代码的深入分析(连接池)
第一版本:
啥也不说了, 直接贴代码了.
public class PooledHttpClientAdaptor {
private static final int DEFAULT_POOL_MAX_TOTAL = 200;
private static final int DEFAULT_POOL_MAX_PER_ROUTE = 200;
private static final int DEFAULT_CONNECT_TIMEOUT = 500;
private static final int DEFAULT_CONNECT_REQUEST_TIMEOUT = 500;
private static final int DEFAULT_SOCKET_TIMEOUT = 2000;
private PoolingHttpClientConnectionManager gcm = null;
private CloseableHttpClient httpClient = null;
// 连接池的最大连接数
private final int maxTotal;
// 连接池按route配置的最大连接数
private final int maxPerRoute;
// tcp connect的超时时间
private final int connectTimeout;
// 从连接池获取连接的超时时间
private final int connectRequestTimeout;
// tcp io的读写超时时间
private final int socketTimeout;
public PooledHttpClientAdaptor() {
this(
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_TOTAL,
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_PER_ROUTE,
PooledHttpClientAdaptor.DEFAULT_CONNECT_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_CONNECT_REQUEST_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_SOCKET_TIMEOUT
);
}
public PooledHttpClientAdaptor(
int maxTotal,
int maxPerRoute,
int connectTimeout,
int connectRequestTimeout,
int socketTimeout
) {
this.maxTotal = maxTotal;
this.maxPerRoute = maxPerRoute;
this.connectTimeout = connectTimeout;
this.connectRequestTimeout = connectRequestTimeout;
this.socketTimeout = socketTimeout;
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
gcm = new PoolingHttpClientConnectionManager(registry);
gcm.setMaxTotal(this.maxTotal);
gcm.setDefaultMaxPerRoute(this.maxPerRoute);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectTimeout) // 设置连接超时
.setSocketTimeout(this.socketTimeout) // 设置读取超时
.setConnectionRequestTimeout(this.connectRequestTimeout) // 设置从连接池获取连接实例的超时
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClient = httpClientBuilder
.setConnectionManager(gcm)
.setDefaultRequestConfig(requestConfig)
.build();
}
public String doGet(String url) {
return this.doGet(url, Collections.EMPTY_MAP, Collections.EMPTY_MAP);
}
public String doGet(String url, Map<String, Object> params) {
return this.doGet(url, Collections.EMPTY_MAP, params);
}
public String doGet(String url,
Map<String, String> headers,
Map<String, Object> params
) {
// *) 构建GET请求头
String apiUrl = getUrlWithParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpGet);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
public String doPost(String apiUrl, Map<String, Object> params) {
return this.doPost(apiUrl, Collections.EMPTY_MAP, params);
}
public String doPost(String apiUrl,
Map<String, String> headers,
Map<String, Object> params
) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求headers
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpPost.addHeader(entry.getKey(), entry.getValue());
}
}
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpPost);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if ( entityRes != null ) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if (response != null) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
private HttpEntity getUrlEncodedFormEntity(Map<String, Object> params) {
List<NameValuePair> pairList = new ArrayList<NameValuePair>(params.size());
for (Map.Entry<String, Object> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue().toString());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
private String getUrlWithParams(String url, Map<String, Object> params) {
boolean first = true;
StringBuilder sb = new StringBuilder(url);
for (String key : params.keySet()) {
char ch = '&';
if (first == true) {
ch = '?';
first = false;
}
String value = params.get(key).toString();
try {
String sval = URLEncoder.encode(value, "UTF-8");
sb.append(ch).append(key).append("=").append(sval);
} catch (UnsupportedEncodingException e) {
}
}
return sb.toString();
}
}
存在问题&解决方案
这个版本基本没啥问题, 但是当流量为0时, 你会发现存在处于ClOSE_WAIT的连接. 究其原因是, httpclient清理过期/被动关闭的socket, 是采用懒惰清理的策略. 它是在连接从连接池取出使用的时候, 检测状态并做相应处理. 如果没有流量, 那这些socket将一直处于CLOSE_WAIT(半连接的状态), 系统资源被浪费.
不过解决方案也相当的简单, 官方的建议引入一个清理线程, 定期主动处理过期/空闲连接, 这样就OK了.
private class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean exitFlag = false;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
this.connMgr = connMgr;
setDaemon(true);
}
@Override
public void run() {
while (!this.exitFlag) {
synchronized (this) {
try {
this.wait(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 关闭失效的连接
connMgr.closeExpiredConnections();
// 可选的, 关闭30秒内不活动的连接
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
public void shutdown() {
this.exitFlag = true;
synchronized (this) {
notify();
}
}
}
最终版本
直接整合贴代码, ^_^.
public class PooledHttpClientAdaptor {
private static final int DEFAULT_POOL_MAX_TOTAL = 200;
private static final int DEFAULT_POOL_MAX_PER_ROUTE = 200;
private static final int DEFAULT_CONNECT_TIMEOUT = 500;
private static final int DEFAULT_CONNECT_REQUEST_TIMEOUT = 500;
private static final int DEFAULT_SOCKET_TIMEOUT = 2000;
private PoolingHttpClientConnectionManager gcm = null;
private CloseableHttpClient httpClient = null;
private IdleConnectionMonitorThread idleThread = null;
// 连接池的最大连接数
private final int maxTotal;
// 连接池按route配置的最大连接数
private final int maxPerRoute;
// tcp connect的超时时间
private final int connectTimeout;
// 从连接池获取连接的超时时间
private final int connectRequestTimeout;
// tcp io的读写超时时间
private final int socketTimeout;
public PooledHttpClientAdaptor() {
this(
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_TOTAL,
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_PER_ROUTE,
PooledHttpClientAdaptor.DEFAULT_CONNECT_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_CONNECT_REQUEST_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_SOCKET_TIMEOUT
);
}
public PooledHttpClientAdaptor(
int maxTotal,
int maxPerRoute,
int connectTimeout,
int connectRequestTimeout,
int socketTimeout
) {
this.maxTotal = maxTotal;
this.maxPerRoute = maxPerRoute;
this.connectTimeout = connectTimeout;
this.connectRequestTimeout = connectRequestTimeout;
this.socketTimeout = socketTimeout;
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
this.gcm = new PoolingHttpClientConnectionManager(registry);
this.gcm.setMaxTotal(this.maxTotal);
this.gcm.setDefaultMaxPerRoute(this.maxPerRoute);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectTimeout) // 设置连接超时
.setSocketTimeout(this.socketTimeout) // 设置读取超时
.setConnectionRequestTimeout(this.connectRequestTimeout) // 设置从连接池获取连接实例的超时
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClient = httpClientBuilder
.setConnectionManager(this.gcm)
.setDefaultRequestConfig(requestConfig)
.build();
idleThread = new IdleConnectionMonitorThread(this.gcm);
idleThread.start();
}
public String doGet(String url) {
return this.doGet(url, Collections.EMPTY_MAP, Collections.EMPTY_MAP);
}
public String doGet(String url, Map<String, Object> params) {
return this.doGet(url, Collections.EMPTY_MAP, params);
}
public String doGet(String url,
Map<String, String> headers,
Map<String, Object> params
) {
// *) 构建GET请求头
String apiUrl = getUrlWithParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpGet);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
public String doPost(String apiUrl, Map<String, Object> params) {
return this.doPost(apiUrl, Collections.EMPTY_MAP, params);
}
public String doPost(String apiUrl,
Map<String, String> headers,
Map<String, Object> params
) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求headers
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpPost.addHeader(entry.getKey(), entry.getValue());
}
}
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpPost);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if ( entityRes != null ) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if (response != null) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
private HttpEntity getUrlEncodedFormEntity(Map<String, Object> params) {
List<NameValuePair> pairList = new ArrayList<NameValuePair>(params.size());
for (Map.Entry<String, Object> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue().toString());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
private String getUrlWithParams(String url, Map<String, Object> params) {
boolean first = true;
StringBuilder sb = new StringBuilder(url);
for (String key : params.keySet()) {
char ch = '&';
if (first == true) {
ch = '?';
first = false;
}
String value = params.get(key).toString();
try {
String sval = URLEncoder.encode(value, "UTF-8");
sb.append(ch).append(key).append("=").append(sval);
} catch (UnsupportedEncodingException e) {
}
}
return sb.toString();
}
public void shutdown() {
idleThread.shutdown();
}
// 监控有异常的链接
private class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean exitFlag = false;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
this.connMgr = connMgr;
setDaemon(true);
}
@Override
public void run() {
while (!this.exitFlag) {
synchronized (this) {
try {
this.wait(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 关闭失效的连接
connMgr.closeExpiredConnections();
// 可选的, 关闭30秒内不活动的连接
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
public void shutdown() {
this.exitFlag = true;
synchronized (this) {
notify();
}
}
}
}
总结:
其实没啥难道, 主要就是买个安心, 这样写是安全的, 是经得起线上考验的.
HttpClient(4.5.x)正确的使用姿势的更多相关文章
- 如约而至，Java 10 正式发布！ Spring+SpringMVC+MyBatis+easyUI整合进阶篇（十四）Redis缓存正确的使用姿势 努力的孩子运气不会太差，跌宕的人生定当更加精彩 优先队列详解(转载)
如约而至,Java 10 正式发布! 3 月 20 日,Oracle 宣布 Java 10 正式发布. 官方已提供下载:http://www.oracle.com/technetwork/java ...
- Servlet3.0+springmvc5+log4j2正确的开启姿势(WebLookUp)
前言 java社区占据市场份额比较大的日志组件由log4j 1.×,到logback,再到整合后的升级版 log4j 2.×,有网友测试后据说log4j2的性能最NB.于是开始往自己的springmv ...
- 论consul正确的关闭姿势
最近在工作中发现一个有意思的现象,我用 ctrl+c 关闭本地 consul 的时候,报警系统并没有发出告警,说我的 node 异常,自己看了一下代码,发现 consul 的关闭还是有点猫腻的,仔细来 ...
- Fragment全解析系列（二）：正确的使用姿势
作为一个稳定的app,从后台且回到前台,一定会在任何情况都能恢复到离开前的页面,并且保证数据的完整性. 如果你没看过本系列的第一篇,为了方便后面文章的介绍,先规定一个"术语",安卓 ...
- Log.isLoggable之一正确的使用姿势
DEBUG方法比较当我们在做APP开发时,需要设置调试开关打印Log,下面我列举出3种方法: 方法一:直接赋值public static final boolean DEBUG = true;//fa ...
- How Javascript works (Javascript工作原理) (五) 深入理解 WebSockets 和带有 SSE 机制的HTTP/2 以及正确的使用姿势
个人总结: 1.长连接机制——分清Websocket,http2,SSE: 1)HTTP/2 引进了 Server Push 技术用来让服务器主动向客户端缓存发送数据.然而,它并不允许直接向客户端程序 ...
- Spring Boot程序正确停止的姿势
Spring Boot提供了2种优雅关闭进程的方式: 基于管理端口关闭进程 基于系统服务方式关闭进程 基于管理端口关闭进程 基于管理端口方式实现进程关闭实际上是模块spring-boot-actuat ...
- Spring+SpringMVC+MyBatis+easyUI整合进阶篇（十四）Redis缓存正确的使用姿势
作者:13 GitHub:https://github.com/ZHENFENG13 版权声明:本文为原创文章,未经允许不得转载. 简介 这是一篇关于Redis使用的总结类型文章,会先简单的谈一下缓存 ...
- 正确的上网姿势:ubuntu18.04使用clash
本文为本人将CFW(Clash For Windows)上的配置文件应用到ubuntu上面的操作备忘,仅供个人使用 首先下载已经打包的clash压缩包:https://github.com/Dream ...
随机推荐
- [poj 2453] An Easy Problem
An Easy Problem Time Limit: 1000MS Memory Limit: 65536K Total Submissions: 8371 Accepted: 5009 D ...
- [CodeForces - 447D] D - DZY Loves Modification
D - DZY Loves Modification As we know, DZY loves playing games. One day DZY decided to play with a n ...
- 利用NPOI解析Excel的通用类
using System.Collections.Generic; using System.Data; using System.IO; using System.Linq; using NPOI. ...
- 百度AI认为最漂亮的中国女星是----范冰冰
一.程序说明 1.1 程序说明 之前写调用百度AI接口的程序,然后刷到了两条明星的新闻,就想到了写个给明星颜值排下名的程序. 程序的关键点是两个,第一个是百度AI接口的调用这点其实直接使用早前实现的类 ...
- 微信订阅号，获取用户openid
在微信后台,启用服务器配置. 服务器URL地址,要通过Token的验证. private void Auth() { string token = ConfigurationManager.AppSe ...
- Java并发编程（五）JVM指令重排
我是不是学了一门假的java...... 引言:在Java中看似顺序的代码在JVM中,可能会出现编译器或者CPU对这些操作指令进行了重新排序:在特定情况下,指令重排将会给我们的程序带来不确定的结果.. ...
- 【MVC】快速构建一个图片浏览网站
当抄完MusicStore时,你应该对MVC有一个比较清晰的认识了.接下来就需要做个网站来继续增加自己的知识了.那么,该做个什么网站呢.做个图片浏览网站吧,简单而实用. 简单设计 1.首先,页面中间是 ...
- C++ Templates 关于程序库的概念和通用工具
using namespace std所谓的命名空间,就是一种将程序库名称封装起来的方法,它就像在程序库中竖立了一道围墙 标准程序库中有一部分,比如string classes,支持具体的错误处理,它 ...
- (C/C++学习笔记) 二十三. 运行时类型识别
二十三. 运行时类型识别 ● 定义 运行时类型识别(Run-time Type Identification, RTTI) 通过RTTI, 程序能够使用基类的指针或引用来检查(check)这些指针或引 ...
- jdk8--stream并行流
stream的并行流要理解一个框架如下: 单线程,多线程和并行流对比 package com.atguigu.java8; import java.util.concurrent.ForkJoinPo ...