HttpClient(4.5.x)正确的使用姿势
前言:
httpclient(4.5.x)默认是启动连接池的, 其降低时耗(避免连接初3次握手, 以及关闭4次握手的消耗), 显著提升高并发处理能力(大量减少time_wait), 确实扮演了重要的角色. 但是封装httpclient, 需要了解不少细节, 还要根据业务合理配置参数.
这里结合这段时间深入httpclient(4.5.x)源码分析, 结合网上的代码案例, 以及线下测试的结果. 尝试写一个可用的的httpclient封装类, 兼顾性能, 接口友好. 感谢cctv, ^_^.
相关文章:
1. HttpClient官方sample代码的深入分析(连接池)
第一版本:
啥也不说了, 直接贴代码了.
public class PooledHttpClientAdaptor {
private static final int DEFAULT_POOL_MAX_TOTAL = 200;
private static final int DEFAULT_POOL_MAX_PER_ROUTE = 200;
private static final int DEFAULT_CONNECT_TIMEOUT = 500;
private static final int DEFAULT_CONNECT_REQUEST_TIMEOUT = 500;
private static final int DEFAULT_SOCKET_TIMEOUT = 2000;
private PoolingHttpClientConnectionManager gcm = null;
private CloseableHttpClient httpClient = null;
// 连接池的最大连接数
private final int maxTotal;
// 连接池按route配置的最大连接数
private final int maxPerRoute;
// tcp connect的超时时间
private final int connectTimeout;
// 从连接池获取连接的超时时间
private final int connectRequestTimeout;
// tcp io的读写超时时间
private final int socketTimeout;
public PooledHttpClientAdaptor() {
this(
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_TOTAL,
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_PER_ROUTE,
PooledHttpClientAdaptor.DEFAULT_CONNECT_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_CONNECT_REQUEST_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_SOCKET_TIMEOUT
);
}
public PooledHttpClientAdaptor(
int maxTotal,
int maxPerRoute,
int connectTimeout,
int connectRequestTimeout,
int socketTimeout
) {
this.maxTotal = maxTotal;
this.maxPerRoute = maxPerRoute;
this.connectTimeout = connectTimeout;
this.connectRequestTimeout = connectRequestTimeout;
this.socketTimeout = socketTimeout;
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
gcm = new PoolingHttpClientConnectionManager(registry);
gcm.setMaxTotal(this.maxTotal);
gcm.setDefaultMaxPerRoute(this.maxPerRoute);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectTimeout) // 设置连接超时
.setSocketTimeout(this.socketTimeout) // 设置读取超时
.setConnectionRequestTimeout(this.connectRequestTimeout) // 设置从连接池获取连接实例的超时
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClient = httpClientBuilder
.setConnectionManager(gcm)
.setDefaultRequestConfig(requestConfig)
.build();
}
public String doGet(String url) {
return this.doGet(url, Collections.EMPTY_MAP, Collections.EMPTY_MAP);
}
public String doGet(String url, Map<String, Object> params) {
return this.doGet(url, Collections.EMPTY_MAP, params);
}
public String doGet(String url,
Map<String, String> headers,
Map<String, Object> params
) {
// *) 构建GET请求头
String apiUrl = getUrlWithParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpGet);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
public String doPost(String apiUrl, Map<String, Object> params) {
return this.doPost(apiUrl, Collections.EMPTY_MAP, params);
}
public String doPost(String apiUrl,
Map<String, String> headers,
Map<String, Object> params
) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求headers
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpPost.addHeader(entry.getKey(), entry.getValue());
}
}
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpPost);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if ( entityRes != null ) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if (response != null) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
private HttpEntity getUrlEncodedFormEntity(Map<String, Object> params) {
List<NameValuePair> pairList = new ArrayList<NameValuePair>(params.size());
for (Map.Entry<String, Object> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue().toString());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
private String getUrlWithParams(String url, Map<String, Object> params) {
boolean first = true;
StringBuilder sb = new StringBuilder(url);
for (String key : params.keySet()) {
char ch = '&';
if (first == true) {
ch = '?';
first = false;
}
String value = params.get(key).toString();
try {
String sval = URLEncoder.encode(value, "UTF-8");
sb.append(ch).append(key).append("=").append(sval);
} catch (UnsupportedEncodingException e) {
}
}
return sb.toString();
}
}
存在问题&解决方案
这个版本基本没啥问题, 但是当流量为0时, 你会发现存在处于ClOSE_WAIT的连接. 究其原因是, httpclient清理过期/被动关闭的socket, 是采用懒惰清理的策略. 它是在连接从连接池取出使用的时候, 检测状态并做相应处理. 如果没有流量, 那这些socket将一直处于CLOSE_WAIT(半连接的状态), 系统资源被浪费.
不过解决方案也相当的简单, 官方的建议引入一个清理线程, 定期主动处理过期/空闲连接, 这样就OK了.
private class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean exitFlag = false;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
this.connMgr = connMgr;
setDaemon(true);
}
@Override
public void run() {
while (!this.exitFlag) {
synchronized (this) {
try {
this.wait(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 关闭失效的连接
connMgr.closeExpiredConnections();
// 可选的, 关闭30秒内不活动的连接
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
public void shutdown() {
this.exitFlag = true;
synchronized (this) {
notify();
}
}
}
最终版本
直接整合贴代码, ^_^.
public class PooledHttpClientAdaptor {
private static final int DEFAULT_POOL_MAX_TOTAL = 200;
private static final int DEFAULT_POOL_MAX_PER_ROUTE = 200;
private static final int DEFAULT_CONNECT_TIMEOUT = 500;
private static final int DEFAULT_CONNECT_REQUEST_TIMEOUT = 500;
private static final int DEFAULT_SOCKET_TIMEOUT = 2000;
private PoolingHttpClientConnectionManager gcm = null;
private CloseableHttpClient httpClient = null;
private IdleConnectionMonitorThread idleThread = null;
// 连接池的最大连接数
private final int maxTotal;
// 连接池按route配置的最大连接数
private final int maxPerRoute;
// tcp connect的超时时间
private final int connectTimeout;
// 从连接池获取连接的超时时间
private final int connectRequestTimeout;
// tcp io的读写超时时间
private final int socketTimeout;
public PooledHttpClientAdaptor() {
this(
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_TOTAL,
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_PER_ROUTE,
PooledHttpClientAdaptor.DEFAULT_CONNECT_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_CONNECT_REQUEST_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_SOCKET_TIMEOUT
);
}
public PooledHttpClientAdaptor(
int maxTotal,
int maxPerRoute,
int connectTimeout,
int connectRequestTimeout,
int socketTimeout
) {
this.maxTotal = maxTotal;
this.maxPerRoute = maxPerRoute;
this.connectTimeout = connectTimeout;
this.connectRequestTimeout = connectRequestTimeout;
this.socketTimeout = socketTimeout;
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
this.gcm = new PoolingHttpClientConnectionManager(registry);
this.gcm.setMaxTotal(this.maxTotal);
this.gcm.setDefaultMaxPerRoute(this.maxPerRoute);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectTimeout) // 设置连接超时
.setSocketTimeout(this.socketTimeout) // 设置读取超时
.setConnectionRequestTimeout(this.connectRequestTimeout) // 设置从连接池获取连接实例的超时
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClient = httpClientBuilder
.setConnectionManager(this.gcm)
.setDefaultRequestConfig(requestConfig)
.build();
idleThread = new IdleConnectionMonitorThread(this.gcm);
idleThread.start();
}
public String doGet(String url) {
return this.doGet(url, Collections.EMPTY_MAP, Collections.EMPTY_MAP);
}
public String doGet(String url, Map<String, Object> params) {
return this.doGet(url, Collections.EMPTY_MAP, params);
}
public String doGet(String url,
Map<String, String> headers,
Map<String, Object> params
) {
// *) 构建GET请求头
String apiUrl = getUrlWithParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpGet);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
public String doPost(String apiUrl, Map<String, Object> params) {
return this.doPost(apiUrl, Collections.EMPTY_MAP, params);
}
public String doPost(String apiUrl,
Map<String, String> headers,
Map<String, Object> params
) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求headers
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpPost.addHeader(entry.getKey(), entry.getValue());
}
}
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpPost);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if ( entityRes != null ) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if (response != null) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
private HttpEntity getUrlEncodedFormEntity(Map<String, Object> params) {
List<NameValuePair> pairList = new ArrayList<NameValuePair>(params.size());
for (Map.Entry<String, Object> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue().toString());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
private String getUrlWithParams(String url, Map<String, Object> params) {
boolean first = true;
StringBuilder sb = new StringBuilder(url);
for (String key : params.keySet()) {
char ch = '&';
if (first == true) {
ch = '?';
first = false;
}
String value = params.get(key).toString();
try {
String sval = URLEncoder.encode(value, "UTF-8");
sb.append(ch).append(key).append("=").append(sval);
} catch (UnsupportedEncodingException e) {
}
}
return sb.toString();
}
public void shutdown() {
idleThread.shutdown();
}
// 监控有异常的链接
private class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean exitFlag = false;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
this.connMgr = connMgr;
setDaemon(true);
}
@Override
public void run() {
while (!this.exitFlag) {
synchronized (this) {
try {
this.wait(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 关闭失效的连接
connMgr.closeExpiredConnections();
// 可选的, 关闭30秒内不活动的连接
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
public void shutdown() {
this.exitFlag = true;
synchronized (this) {
notify();
}
}
}
}
总结:
其实没啥难道, 主要就是买个安心, 这样写是安全的, 是经得起线上考验的.
HttpClient(4.5.x)正确的使用姿势的更多相关文章
- 如约而至，Java 10 正式发布！ Spring+SpringMVC+MyBatis+easyUI整合进阶篇（十四）Redis缓存正确的使用姿势 努力的孩子运气不会太差，跌宕的人生定当更加精彩 优先队列详解(转载)
如约而至,Java 10 正式发布! 3 月 20 日,Oracle 宣布 Java 10 正式发布. 官方已提供下载:http://www.oracle.com/technetwork/java ...
- Servlet3.0+springmvc5+log4j2正确的开启姿势(WebLookUp)
前言 java社区占据市场份额比较大的日志组件由log4j 1.×,到logback,再到整合后的升级版 log4j 2.×,有网友测试后据说log4j2的性能最NB.于是开始往自己的springmv ...
- 论consul正确的关闭姿势
最近在工作中发现一个有意思的现象,我用 ctrl+c 关闭本地 consul 的时候,报警系统并没有发出告警,说我的 node 异常,自己看了一下代码,发现 consul 的关闭还是有点猫腻的,仔细来 ...
- Fragment全解析系列（二）：正确的使用姿势
作为一个稳定的app,从后台且回到前台,一定会在任何情况都能恢复到离开前的页面,并且保证数据的完整性. 如果你没看过本系列的第一篇,为了方便后面文章的介绍,先规定一个"术语",安卓 ...
- Log.isLoggable之一正确的使用姿势
DEBUG方法比较当我们在做APP开发时,需要设置调试开关打印Log,下面我列举出3种方法: 方法一:直接赋值public static final boolean DEBUG = true;//fa ...
- How Javascript works (Javascript工作原理) (五) 深入理解 WebSockets 和带有 SSE 机制的HTTP/2 以及正确的使用姿势
个人总结: 1.长连接机制——分清Websocket,http2,SSE: 1)HTTP/2 引进了 Server Push 技术用来让服务器主动向客户端缓存发送数据.然而,它并不允许直接向客户端程序 ...
- Spring Boot程序正确停止的姿势
Spring Boot提供了2种优雅关闭进程的方式: 基于管理端口关闭进程 基于系统服务方式关闭进程 基于管理端口关闭进程 基于管理端口方式实现进程关闭实际上是模块spring-boot-actuat ...
- Spring+SpringMVC+MyBatis+easyUI整合进阶篇（十四）Redis缓存正确的使用姿势
作者:13 GitHub:https://github.com/ZHENFENG13 版权声明:本文为原创文章,未经允许不得转载. 简介 这是一篇关于Redis使用的总结类型文章,会先简单的谈一下缓存 ...
- 正确的上网姿势:ubuntu18.04使用clash
本文为本人将CFW(Clash For Windows)上的配置文件应用到ubuntu上面的操作备忘,仅供个人使用 首先下载已经打包的clash压缩包:https://github.com/Dream ...
随机推荐
- git设置远程同步分支
git push --set-upstream origin yourBranchName
- Xshell中文乱码怎么处理？
改成如下图:
- JavaScript中var和this定义变量的区别
JavaScript中var和this定义变量的区别 在js中声明变量时可以使用var和this,但使用this的有很大一部分参考书是没有的,经过查阅相关资料总结如下: 用var和this声明变量,存 ...
- 【转】Vue中mintui的field实现blur和focus事件
首先上代码说总结: <mt-field label="卡号" v-model="card.cardNo" @blur.native.capture=&qu ...
- ActiveMQ 配置jdbc主从
使用 jdbc 方式配置主从模式,持久化消息存放在数据库中. 在同一时刻,只有一个 master broker,master 接受客户端的连接,slave 不接受连接.当 master 因为关机而下线 ...
- a标签在编辑器中可以整体删除并且a标签为不可编辑的情况下 标签依然存在(棒棒哒)
a标签在编辑器中可以整体删除并且a标签为不可编辑的情况下 标签依然存在 因为给a标签的后面 添加了一个空元素,如<i></i>(棒棒哒)<div contentEdita ...
- get url img
selenium 1● 了解selenium 这个是做 web页面测试,模拟用户测试 =====> pip install selenium 安装 2● django 用于接 ...
- select * from dim.dim_area_no@to_dw
应该是建的有database linksdim是用户名,dim_area_no是表名,to_dw 是建的database links的名,dim_area_no表属于dim用户创建database l ...
- TCP拥塞控制和流量控制
TCP 的流量控制与拥塞控制可以说是一体的.流量控制是通过滑动窗口实现的,拥塞避免主要包含以下2个内容: (1)慢开始,拥塞避免 (2)快重传,快恢复 1.流量控制——滑动窗口 TCP采用大小可变 ...
- Python3虚拟环境安装:virtualenv、virtualenvwralpper
一:通过pip3(因python2和3共存,前文修改过pip3软连接,pip3可以安装到python3)安装virtualenv,pip3 install virtuale 二:建立虚拟环境文件目录 ...