SpringCloud：搭建基于Gateway的微服务网关（二）

0.代码

1.引入相关依赖

pom文件如下：

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"

         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <parent>

        <artifactId>OpenAPI</artifactId>

        <groupId>com.fdzang.microservice</groupId>

        <version>1.0-SNAPSHOT</version>

    </parent>

    <modelVersion>4.0.0</modelVersion>

    <artifactId>api-gateway</artifactId>

    <dependencies>
　　　　 <!-- 公共模块引入了web模块，会与gateway产生冲突，故排除 -->

        <dependency>

            <groupId>com.fdzang.microservice</groupId>

            <artifactId>api-common</artifactId>

            <version>1.0-SNAPSHOT</version>

            <exclusions>

                <exclusion>

                    <groupId>org.springframework.boot</groupId>

                    <artifactId>spring-boot-starter-web</artifactId>

                </exclusion>

            </exclusions>

        </dependency>

        <dependency>

            <groupId>org.projectlombok</groupId>

            <artifactId>lombok</artifactId>

        </dependency>

        <dependency>

            <groupId>org.apache.commons</groupId>

            <artifactId>commons-collections4</artifactId>

        </dependency>

        <dependency>

            <groupId>commons-io</groupId>

            <artifactId>commons-io</artifactId>

        </dependency>

        <dependency>

            <groupId>commons-codec</groupId>

            <artifactId>commons-codec</artifactId>

        </dependency>

        <dependency>

            <groupId>com.google.guava</groupId>

            <artifactId>guava</artifactId>

        </dependency>

        <dependency>

            <groupId>com.google.code.gson</groupId>

            <artifactId>gson</artifactId>

        </dependency>

        <dependency>

            <groupId>com.alibaba</groupId>

            <artifactId>fastjson</artifactId>

            <version>${fastjson.version}</version>

        </dependency>

　　　　 <!-- 引入gateway模块 -->　　　　

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-starter-gateway</artifactId>

            <version>${spring.cloud.starter.version}</version>

        </dependency>


　　　　 <!-- 引入eureka模块 -->

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>

            <version>${spring.cloud.starter.version}</version>

        </dependency>


　　　　　<!-- 引入openfeign模块,这里不要用feign，Springboot2.0已弃用 -->

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-starter-openfeign</artifactId>

            <version>${spring.cloud.starter.version}</version>

        </dependency>

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>

            <version>${spring.cloud.starter.version}</version>

        </dependency>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-configuration-processor</artifactId>

            <version>${spring.boot.version}</version>

            <optional>true</optional>

        </dependency>

    </dependencies>

</project>

2.配置Gateway

server:

  port: 7000


#注册到eureka

eureka:

  client:

    service-url:

      defaultZone: http://127.0.0.1:7003/eureka/


#配置gateway拦截规则

spring:

  application:

    name: api-gateway

  cloud:

    gateway:

      discovery:

        locator:

          enabled: true

      routes:

        - id: gateway

          uri: http://www.baidu.com

          predicates:

            - Path=/**


#这里定义了鉴权的服务名，以及白名单

auth:

  service-id: api-auth-v1

  gateway:

    white:

      - /login

#这里是id生成器的配置，Twitter-Snowflake

IdWorker:

  workerId: 122

  datacenterId: 1231

3.过滤器

3.1.ID生成拦截

对每个请求生成一个唯一的请求id

package com.fdzang.microservice.gateway.gateway;

import com.fdzang.microservice.gateway.util.GatewayConstant;

import com.fdzang.microservice.gateway.util.IdWorker;

import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang.StringUtils;

import org.springframework.cloud.gateway.filter.GatewayFilterChain;

import org.springframework.cloud.gateway.filter.GlobalFilter;

import org.springframework.core.Ordered;

import org.springframework.http.server.reactive.ServerHttpRequest;

import org.springframework.stereotype.Component;

import org.springframework.web.server.ServerWebExchange;

import reactor.core.publisher.Mono;

/**

 * 生成一个请求的特定id

 * @author tanghu

 * @Date: 2019/11/5 18:42

 */

@Slf4j

@Component

public class SerialNoFilter implements GlobalFilter, Ordered {

    @Override

    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {

        ServerHttpRequest request = exchange.getRequest();

        String requestId= request.getHeaders().getFirst(GatewayConstant.REQUEST_TRACE_ID);

        if (StringUtils.isEmpty(requestId)) {

            Object attribute = exchange.getAttribute(GatewayConstant.REQUEST_TRACE_ID);

            if (attribute == null) {

                requestId = String.valueOf(IdWorker.getWorkerId());

                exchange.getAttributes().put(GatewayConstant.REQUEST_TRACE_ID,requestId);

            }

        }else{

            exchange.getAttributes().put(GatewayConstant.REQUEST_TRACE_ID,requestId);

        }

        return chain.filter(exchange);

    }

    @Override

    public int getOrder() {

        return GatewayConstant.Order.SERIAL_NO_ORDER;

    }

}

3.2.鉴权拦截

获取请求头中的鉴权信息，对信息校验，这里暂时没有做（AuthResult authService.auth(AuthRequest request)）,这里需求请求其他模块对请求信息进行校验，返回校验结果

package com.fdzang.microservice.gateway.gateway;

import com.fdzang.microservice.common.entity.auth.AuthCode;

import com.fdzang.microservice.common.entity.auth.AuthRequest;

import com.fdzang.microservice.common.entity.auth.AuthResult;

import com.fdzang.microservice.gateway.service.AuthService;

import com.fdzang.microservice.gateway.util.GatewayConstant;

import com.fdzang.microservice.gateway.util.WhiteUrl;

import com.google.common.base.Joiner;

import com.google.common.base.Splitter;

import lombok.extern.slf4j.Slf4j;

import org.apache.commons.collections4.MapUtils;

import org.apache.commons.lang.StringUtils;

import org.apache.commons.lang.math.NumberUtils;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.cloud.gateway.filter.GatewayFilterChain;

import org.springframework.cloud.gateway.filter.GlobalFilter;

import org.springframework.core.Ordered;

import org.springframework.http.HttpHeaders;

import org.springframework.http.server.reactive.ServerHttpRequest;

import org.springframework.stereotype.Component;

import org.springframework.util.CollectionUtils;

import org.springframework.util.MultiValueMap;

import org.springframework.web.server.ServerWebExchange;

import reactor.core.publisher.Mono;

import java.util.List;

import java.util.Map;

import java.util.TreeMap;

/**

 * 权限校验

 * @author tanghu

 * @Date: 2019/10/22 18:00

 */

@Slf4j

@Component

public class AuthFilter implements GlobalFilter, Ordered {

    @Autowired

    private AuthService authService;

    @Override

    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {

        String requestId = exchange.getAttribute(GatewayConstant.REQUEST_TRACE_ID);

        String url = exchange.getRequest().getURI().getPath();

        ServerHttpRequest request = exchange.getRequest();

        //跳过白名单

        if(null != WhiteUrl.getWhite() && WhiteUrl.getWhite().contains(url)){

            return chain.filter(exchange);

        }

        //获取权限校验部分

        //Authorization: gateway:{AccessId}:{Signature}

        String authHeader = exchange.getRequest().getHeaders().getFirst(GatewayConstant.AUTH_HEADER);

        if(StringUtils.isBlank(authHeader)){

            log.warn("request has no authorization header, uuid:{}, request:{}",requestId, url);

            throw new IllegalArgumentException("bad request");

        }

        List<String> auths = Splitter.on(":").trimResults().omitEmptyStrings().splitToList(authHeader);

        if(CollectionUtils.isEmpty(auths) || auths.size() != 3 || !GatewayConstant.AUTH_LABLE.equals(auths.get(0))){

            log.warn("bad authorization header, uuid:{}, request:[{}], header:{}",

                    requestId, url, authHeader);

            throw new IllegalArgumentException("bad request");

        }

        //校验时间戳是否合法

        String timestamp = exchange.getRequest().getHeaders().getFirst(GatewayConstant.TIMESTAMP_HEADER);

        if (StringUtils.isBlank(timestamp) || isTimestampExpired(timestamp)) {

            log.warn("wrong timestamp:{}, uuid:{}, request:{}",

                    timestamp, requestId, url);

        }

        String accessId = auths.get(1);

        String sign = auths.get(2);

        String stringToSign = getStringToSign(request, timestamp);

        AuthRequest authRequest = new AuthRequest();

        authRequest.setAccessId(accessId);

        authRequest.setSign(sign);

        authRequest.setStringToSign(stringToSign);

        authRequest.setHttpMethod(request.getMethodValue());

        authRequest.setUri(url);

        AuthResult authResult = authService.auth(authRequest);

        if (authResult.getStatus() != AuthCode.SUCEESS.getAuthCode()) {

            log.warn("checkSign failed, uuid:{},  accessId:{}, request:[{}], error:{}",

                    requestId, accessId, url, authResult.getDescription());

            throw new RuntimeException(authResult.getDescription());

        }

        log.info("request auth finished, uuid:{}, orgCode:{}, userName:{}, accessId:{}, request:{}, serviceName:{}",

                requestId, authResult.getOrgCode(),

                authResult.getUsername(), accessId,

                url, authResult.getServiceName());

        exchange.getAttributes().put(GatewayConstant.SERVICE_NAME,authResult.getServiceName());

        return chain.filter(exchange);

    }

    /**

     * 获取原始字符串（签名前）

     * @param request

     * @param timestamp

     * @return

     */

    private String getStringToSign(ServerHttpRequest request, String timestamp){

        // headers

        TreeMap<String, String> headersInSign = new TreeMap<>();

        HttpHeaders headers = request.getHeaders();

        for (Map.Entry<String,List<String>> header:headers.entrySet()) {

            String key = header.getKey();

            if (key.startsWith(GatewayConstant.AUTH_HEADER_PREFIX)) {

                headersInSign.put(key, header.getValue().get(0));

            }

        }

        StringBuilder headerStringBuilder = new StringBuilder();

        for (Map.Entry<String, String> entry : headersInSign.entrySet()) {

            headerStringBuilder.append(entry.getKey()).append(":").append(entry.getValue()).append("\n");

        }

        String headerString = null;

        if (headerStringBuilder.length() != 0) {

            headerString = headerStringBuilder.deleteCharAt(headerStringBuilder.length()-1).toString();

        }

        // Url_String

        TreeMap<String, String> paramsInSign = new TreeMap<>();

        MultiValueMap<String, String> parameterMap = request.getQueryParams();

        if (MapUtils.isNotEmpty(parameterMap)) {

            for (Map.Entry<String, List<String>> entry : parameterMap.entrySet()) {

                paramsInSign.put(entry.getKey(), entry.getValue().get(0));

            }

        }

        // 原始url

        String originalUrl = request.getURI().getPath();

        StringBuilder uriStringBuilder = new StringBuilder(originalUrl);

        if (!parameterMap.isEmpty()) {

            uriStringBuilder.append("?");

            for (Map.Entry<String, String> entry : paramsInSign.entrySet()) {

                uriStringBuilder.append(entry.getKey());

                if (StringUtils.isNotBlank(entry.getValue())) {

                    uriStringBuilder.append("=").append(entry.getValue());

                }

                uriStringBuilder.append("&");

            }

            uriStringBuilder.deleteCharAt(uriStringBuilder.length()-1);

        }

        String uriString = uriStringBuilder.toString();

        String contentType = headers.getFirst(HttpHeaders.CONTENT_TYPE);

        //这里可以对请求参数进行MD5校验，暂时不做

        String contentMd5 = headers.getFirst(GatewayConstant.CONTENTE_MD5);

        String[] parts = {

                request.getMethodValue(),

                StringUtils.isNotBlank(contentMd5) ? contentMd5 : "",

                StringUtils.isNotBlank(contentType) ? contentType : "",

                timestamp,

                headerString,

                uriString

        };

        return Joiner.on(GatewayConstant.STRING_TO_SIGN_DELIM).skipNulls().join(parts);

    }

    /**

     * 校验时间戳是否超时

     * @param timestamp

     * @return

     */

    private boolean isTimestampExpired(String timestamp){

        long l = NumberUtils.toLong(timestamp, 0L);

        if (l == 0) {

            return true;

        }

        return Math.abs(System.currentTimeMillis() - l) > GatewayConstant.EXPIRE_TIME_SECONDS *1000;

    }

    @Override

    public int getOrder() {

        return GatewayConstant.Order.AUTH_ORDER;

    }

}

3.3.服务分发

根据鉴权后的结果能得到服务名，然后重写路由以及请求，对该次请求进行转发

package com.fdzang.microservice.gateway.gateway;

import com.fdzang.microservice.gateway.util.GatewayConstant;

import com.fdzang.microservice.gateway.util.WhiteUrl;

import com.google.common.base.Splitter;

import lombok.extern.slf4j.Slf4j;

import org.springframework.cloud.gateway.filter.GatewayFilterChain;

import org.springframework.cloud.gateway.filter.GlobalFilter;

import org.springframework.cloud.gateway.route.Route;

import org.springframework.cloud.gateway.support.ServerWebExchangeUtils;

import org.springframework.core.Ordered;

import org.springframework.http.server.reactive.ServerHttpRequest;

import org.springframework.stereotype.Component;

import org.springframework.web.server.ServerWebExchange;

import reactor.core.publisher.Mono;

import java.util.List;

/**

 * @author tanghu

 * @Date: 2019/11/6 15:39

 */

@Slf4j

@Component

public class ModifyRequestFilter implements GlobalFilter, Ordered {

    @Override

    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {

        String url = exchange.getRequest().getURI().getPath();

        ServerHttpRequest request = exchange.getRequest();

        //跳过白名单

        if(null != WhiteUrl.getWhite() && WhiteUrl.getWhite().contains(url)){

            return chain.filter(exchange);

        }

        String serviceName = exchange.getAttribute(GatewayConstant.SERVICE_NAME);

        //修改路由

        Route route = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_ROUTE_ATTR);

        Route newRoute = Route.async()

                .asyncPredicate(route.getPredicate())

                .filters(route.getFilters())

                .id(route.getId())

                .order(route.getOrder())

                .uri(GatewayConstant.URI.LOAD_BALANCE+serviceName).build();

        exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_ROUTE_ATTR,newRoute);

        //修改请求路径

        List<String> strings = Splitter.on("/").omitEmptyStrings().trimResults().limit(3).splitToList(url);

        String newServletPath = "/" + strings.get(2);

        ServerHttpRequest newRequest = request.mutate().path(newServletPath).build();

        return chain.filter(exchange.mutate().request(newRequest).build());

    }

    @Override

    public int getOrder() {

        return GatewayConstant.Order.MODIFY_REQUEST_ORDER;

    }

}

3.4.统一响应

对响应进行统一封装

package com.fdzang.microservice.gateway.gateway;

import com.alibaba.fastjson.JSON;

import com.fdzang.microservice.common.entity.ApiResult;

import com.fdzang.microservice.gateway.entity.GatewayError;

import com.fdzang.microservice.gateway.entity.GatewayResult;

import com.fdzang.microservice.gateway.entity.GatewayResultEnums;

import com.fdzang.microservice.gateway.util.GatewayConstant;

import lombok.extern.slf4j.Slf4j;

import org.reactivestreams.Publisher;

import org.springframework.cloud.gateway.filter.GatewayFilterChain;

import org.springframework.cloud.gateway.filter.GlobalFilter;

import org.springframework.cloud.gateway.filter.NettyWriteResponseFilter;

import org.springframework.core.Ordered;

import org.springframework.core.io.buffer.DataBuffer;

import org.springframework.core.io.buffer.DataBufferFactory;

import org.springframework.core.io.buffer.DataBufferUtils;

import org.springframework.http.server.reactive.ServerHttpResponse;

import org.springframework.http.server.reactive.ServerHttpResponseDecorator;

import org.springframework.stereotype.Component;

import org.springframework.web.server.ServerWebExchange;

import reactor.core.publisher.Flux;

import reactor.core.publisher.Mono;

import java.nio.charset.Charset;

/**

 * @author tanghu

 * @Date: 2019/11/7 8:58

 */

@Slf4j

@Component

public class ModifyResponseFilter implements GlobalFilter, Ordered {

    @Override

    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {

        String requestId = exchange.getAttribute(GatewayConstant.REQUEST_TRACE_ID);

        ServerHttpResponse originalResponse = exchange.getResponse();

        DataBufferFactory bufferFactory = originalResponse.bufferFactory();

        ServerHttpResponseDecorator decoratedResponse = new ServerHttpResponseDecorator(originalResponse) {

            @Override

            public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {

                if (body instanceof Flux) {

                    Flux<? extends DataBuffer> fluxBody = (Flux<? extends DataBuffer>) body;

                    return super.writeWith(fluxBody.map(dataBuffer -> {

                        byte[] content = new byte[dataBuffer.readableByteCount()];

                        dataBuffer.read(content);

                        //释放掉内存

                        DataBufferUtils.release(dataBuffer);

                        String originalbody = new String(content, Charset.forName("UTF-8"));

                        String finalBody = originalbody;

                        ApiResult apiResult = JSON.parseObject(originalbody,ApiResult.class);

                        GatewayResult result = new GatewayResult();

                        result.setCode(GatewayResultEnums.SUCC.getCode());

                        result.setMsg(GatewayResultEnums.SUCC.getMsg());

                        result.setReq_id(requestId);

                        if (apiResult.getCode() == null && apiResult.getMsg() == null) {

                            // 尝试解析body为网关的错误信息

                            GatewayError gatewayError = JSON.parseObject(originalbody,GatewayError.class);

                            result.setSub_code(gatewayError.getStatus());

                            result.setSub_msg(gatewayError.getMessage());

                        } else {

                            result.setSub_code(apiResult.getCode());

                            result.setSub_msg(apiResult.getMsg());

                        }

                        result.setData(apiResult.getData());

                        finalBody = JSON.toJSONString(result);

                        return bufferFactory.wrap(finalBody.getBytes());

                    }));

                }

                return super.writeWith(body);

            }

        };

        return chain.filter(exchange.mutate().response(decoratedResponse).build());

    }

    @Override

    public int getOrder() {

        return GatewayConstant.Order.MODIFY_RESPONSE_ORDER;

    }

}

4.测试

10:25:54.961 [main] INFO  c.f.microservice.mock.util.SignUtil - StringToSign:

GET

1573093554201

/v2/base/zuul/tag/getMostUsedTags?from=2017-11-25 00:00:00&plate_num=部A11110&to=2017-11-30 00:00:00

10:25:54.979 [main] INFO  c.f.microservice.mock.util.HttpUtil - sign:Y+usbpHlwOw4F2sq4b0pNjgXGDAXoYgs1syOOPxPFAE=

10:25:59.868 [main] INFO  com.fdzang.microservice.mock.Demo - {"code":0,"data":[{"tagPublishedRefCount":3,"tagTitle":"Solo","id":"1533101769023","tagReferenceCount":3},{"tagPublishedRefCount":1,"tagTitle":"tetet","id":"1559285894006","tagReferenceCount":1}],"msg":"succ","req_id":"2627469547766022144","sub_code":0,"sub_msg":"ok"}

Process finished with exit code 0

由返回结果，可知此次请求完成。

5.注意事项

转发的目标服务需要跟网关注册在同一个注册中心下，路由uri配置为 lb://service_name，则会转发到对应的服务下，并且gateway会自动采用负载均衡机制

响应请求的顺序需要小于 NettyWriteResponseFilter.WRITE_RESPONSE_FILTER_ORDER 该值为-1

其他拦截器的顺序无固定要求，值越小越先执行

SpringCloud：搭建基于Gateway的微服务网关（二）的更多相关文章

SpringCloud：搭建基于Gateway的微服务网关（一）
1.需求最近在尝试着写一个开放平台,于是先搭建网关. 作用:统一的请求入口,完成对请求的跟踪,限流(未做),鉴权,分发,封装响应 2.工作原理 2.1.请求在开放平台中申请对接口的使用,申请通过后 ...
小D课堂 - 新版本微服务springcloud+Docker教程_6-06 zuul微服务网关集群搭建
笔记 6.Zuul微服务网关集群搭建简介:微服务网关Zull集群搭建 1.nginx+lvs+keepalive https://www.cnblogs.com/liuyisai/ ...
【SpringCloud构建微服务系列】微服务网关Zuul
一.为什么要用微服务网关在微服务架构中,一般不同的微服务有不同的网络地址,而外部客户端(如手机APP)可能需要调用多个接口才能完成一次业务需求.例如一个电影购票的手机APP,可能会调用多个微服务的接 ...
基于SpringBoot-Dubbo的微服务快速开发框架
简介: 基于Dubbo的分布式/微服务基础框架,为前端提供脚手架开发服务,结合前一篇--Web AP快速开发基础框架,可快速上手基于Dubbo的分布式服务开发,项目代码: https://github ...
SpringCloud Gateway微服务网关实战与源码分析-上
概述定义 Spring Cloud Gateway 官网地址 https://spring.io/projects/spring-cloud-gateway/ 最新版本3.1.3 Spring Cl ...
springcloud(十四)：搭建Zuul微服务网关
springcloud(十四):搭建Zuul微服务网关 1. 2. 3. 4.
使用 Node.js 搭建微服务网关
目录 Node.js 是什么安装 node.js Node.js 入门 Node.js 应用场景 npm 镜像使用 Node.js 搭建微服务网关什么是微服务架构使用 Node.js 实现反向 ...
微服务网关 Spring Cloud Gateway
1. 为什么是Spring Cloud Gateway 一句话,Spring Cloud已经放弃Netflix Zuul了.现在Spring Cloud中引用的还是Zuul 1.x版本,而这个版本是 ...
微服务网关实战——Spring Cloud Gateway
导读作为Netflix Zuul的替代者,Spring Cloud Gateway是一款非常实用的微服务网关,在Spring Cloud微服务架构体系中发挥非常大的作用.本文对Spring Clou ...

随机推荐

分布式图片服务器FastDFS
1. 什么是FastDFS FastDFS 是用 c 语言编写的一款开源的分布式文件系统.FastDFS 为互联网量身定制,充分考虑了冗余备份.负载均衡.线性扩容等机制,并注重高可用.高性能等指标,使 ...
python爬虫User Agent用户代理
UserAgent简介 UserAgent中文名为用户代理,是Http协议中的一部分,属于头域的组成部分,UserAgent也简称UA.它是一个特殊字符串头,是一种向访问网站提供你所使用的浏览器类型及 ...
consul：connect
官方文档:https://www.consul.io/docs/connect/index.html#getting-started-with-connect consul connect的功能类似与 ...
python之栈与队列
这个在官网中list支持,有实现. 补充一下栈,队列的特性: 1.栈(stacks)是一种只能通过访问其一端来实现数据存储与检索的线性数据结构,具有后进先出(last in first out,LIF ...
jmeter针对websocket协议的压测
之前一直没有接触过websocket协议,所以一直对websocket的压测存在疑惑,在网上参考文章并不断尝试之后,终于有所得:第一次用jmeter的websoket插件,用的ws非加密协议,请求都能 ...
js--同步运动json下
这一节针对上一节讲述的bug,我们来处理一下. 这个bug存在的原因就是,一旦只要有一个属性值达到目标值就会清除定时器,所以我们要改变的就是清除定时器的那么部分.看下面的修改 var timer; ...
[原创]VBA实现汇总excel，将多个Excel文件内容复制到一个Excel文件中
功能:遍历用户指定的文件夹,把文件夹中所有的excel文件的第一个表格的数据复制到本excel文件中.注意,每个excel文件中有效数据行的判断标准是A列的最后一个有数据的单元格的行号,比如A列到第1 ...
lxml获取结点属性整并转换为字典参数
#-*- coding: UTF- -*- from lxml import etree source = u''' <div><p ">测试数据1</p> ...
导入Excel——解析Excel——优化
package com.it.excel.excelLearn; import java.io.FileInputStream; import java.io.IOException; import ...
sparkjob的提交流程
在使用spark-submit提交一个Spark应用之后,Driver程序会向集群申请一定的资源来启动东若干个Executors用来计算,当这些Executors启动后,它们会向Driver端的Sch ...