因为需要研究下断点上传的问题。找了很久终于找到一个比较好的项目。

效果：

上传中，显示进度，时间，百分比。

点击【Pause】暂停，点击【Resume】继续。

2，代码分析

项目进行了封装使用最简单的方法实现了http的断点上传。

因为html5 里面有读取文件分割文件的类库，所以才可以支持断点上传，所以这个只能在html5 支持的浏览器上面展示。

同时，在js 和 java 同时使用 cr32 进行文件块的校验，保证数据上传正确。

代码在使用了最新的servlet 3.0 的api，使用了异步执行，监听等方法。

上传类UploadServlet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
@Component("javaLargeFileUploaderServlet") 
@WebServlet(name = "javaLargeFileUploaderServlet", urlPatterns = { "/javaLargeFileUploaderServlet" }) 
public class UploadServlet extends HttpRequestHandlerServlet 
        implements HttpRequestHandler { 
   
    private static final Logger log = LoggerFactory.getLogger(UploadServlet.class); 
   
    @Autowired 
    UploadProcessor uploadProcessor; 
   
    @Autowired 
    FileUploaderHelper fileUploaderHelper; 
   
    @Autowired 
    ExceptionCodeMappingHelper exceptionCodeMappingHelper; 
   
    @Autowired 
    Authorizer authorizer; 
   
    @Autowired 
    StaticStateIdentifierManager staticStateIdentifierManager; 
   
   
   
    @Override 
    public void handleRequest(HttpServletRequest request, HttpServletResponse response) 
            throws IOException { 
        log.trace("Handling request"); 
   
        Serializable jsonObject = null; 
        try { 
            // extract the action from the request 
            UploadServletAction actionByParameterName = 
                    UploadServletAction.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.action)); 
   
            // check authorization 
            checkAuthorization(request, actionByParameterName); 
   
            // then process the asked action 
            jsonObject = processAction(actionByParameterName, request); 
   
   
            // if something has to be written to the response 
            if (jsonObject != null) { 
                fileUploaderHelper.writeToResponse(jsonObject, response); 
            } 
   
        } 
        // If exception, write it 
        catch (Exception e) { 
            exceptionCodeMappingHelper.processException(e, response); 
        } 
   
    } 
   
   
    private void checkAuthorization(HttpServletRequest request, UploadServletAction actionByParameterName) 
            throws MissingParameterException, AuthorizationException { 
   
        // check authorization 
        // if its not get progress (because we do not really care about authorization for get 
        // progress and it uses an array of file ids) 
        if (!actionByParameterName.equals(UploadServletAction.getProgress)) { 
   
            // extract uuid 
            final String fileIdFieldValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId, false); 
   
            // if this is init, the identifier is the one in parameter 
            UUID clientOrJobId; 
            String parameter = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false); 
            if (actionByParameterName.equals(UploadServletAction.getConfig) && parameter != null) { 
                clientOrJobId = UUID.fromString(parameter); 
            } 
            // if not, get it from manager 
            else { 
                clientOrJobId = staticStateIdentifierManager.getIdentifier(); 
            } 
   
               
            // call authorizer 
            authorizer.getAuthorization( 
                    request, 
                    actionByParameterName, 
                    clientOrJobId, 
                    fileIdFieldValue != null ? getFileIdsFromString(fileIdFieldValue).toArray(new UUID[] {}) : null); 
   
        } 
    } 
   
   
    private Serializable processAction(UploadServletAction actionByParameterName, HttpServletRequest request) 
            throws Exception { 
        log.debug("Processing action " + actionByParameterName.name()); 
   
        Serializable returnObject = null; 
        switch (actionByParameterName) { 
            case getConfig: 
                String parameterValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false); 
                returnObject = 
                        uploadProcessor.getConfig( 
                                parameterValue != null ? UUID.fromString(parameterValue) : null); 
                break; 
            case verifyCrcOfUncheckedPart: 
                returnObject = verifyCrcOfUncheckedPart(request); 
                break; 
            case prepareUpload: 
                returnObject = prepareUpload(request); 
                break; 
            case clearFile: 
                uploadProcessor.clearFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId))); 
                break; 
            case clearAll: 
                uploadProcessor.clearAll(); 
                break; 
            case pauseFile: 
                List<UUID> uuids = getFileIdsFromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)); 
                uploadProcessor.pauseFile(uuids); 
                break; 
            case resumeFile: 
                returnObject = 
                        uploadProcessor.resumeFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId))); 
                break; 
            case setRate: 
                uploadProcessor.setUploadRate(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)), 
                        Long.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.rate))); 
                break; 
            case getProgress: 
                returnObject = getProgress(request); 
                break; 
        } 
        return returnObject; 
    } 
   
   
    List<UUID> getFileIdsFromString(String fileIds) { 
        String[] splittedFileIds = fileIds.split(","); 
        List<UUID> uuids = Lists.newArrayList(); 
        for (int i = 0; i < splittedFileIds.length; i++) { 
            uuids.add(UUID.fromString(splittedFileIds[i])); 
        }  
        return uuids; 
    } 
   
   
    private Serializable getProgress(HttpServletRequest request) 
            throws MissingParameterException { 
        Serializable returnObject; 
        String[] ids = 
                new Gson() 
                        .fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId), String[].class); 
        Collection<UUID> uuids = Collections2.transform(Arrays.asList(ids), new Function<String, UUID>() { 
   
            @Override 
            public UUID apply(String input) { 
                return UUID.fromString(input); 
            } 
   
        }); 
        returnObject = Maps.newHashMap(); 
        for (UUID fileId : uuids) { 
            try { 
                ProgressJson progress = uploadProcessor.getProgress(fileId); 
                ((HashMap<String, ProgressJson>) returnObject).put(fileId.toString(), progress); 
            } 
            catch (FileNotFoundException e) { 
                log.debug("No progress will be retrieved for " + fileId + " because " + e.getMessage()); 
            } 
        } 
        return returnObject; 
    } 
   
   
    private Serializable prepareUpload(HttpServletRequest request) 
            throws MissingParameterException, IOException { 
   
        // extract file information 
        PrepareUploadJson[] fromJson = 
                new Gson() 
                        .fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.newFiles), PrepareUploadJson[].class); 
   
        // prepare them 
        final HashMap<String, UUID> prepareUpload = uploadProcessor.prepareUpload(fromJson); 
   
        // return them 
        return Maps.newHashMap(Maps.transformValues(prepareUpload, new Function<UUID, String>() { 
   
            public String apply(UUID input) { 
                return input.toString(); 
            }; 
        })); 
    } 
   
   
    private Boolean verifyCrcOfUncheckedPart(HttpServletRequest request) 
            throws IOException, MissingParameterException, FileCorruptedException, FileStillProcessingException { 
        UUID fileId = UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)); 
        try { 
            uploadProcessor.verifyCrcOfUncheckedPart(fileId, 
                    fileUploaderHelper.getParameterValue(request, UploadServletParameter.crc)); 
        } 
        catch (InvalidCrcException e) { 
            // no need to log this exception, a fallback behaviour is defined in the 
            // throwing method. 
            // but we need to return something! 
            return Boolean.FALSE; 
        } 
        return Boolean.TRUE; 
    } 
}

异步上传UploadServletAsync

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
@Component("javaLargeFileUploaderAsyncServlet") 
@WebServlet(name = "javaLargeFileUploaderAsyncServlet", urlPatterns = { "/javaLargeFileUploaderAsyncServlet" }, asyncSupported = true) 
public class UploadServletAsync extends HttpRequestHandlerServlet 
        implements HttpRequestHandler { 
   
    private static final Logger log = LoggerFactory.getLogger(UploadServletAsync.class); 
   
    @Autowired 
    ExceptionCodeMappingHelper exceptionCodeMappingHelper; 
   
    @Autowired 
    UploadServletAsyncProcessor uploadServletAsyncProcessor; 
       
    @Autowired 
    StaticStateIdentifierManager staticStateIdentifierManager; 
   
    @Autowired 
    StaticStateManager<StaticStatePersistedOnFileSystemEntity> staticStateManager; 
   
    @Autowired 
    FileUploaderHelper fileUploaderHelper; 
   
    @Autowired 
    Authorizer authorizer; 
   
    /**
     * Maximum time that a streaming request can take.<br>
     */ 
    private long taskTimeOut = DateUtils.MILLIS_PER_HOUR; 
   
   
    @Override 
    public void handleRequest(final HttpServletRequest request, final HttpServletResponse response) 
            throws ServletException, IOException { 
   
        // process the request 
        try { 
   
            //check if uploads are allowed 
            if (!uploadServletAsyncProcessor.isEnabled()) { 
                throw new UploadIsCurrentlyDisabled(); 
            } 
               
            // extract stuff from request 
            final FileUploadConfiguration process = fileUploaderHelper.extractFileUploadConfiguration(request); 
   
            log.debug("received upload request with config: "+process); 
   
            // verify authorization 
            final UUID clientId = staticStateIdentifierManager.getIdentifier(); 
            authorizer.getAuthorization(request, UploadServletAction.upload, clientId, process.getFileId()); 
   
            //check if that file is not paused 
            if (uploadServletAsyncProcessor.isFilePaused(process.getFileId())) { 
                log.debug("file "+process.getFileId()+" is paused, ignoring async request."); 
                return; 
            } 
               
            // get the model 
            StaticFileState fileState = staticStateManager.getEntityIfPresent().getFileStates().get(process.getFileId()); 
            if (fileState == null) { 
                throw new FileNotFoundException("File with id " + process.getFileId() + " not found"); 
            } 
   
            // process the request asynchronously 
            final AsyncContext asyncContext = request.startAsync(); 
            asyncContext.setTimeout(taskTimeOut); 
   
   
            // add a listener to clear bucket and close inputstream when process is complete or 
            // with 
            // error 
            asyncContext.addListener(new UploadServletAsyncListenerAdapter(process.getFileId()) { 
   
                @Override 
                void clean() { 
                    log.debug("request " + request + " completed."); 
                    // we do not need to clear the inputstream here. 
                    // and tell processor to clean its shit! 
                    uploadServletAsyncProcessor.clean(clientId, process.getFileId()); 
                } 
            }); 
   
            // then process 
            uploadServletAsyncProcessor.process(fileState, process.getFileId(), process.getCrc(), process.getInputStream(), 
                    new WriteChunkCompletionListener() { 
   
                        @Override 
                        public void success() { 
                            asyncContext.complete(); 
                        } 
   
   
                        @Override 
                        public void error(Exception exception) { 
                            // handles a stream ended unexpectedly , it just means the user has 
                            // stopped the 
                            // stream 
                            if (exception.getMessage() != null) { 
                                if (exception.getMessage().equals("Stream ended unexpectedly")) { 
                                    log.warn("User has stopped streaming for file " + process.getFileId()); 
                                } 
                                else if (exception.getMessage().equals("User cancellation")) { 
                                    log.warn("User has cancelled streaming for file id " + process.getFileId()); 
                                    // do nothing 
                                } 
                                else { 
                                    exceptionCodeMappingHelper.processException(exception, response); 
                                } 
                            } 
                            else { 
                                exceptionCodeMappingHelper.processException(exception, response); 
                            } 
   
                            asyncContext.complete(); 
                        } 
   
                    }); 
        } 
        catch (Exception e) { 
            exceptionCodeMappingHelper.processException(e, response); 
        } 
   
    } 
   
}

3，请求流程图：

主要思路就是将文件切分，然后分块上传。

DEMO下载地址：https://dwz.cn/fgXtRtnu

java http大文件断点续传上传的更多相关文章

  1. java springboot 大文件分片上传处理

    参考自:https://blog.csdn.net/u014150463/article/details/74044467 这里只写后端的代码,基本的思想就是,前端将文件分片,然后每次访问上传接口的时 ...

  2. Ajax+Java实现大文件切割上传

    技术体系:html5(formdata) + java + servlet3.0+maven + tomcat7 <!DOCTYPE html> <html> <head ...

  3. 关于：基于http协议大文件断点续传上传至web服务器

    关键部分 前端用file.slice()分块 前端用FileReader获取每一分块的md5值 后端用MultipartFile接受分块文件 后端用FileOutputStream拼装分块文件 话不多 ...

  4. iOS大文件分片上传和断点续传

    总结一下大文件分片上传和断点续传的问题.因为文件过大(比如1G以上),必须要考虑上传过程网络中断的情况.http的网络请求中本身就已经具备了分片上传功能,当传输的文件比较大时,http协议自动会将文件 ...

  5. js大文件分块上传断点续传demo

    文件夹上传:从前端到后端 文件上传是 Web 开发肯定会碰到的问题,而文件夹上传则更加难缠.网上关于文件夹上传的资料多集中在前端,缺少对于后端的关注,然后讲某个后端框架文件上传的文章又不会涉及文件夹. ...

  6. 使用webuploader组件实现大文件分片上传，断点续传

    本人在2010年时使用swfupload为核心进行文件的批量上传的解决方案.见文章:WEB版一次选择多个文件进行批量上传(swfupload)的解决方案. 本人在2013年时使用plupload为核心 ...

  7. vue大文件分片上传插件

    最近遇见一个需要上传百兆大文件的需求,调研了七牛和腾讯云的切片分段上传功能,因此在此整理前端大文件上传相关功能的实现. 在某些业务中,大文件上传是一个比较重要的交互场景,如上传入库比较大的Excel表 ...

  8. Webuploader 大文件分片上传

    百度Webuploader 大文件分片上传(.net接收)   前阵子要做个大文件上传的功能,找来找去发现Webuploader还不错,关于她的介绍我就不再赘述. 动手前,在园子里找到了一篇不错的分片 ...

  9. Hadoop如何将TB级大文件的上传性能优化上百倍？

    这篇文章,我们来看看,Hadoop的HDFS分布式文件系统的文件上传的性能优化. 首先,我们还是通过一张图来回顾一下文件上传的大概的原理. 由上图所示,文件上传的原理,其实说出来也简单. 比如有个TB ...

随机推荐

  1. 使用Python3.x抓取58同城（南京站）的演出票的信息

    #!/usr/bin/env python #-*-coding: utf-8 -*- import re import urllib.request as request from bs4 impo ...

  2. stark组件开发之提取公共视图函数

     路由问题, 已经解决! 然后就是视图函数的问题了: 不想重复写的解决途径就是, python  类的继承了! 写一个基类, 基类定义 增删改查. 然后其他的,全部去继承他! from django. ...

  3. Struts2框架的数据封装一之属性封装（属性封装的第一种方式：对参数进行封装）

    request带着参数来,aciton对其进行处理.在学习action之前,使用的是servlet对request进行处理.request请求时会带有参数,所以我们要对这些参数进行封装. 1. 为什么 ...

  4. vue2.0一安装的插件详解

    babel-runtime 对ES语法转义 fastclick  移动端300ms兼容 babel-polyfill 兼容Ie浏览器 //babel-polyfill引用 推荐采用webpack入口文 ...

  5. Codeforces559C Gerald and Giant Chess

    一道计数类\(DP\) 原题链接 我们可以先计算从左上角到右下角总的路径,再减去经过黑色方格的路径即是答案. 总路径数可以用组合数直接计算:\(C_{H+W-2}^{H-1}\) 因为从左上角到右下角 ...

  6. Ubuntu12.04软件安装指南

    更新升级源 首先编辑软件源,在终端输入下面命令: sudo gedit /etc/apt/sources.list 较快速的升级源有163,台湾源,科大源,搜狐源等,大家将新的升级源全部覆盖原文件so ...

  7. 设计师们做UI设计和交互设计、界面设计等一般会去什么网站呢？

    明明可靠颜值吃饭,却偏偏要靠才华立身,UI设计师就是这样一群神奇的物种.面对“大的同时小一点”.“五彩斑斓黑”.“下班之前给我”……这些甲方大大刁钻的需求,设计师每天都在咬牙微笑讨生活.你可以批评我的 ...

  8. insert执行错误，怎么样获取具体的错误原因

    1.开启debug 2.去runtime里面去找最后执行的SQL

  9. JSON数据映射之元素可见控制

    1.效果: 2.demo 源码 <!DOCTYPE html> <html> <head> <meta charset="UTF-8"&g ...

  10. 将hibernate框架融入到spring框架中

    第一步:首先创建表: create table  user( id int(2) primary key,name varchar(20),password varchar(20)); 第二步:建立d ...