因为需要研究下断点上传的问题。找了很久终于找到一个比较好的项目。

效果：

上传中，显示进度，时间，百分比。

点击【Pause】暂停，点击【Resume】继续。

2，代码分析

项目进行了封装使用最简单的方法实现了http的断点上传。

因为html5 里面有读取文件分割文件的类库，所以才可以支持断点上传，所以这个只能在html5 支持的浏览器上面展示。

同时，在js 和 java 同时使用 cr32 进行文件块的校验，保证数据上传正确。

代码在使用了最新的servlet 3.0 的api，使用了异步执行，监听等方法。

上传类UploadServlet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
@Component("javaLargeFileUploaderServlet") 
@WebServlet(name = "javaLargeFileUploaderServlet", urlPatterns = { "/javaLargeFileUploaderServlet" }) 
public class UploadServlet extends HttpRequestHandlerServlet 
        implements HttpRequestHandler { 
   
    private static final Logger log = LoggerFactory.getLogger(UploadServlet.class); 
   
    @Autowired 
    UploadProcessor uploadProcessor; 
   
    @Autowired 
    FileUploaderHelper fileUploaderHelper; 
   
    @Autowired 
    ExceptionCodeMappingHelper exceptionCodeMappingHelper; 
   
    @Autowired 
    Authorizer authorizer; 
   
    @Autowired 
    StaticStateIdentifierManager staticStateIdentifierManager; 
   
   
   
    @Override 
    public void handleRequest(HttpServletRequest request, HttpServletResponse response) 
            throws IOException { 
        log.trace("Handling request"); 
   
        Serializable jsonObject = null; 
        try { 
            // extract the action from the request 
            UploadServletAction actionByParameterName = 
                    UploadServletAction.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.action)); 
   
            // check authorization 
            checkAuthorization(request, actionByParameterName); 
   
            // then process the asked action 
            jsonObject = processAction(actionByParameterName, request); 
   
   
            // if something has to be written to the response 
            if (jsonObject != null) { 
                fileUploaderHelper.writeToResponse(jsonObject, response); 
            } 
   
        } 
        // If exception, write it 
        catch (Exception e) { 
            exceptionCodeMappingHelper.processException(e, response); 
        } 
   
    } 
   
   
    private void checkAuthorization(HttpServletRequest request, UploadServletAction actionByParameterName) 
            throws MissingParameterException, AuthorizationException { 
   
        // check authorization 
        // if its not get progress (because we do not really care about authorization for get 
        // progress and it uses an array of file ids) 
        if (!actionByParameterName.equals(UploadServletAction.getProgress)) { 
   
            // extract uuid 
            final String fileIdFieldValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId, false); 
   
            // if this is init, the identifier is the one in parameter 
            UUID clientOrJobId; 
            String parameter = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false); 
            if (actionByParameterName.equals(UploadServletAction.getConfig) && parameter != null) { 
                clientOrJobId = UUID.fromString(parameter); 
            } 
            // if not, get it from manager 
            else { 
                clientOrJobId = staticStateIdentifierManager.getIdentifier(); 
            } 
   
               
            // call authorizer 
            authorizer.getAuthorization( 
                    request, 
                    actionByParameterName, 
                    clientOrJobId, 
                    fileIdFieldValue != null ? getFileIdsFromString(fileIdFieldValue).toArray(new UUID[] {}) : null); 
   
        } 
    } 
   
   
    private Serializable processAction(UploadServletAction actionByParameterName, HttpServletRequest request) 
            throws Exception { 
        log.debug("Processing action " + actionByParameterName.name()); 
   
        Serializable returnObject = null; 
        switch (actionByParameterName) { 
            case getConfig: 
                String parameterValue = fileUploaderHelper.getParameterValue(request, UploadServletParameter.clientId, false); 
                returnObject = 
                        uploadProcessor.getConfig( 
                                parameterValue != null ? UUID.fromString(parameterValue) : null); 
                break; 
            case verifyCrcOfUncheckedPart: 
                returnObject = verifyCrcOfUncheckedPart(request); 
                break; 
            case prepareUpload: 
                returnObject = prepareUpload(request); 
                break; 
            case clearFile: 
                uploadProcessor.clearFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId))); 
                break; 
            case clearAll: 
                uploadProcessor.clearAll(); 
                break; 
            case pauseFile: 
                List<UUID> uuids = getFileIdsFromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)); 
                uploadProcessor.pauseFile(uuids); 
                break; 
            case resumeFile: 
                returnObject = 
                        uploadProcessor.resumeFile(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId))); 
                break; 
            case setRate: 
                uploadProcessor.setUploadRate(UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)), 
                        Long.valueOf(fileUploaderHelper.getParameterValue(request, UploadServletParameter.rate))); 
                break; 
            case getProgress: 
                returnObject = getProgress(request); 
                break; 
        } 
        return returnObject; 
    } 
   
   
    List<UUID> getFileIdsFromString(String fileIds) { 
        String[] splittedFileIds = fileIds.split(","); 
        List<UUID> uuids = Lists.newArrayList(); 
        for (int i = 0; i < splittedFileIds.length; i++) { 
            uuids.add(UUID.fromString(splittedFileIds[i])); 
        }  
        return uuids; 
    } 
   
   
    private Serializable getProgress(HttpServletRequest request) 
            throws MissingParameterException { 
        Serializable returnObject; 
        String[] ids = 
                new Gson() 
                        .fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId), String[].class); 
        Collection<UUID> uuids = Collections2.transform(Arrays.asList(ids), new Function<String, UUID>() { 
   
            @Override 
            public UUID apply(String input) { 
                return UUID.fromString(input); 
            } 
   
        }); 
        returnObject = Maps.newHashMap(); 
        for (UUID fileId : uuids) { 
            try { 
                ProgressJson progress = uploadProcessor.getProgress(fileId); 
                ((HashMap<String, ProgressJson>) returnObject).put(fileId.toString(), progress); 
            } 
            catch (FileNotFoundException e) { 
                log.debug("No progress will be retrieved for " + fileId + " because " + e.getMessage()); 
            } 
        } 
        return returnObject; 
    } 
   
   
    private Serializable prepareUpload(HttpServletRequest request) 
            throws MissingParameterException, IOException { 
   
        // extract file information 
        PrepareUploadJson[] fromJson = 
                new Gson() 
                        .fromJson(fileUploaderHelper.getParameterValue(request, UploadServletParameter.newFiles), PrepareUploadJson[].class); 
   
        // prepare them 
        final HashMap<String, UUID> prepareUpload = uploadProcessor.prepareUpload(fromJson); 
   
        // return them 
        return Maps.newHashMap(Maps.transformValues(prepareUpload, new Function<UUID, String>() { 
   
            public String apply(UUID input) { 
                return input.toString(); 
            }; 
        })); 
    } 
   
   
    private Boolean verifyCrcOfUncheckedPart(HttpServletRequest request) 
            throws IOException, MissingParameterException, FileCorruptedException, FileStillProcessingException { 
        UUID fileId = UUID.fromString(fileUploaderHelper.getParameterValue(request, UploadServletParameter.fileId)); 
        try { 
            uploadProcessor.verifyCrcOfUncheckedPart(fileId, 
                    fileUploaderHelper.getParameterValue(request, UploadServletParameter.crc)); 
        } 
        catch (InvalidCrcException e) { 
            // no need to log this exception, a fallback behaviour is defined in the 
            // throwing method. 
            // but we need to return something! 
            return Boolean.FALSE; 
        } 
        return Boolean.TRUE; 
    } 
}

异步上传UploadServletAsync

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
@Component("javaLargeFileUploaderAsyncServlet") 
@WebServlet(name = "javaLargeFileUploaderAsyncServlet", urlPatterns = { "/javaLargeFileUploaderAsyncServlet" }, asyncSupported = true) 
public class UploadServletAsync extends HttpRequestHandlerServlet 
        implements HttpRequestHandler { 
   
    private static final Logger log = LoggerFactory.getLogger(UploadServletAsync.class); 
   
    @Autowired 
    ExceptionCodeMappingHelper exceptionCodeMappingHelper; 
   
    @Autowired 
    UploadServletAsyncProcessor uploadServletAsyncProcessor; 
       
    @Autowired 
    StaticStateIdentifierManager staticStateIdentifierManager; 
   
    @Autowired 
    StaticStateManager<StaticStatePersistedOnFileSystemEntity> staticStateManager; 
   
    @Autowired 
    FileUploaderHelper fileUploaderHelper; 
   
    @Autowired 
    Authorizer authorizer; 
   
    /**
     * Maximum time that a streaming request can take.<br>
     */ 
    private long taskTimeOut = DateUtils.MILLIS_PER_HOUR; 
   
   
    @Override 
    public void handleRequest(final HttpServletRequest request, final HttpServletResponse response) 
            throws ServletException, IOException { 
   
        // process the request 
        try { 
   
            //check if uploads are allowed 
            if (!uploadServletAsyncProcessor.isEnabled()) { 
                throw new UploadIsCurrentlyDisabled(); 
            } 
               
            // extract stuff from request 
            final FileUploadConfiguration process = fileUploaderHelper.extractFileUploadConfiguration(request); 
   
            log.debug("received upload request with config: "+process); 
   
            // verify authorization 
            final UUID clientId = staticStateIdentifierManager.getIdentifier(); 
            authorizer.getAuthorization(request, UploadServletAction.upload, clientId, process.getFileId()); 
   
            //check if that file is not paused 
            if (uploadServletAsyncProcessor.isFilePaused(process.getFileId())) { 
                log.debug("file "+process.getFileId()+" is paused, ignoring async request."); 
                return; 
            } 
               
            // get the model 
            StaticFileState fileState = staticStateManager.getEntityIfPresent().getFileStates().get(process.getFileId()); 
            if (fileState == null) { 
                throw new FileNotFoundException("File with id " + process.getFileId() + " not found"); 
            } 
   
            // process the request asynchronously 
            final AsyncContext asyncContext = request.startAsync(); 
            asyncContext.setTimeout(taskTimeOut); 
   
   
            // add a listener to clear bucket and close inputstream when process is complete or 
            // with 
            // error 
            asyncContext.addListener(new UploadServletAsyncListenerAdapter(process.getFileId()) { 
   
                @Override 
                void clean() { 
                    log.debug("request " + request + " completed."); 
                    // we do not need to clear the inputstream here. 
                    // and tell processor to clean its shit! 
                    uploadServletAsyncProcessor.clean(clientId, process.getFileId()); 
                } 
            }); 
   
            // then process 
            uploadServletAsyncProcessor.process(fileState, process.getFileId(), process.getCrc(), process.getInputStream(), 
                    new WriteChunkCompletionListener() { 
   
                        @Override 
                        public void success() { 
                            asyncContext.complete(); 
                        } 
   
   
                        @Override 
                        public void error(Exception exception) { 
                            // handles a stream ended unexpectedly , it just means the user has 
                            // stopped the 
                            // stream 
                            if (exception.getMessage() != null) { 
                                if (exception.getMessage().equals("Stream ended unexpectedly")) { 
                                    log.warn("User has stopped streaming for file " + process.getFileId()); 
                                } 
                                else if (exception.getMessage().equals("User cancellation")) { 
                                    log.warn("User has cancelled streaming for file id " + process.getFileId()); 
                                    // do nothing 
                                } 
                                else { 
                                    exceptionCodeMappingHelper.processException(exception, response); 
                                } 
                            } 
                            else { 
                                exceptionCodeMappingHelper.processException(exception, response); 
                            } 
   
                            asyncContext.complete(); 
                        } 
   
                    }); 
        } 
        catch (Exception e) { 
            exceptionCodeMappingHelper.processException(e, response); 
        } 
   
    } 
   
}

3，请求流程图：

主要思路就是将文件切分，然后分块上传。

DEMO下载地址：https://dwz.cn/fgXtRtnu

java http大文件断点续传上传的更多相关文章

  1. java springboot 大文件分片上传处理

    参考自:https://blog.csdn.net/u014150463/article/details/74044467 这里只写后端的代码,基本的思想就是,前端将文件分片,然后每次访问上传接口的时 ...

  2. Ajax+Java实现大文件切割上传

    技术体系:html5(formdata) + java + servlet3.0+maven + tomcat7 <!DOCTYPE html> <html> <head ...

  3. 关于：基于http协议大文件断点续传上传至web服务器

    关键部分 前端用file.slice()分块 前端用FileReader获取每一分块的md5值 后端用MultipartFile接受分块文件 后端用FileOutputStream拼装分块文件 话不多 ...

  4. iOS大文件分片上传和断点续传

    总结一下大文件分片上传和断点续传的问题.因为文件过大(比如1G以上),必须要考虑上传过程网络中断的情况.http的网络请求中本身就已经具备了分片上传功能,当传输的文件比较大时,http协议自动会将文件 ...

  5. js大文件分块上传断点续传demo

    文件夹上传:从前端到后端 文件上传是 Web 开发肯定会碰到的问题,而文件夹上传则更加难缠.网上关于文件夹上传的资料多集中在前端,缺少对于后端的关注,然后讲某个后端框架文件上传的文章又不会涉及文件夹. ...

  6. 使用webuploader组件实现大文件分片上传，断点续传

    本人在2010年时使用swfupload为核心进行文件的批量上传的解决方案.见文章:WEB版一次选择多个文件进行批量上传(swfupload)的解决方案. 本人在2013年时使用plupload为核心 ...

  7. vue大文件分片上传插件

    最近遇见一个需要上传百兆大文件的需求,调研了七牛和腾讯云的切片分段上传功能,因此在此整理前端大文件上传相关功能的实现. 在某些业务中,大文件上传是一个比较重要的交互场景,如上传入库比较大的Excel表 ...

  8. Webuploader 大文件分片上传

    百度Webuploader 大文件分片上传(.net接收)   前阵子要做个大文件上传的功能,找来找去发现Webuploader还不错,关于她的介绍我就不再赘述. 动手前,在园子里找到了一篇不错的分片 ...

  9. Hadoop如何将TB级大文件的上传性能优化上百倍？

    这篇文章,我们来看看,Hadoop的HDFS分布式文件系统的文件上传的性能优化. 首先,我们还是通过一张图来回顾一下文件上传的大概的原理. 由上图所示,文件上传的原理,其实说出来也简单. 比如有个TB ...

随机推荐

  1. 在做excel导出时如何将workbook直接写在输出流中

    参考网址 https://blog.csdn.net/u011109420/article/details/51330677 https://blog.csdn.net/u012116457/arti ...

  2. 各个JSON技术的比较（Jackson，Gson，Fastjson）的对比

    JSON技术的调研报告 一 .各个JSON技术的简介和优劣 1.json-lib json-lib最开始的也是应用最广泛的json解析工具,json-lib 不好的地方确实是依赖于很多第三方包, 包括 ...

  3. CMakeLists.txt的写法

    [1]ADD_LIBRARY: Add a library to the project using the specified source files.要求CMake根据指定的源文件生成库文件 . ...

  4. centos7下的glusterfs的安装与使用

    环境说明: 两台虚拟主机,IP分别为192.168.1.222和192.168.1.233,分别装上glusterfs. 一.下载glusterfs源码包 https://download.glust ...

  5. 如何将你拍摄的照片转换成全景图及六面体（PTGui）

    在完成全景照片的拍摄之后,接下来,我们需要的是进行全景图的拼接.全景图片分为两种类型1.立方体全景图(6面体)制作全景时通常使用该种格式 如下图 2.球形图(2:1的单张全景图片)2:1全景图宽高比例 ...

  6. C# 通过api函数GetPrivateProfileString读取ini文件，取不到值

    通过api函数GetPrivateProfileString读取ini文件,取不到值,测试了好长时间,都不行 确认程序,ini文件都没有错误的情况,最后发现是ini文件编码的原因. 将ini文件的编码 ...

  7. PAT 1056 组合数的和（15）（代码）

    1056 组合数的和(15 分) 给定 N 个非 0 的个位数字,用其中任意 2 个数字都可以组合成 1 个 2 位的数字.要求所有可能组合出来的 2 位数字的和.例如给定 2.5.8,则可以组合出: ...

  8. linux利用crontab设置定时任务运行jar包

    参考链接: 1.http://blog.csdn.net/javadhh/article/details/42779505 2.http://blog.csdn.net/cctv_liu/articl ...

  9. mysql 优化之一

    提升速度 show  variables like 'innodb_flush_log_at_trx_commit'; 会显示为1 set global innodb_flush_log_at_trx ...

  10. Using The jQuery Migrate Plugin

    jQuery( html [, ownerDocument ] )Returns: jQuery Description: Creates DOM elements on the fly from t ...