目錄
- 文件分片上傳、斷點續傳及秒傳
- 代碼實現
- web端
- 官方原生的案例修改
- 自定義uploader1
- 自定義uploader2
- SpringBoot實現后端
- 源碼下載
文件分片上傳、斷點續傳及秒傳
功能介紹
文件上傳
小文件（圖片、文檔、視頻）上傳可以直接使用很多ui框架封裝的上傳組件，或者自己寫一個input 上傳，利用FormData 對象提交文件數據，后端使用spring提供的MultipartFile進行文件的接收，然后寫入即可。但是對于比較大的文件，比如上傳2G左右的文件（http上傳），就需要將文件分片上傳（file.slice()），否則中間http長時間連接可能會斷掉。
分片上傳
分片上傳，就是將所要上傳的文件，按照一定的大小，將整個文件分隔成多個數據塊（我們稱之為Part）來進行分別上傳，上傳完之后再由服務端對所有上傳的文件進行匯總整合成原始的文件。
秒傳
通俗的說，你把要上傳的東西上傳，服務器會先做MD5校驗，如果服務器上有一樣的東西，它就直接給你個新地址，其實你下載的都是服務器上的同一個文件，想要不秒傳，其實只要讓MD5改變，就是對文件本身做一下修改（改名字不行），例如一個文本文件，你多加幾個字，MD5就變了，就不會秒傳了.
斷點續傳
斷點續傳是在下載或上傳時，將下載或上傳任務（一個文件或一個壓縮包）人為的劃分為幾個部分，每一個部分采用一個線程進行上傳或下載，如果碰到網絡故障，可以從已經上傳或下載的部分開始繼續上傳或者下載未完成的部分，而沒有必要從頭開始上傳或者下載。本文的斷點續傳主要是針對斷點上傳場景。
相關插件技術介紹
vue-simple-uploader
前端使用vue-simple-uploader,一個基于simple-uploader封裝的上傳插件，imple-uploader.js（也稱 Uploader) 是一個上傳庫，支持多并發上傳，文件夾、拖拽、可暫停繼續、秒傳、分塊上傳、出錯自動重傳、手工重傳、進度、剩余時間、上傳速度等特性。
simple-uploader文檔案例：https://github.com/simple-uploader/vue-uploader
vue-simple-uploader文檔案例：https://github.com/simple-uploader/Uploader/blob/develop/README_zh-CN.md
使用前必須要了解的概念和方法
相關概念
chunkNumber: 當前塊的次序，第一個塊是 1，注意不是從 0 開始的。
totalChunks: 文件被分成塊的總數。
chunkSize: 分塊大小，根據 totalSize 和這個值你就可以計算出總共的塊數。注意最后一塊的大小可能會比這個要大。
currentChunkSize: 當前塊的大小，實際大小。
totalSize: 文件總大小。
identifier: 這個就是MD5值，每個文件的唯一標示。
filename: 文件名
相關方法
.upload() 開始或者繼續上傳。
.pause() 暫停上傳。
.resume() 繼續上傳。
.cancel() 取消所有上傳文件，文件會被移除掉。
.progress() 返回一個0-1的浮點數，當前上傳進度。
.isUploading() 返回一個布爾值標示是否還有文件正在上傳中。
.addFile(file) 添加一個原生的文件對象到上傳列表中。
.removeFile(file) 從上傳列表中移除一個指定的 Uploader.File 實例對象。
MD5加密
md5加密是可加鹽的非對稱加密算法。
java使用MD5加密案例可以查看：https://qkongtao.cn/?p=580#h3-7
web對文件的MD5加密可以使用：spark-md5
spark-md5.js號稱是最適合前端最快的算法，能快速計算文件的md5。
快速安裝：
npm install --save spark-md5
在組件中使用spark-md5時先引入：
import SparkMD5 from 'spark-md5';
spark-md5提供了兩個計算md5的方法。一種是用SparkMD5.hashBinary() 直接將整個文件的二進制碼傳入，直接返回文件的md5。這種方法對于小文件會比較有優勢——簡單而且速度超快。
另一種方法是利用js中File對象的slice()方法（File.prototype.slice）將文件分片后逐個傳入spark.appendBinary()方法來計算、最后通過spark.end()方法輸出md5。很顯然，此方法就是我們前面講到的分片計算md5。這種方法對于大文件和超大文件會非常有利，不容易出錯，不占用大內存，并且能夠提供計算的進度信息。
大文件上傳流程
前端對文件進行MD5加密，并且將文件按一定的規則分片vue-simple-uploader先會發送get請求校驗分片數據在服務端是否完整，如果完整則進行秒傳，如果不完整或者無數據，則進行分片上傳。后臺校驗MD5值，根據上傳的序號和分片大小計算相應的開始位置并寫入該分片數據到文件中。
代碼實現
web端
源碼鏈接： https://gitee.com/KT1205529635/simple-uploader/tree/master/vue-uploader-master
本次參考了官方文檔已經給位大佬的案例，根據自己的想法，實現了大文件的分片上傳、斷點續傳及秒傳
其中前端寫了三個案例
- 官方原生的案例修改
- 自己根據插件提供的api和鉤子，自己diy自定義上傳（配合springboot后臺，文件夾上傳未作處理）
- 自己diy自定義上傳的基礎上，在前端處理文件夾上傳（文件夾只接收文件夾里的所有文件，未處理文件夾相對目錄，可自己拓展）
官方原生的案例修改
效果如下
代碼如下
VueUploader.vue
https://gitee.com/KT1205529635/simple-uploader/blob/master/vue-uploader-master/src/views/VueUploader.vue#
<template><div class="container"><div class="logo"><img src="@/assets/logo.png" /></div><uploaderref="uploader":options="options":autoStart="false":file-status-text="fileStatusText"@file-added="onFileAdded"@file-success="onFileSuccess"@file-error="onFileError"@file-progress="onFileProgress"class="uploader-example"><uploader-unsupport></uploader-unsupport><uploader-drop><p>拖動文件到這里上傳
</p><uploader-btn>選擇文件
</uploader-btn><uploader-btn :directory="true">選擇文件夾
</uploader-btn></uploader-drop><uploader-list><div class="file-panel" :class="{ collapse: collapse }"><div class="file-title"><p class="file-list-title">文件列表
</p><div class="operate"><el-buttontype="text"@click="operate":title="collapse ? '折疊' : '展開'"><iclass="icon":class="collapse ? 'el-icon-caret-bottom' : 'el-icon-caret-top'"></i></el-button><el-button type="text" @click="close" title="關閉"><i class="icon el-icon-close"></i></el-button></div></div><ulclass="file-list":class="collapse ? 'uploader-list-ul-show' : 'uploader-list-ul-hidden'"><li v-for="file in uploadFileList" :key="file.id"><uploader-file:class="'file_' + file.id"ref="files":file="file":list="true"></uploader-file></li><div class="no-file" v-if="!uploadFileList.length"><i class="icon icon-empty-file"></i> 暫無待上傳文件
</div></ul></div></uploader-list><span>下載
</span></uploader></div>
</template><script>
import SparkMD5 from "spark-md5";
const FILE_UPLOAD_ID_KEY = "file_upload_id";
const CHUNK_SIZE = 20 * 1024 * 1024;
export default {data() {return {options: {target: "http://127.0.0.1:8025/api/upload",testChunks: true,uploadMethod: "post",chunkSize: CHUNK_SIZE,simultaneousUploads: 3,checkChunkUploadedByResponse: (chunk, message) => {let messageObj = JSON.parse(message);let dataObj = messageObj.data;if (dataObj.uploaded !== undefined) {return dataObj.uploaded;}return (dataObj.uploadedChunks || []).indexOf(chunk.offset + 1) >= 0;},parseTimeRemaining: function (timeRemaining, parsedTimeRemaining) {return parsedTimeRemaining.replace(/\syears?/, "年").replace(/\days?/, "天").replace(/\shours?/, "小時").replace(/\sminutes?/, "分鐘").replace(/\sseconds?/, "秒");},},fileStatusTextObj: {success: "上傳成功",error: "上傳錯誤",uploading: "正在上傳",paused: "停止上傳",waiting: "等待中",},uploadIdInfo: null,uploadFileList: [],fileChunkList: [],collapse: true,};},created() {},methods: {onFileAdded(file, event) {console.log("file :>> ", file);console.log("文件類型：" + file.fileType);console.log("文件大小：" + file.size + "B");console.log("校驗MD5");this.getFileMD5(file, (md5) => {if (md5 != "") {file.uniqueIdentifier = md5;file.resume();}});},onFileSuccess(rootFile, file, response, chunk) {this.uploadFileList = this.$refs.uploader.fileList;console.log(this.uploadFileList);console.log("上傳成功");},onFileError(rootFile, file, message, chunk) {console.log("上傳出錯：" + message);},onFileProgress(rootFile, file, chunk) {console.log(`當前進度：${Math.ceil(file._prevProgress * 100)}%`);},getFileMD5(file, callback) {let spark = new SparkMD5.ArrayBuffer();let fileReader = new FileReader();let blobSlice =File.prototype.slice ||File.prototype.mozSlice ||File.prototype.webkitSlice;let currentChunk = 0;let chunks = Math.ceil(file.size / CHUNK_SIZE);let startTime = new Date().getTime();file.pause();loadNext();fileReader.onload = function (e) {spark.append(e.target.result);if (currentChunk < chunks) {currentChunk++;loadNext();} else {let md5 = spark.end();console.log(`MD5計算完畢：${md5}，耗時：${new Date().getTime() - startTime} ms.`);callback(md5);}};fileReader.onerror = function () {this.$message.error("文件讀取錯誤");file.cancel();};function loadNext() {const start = currentChunk * CHUNK_SIZE;const end =start + CHUNK_SIZE >= file.size ? file.size : start + CHUNK_SIZE;fileReader.readAsArrayBuffer(blobSlice.call(file.file, start, end));}},fileStatusText(status, response) {if (status === "md5") {return "校驗MD5";} else {return this.fileStatusTextObj[status];}},operate() {if (this.collapse === false) {this.collapse = true;} else {this.collapse = false;}},close() {this.uploaderPanelShow = false;},},
};
</script><style lang="less" scoped>
.logo {font-family: "Avenir", Helvetica, Arial, sans-serif;-webkit-font-smoothing: antialiased;-moz-osx-font-smoothing: grayscale;text-align: center;color: #2c3e50;margin-top: 60px;
}
.uploader-example {width: 880px;padding: 15px;margin: 40px auto 0;font-size: 12px;box-shadow: 0 0 10px rgba(0, 0, 0, 0.4);
}
.uploader-example .uploader-btn {margin-right: 4px;
}
.uploader-example .uploader-list {max-height: 440px;overflow: auto;overflow-x: hidden;overflow-y: auto;
}#global-uploader {position: fixed;z-index: 20;right: 15px;bottom: 15px;width: 550px;
}.file-panel {background-color: #fff;border: 1px solid #e2e2e2;border-radius: 7px 7px 0 0;box-shadow: 0 0 10px rgba(0, 0, 0, 0.2);
}.file-title {display: flex;height: 60px;line-height: 30px;padding: 0 15px;border-bottom: 1px solid #ddd;
}.file-title {background-color: #e7ecf2;
}.uploader-file-meta {display: none !important;
}.operate {flex: 1;text-align: right;
}.file-list {position: relative;height: 240px;overflow-x: hidden;overflow-y: auto;background-color: #fff;padding: 0px;margin: 0 auto;transition: all 0.5s;
}.uploader-file-size {width: 15% !important;
}.uploader-file-status {width: 32.5% !important;text-align: center !important;
}li {background-color: #fff;list-style-type: none;
}.no-file {position: absolute;top: 50%;left: 50%;transform: translate(-50%, -50%);font-size: 16px;
}
.global-uploader-btn {display: none !important;clip: rect(0, 0, 0, 0);
}.file-list-title {font-size: 16px;
}.uploader-file-name {width: 36% !important;
}.uploader-file-actions {float: right !important;
}.uploader-list-ul-hidden {height: 0px;
}
</style>
自定義uploader1
根據插槽和鉤子函數，實現自定義插件樣式，也實現簡單的下載。
效果如下
代碼如下
DiyUpload1.vue
https://gitee.com/KT1205529635/simple-uploader/blob/master/vue-uploader-master/src/views/DiyUpload1.vue#
<template><div class="container"><div class="logo"><img src="@/assets/logo.png" /></div><uploaderref="uploader":options="options":autoStart="false":file-status-text="fileStatusText"@file-added="onFileAdded"@file-success="onFileSuccess"@file-error="onFileError"@file-progress="onFileProgress"class="uploader-example"><uploader-unsupport></uploader-unsupport><uploader-drop><p>拖動文件到這里上傳
</p><uploader-btn>選擇文件
</uploader-btn></uploader-drop><uploader-list><div class="file-panel" :class="{ collapse: collapse }"><div class="file-title"><p class="file-list-title">文件列表
</p><div class="operate"><el-buttontype="text"@click="operate":title="collapse ? '折疊' : '展開'"><iclass="icon":class="collapse ? 'el-icon-caret-bottom' : 'el-icon-caret-top'"></i></el-button><el-button type="text" @click="close" title="關閉"><i class="icon el-icon-close"></i></el-button></div></div><ulclass="file-list":class="collapse ? 'uploader-list-ul-show' : 'uploader-list-ul-hidden'"><li v-for="file in uploadFileList" :key="file.id"><uploader-file :file="file" :list="true" ref="uploaderFile"><template slot-scope="props"><div class="filebox"><p class="fileNameBox"><span class="fileIcon"></span>{{ file.name }}
</p><p class="fileProgressBox"><el-progressclass="progressLength":stroke-width="18":percentage="parseInt(props.progress.toFixed(2) * 100 - 1 < 0? 0: props.progress.toFixed(2) * 100)"></el-progress><spanclass="statusBtn progressBtn"v-if="!file.completed"@click="pause(file)"><iclass="el-icon-video-pause"v-if="!file.paused"title="暫停"></i><i class="el-icon-video-play" v-else title="繼續"></i></span><spanv-elseclass="downloadBtn progressBtn"@click="download(file)"><i class="el-icon-download" title="下載"></i></span><span class="cancelBtn progressBtn" @click="remove(file)"><i class="el-icon-error" title="刪除"></i></span></p><p class="fileInfoBox" v-if="!file.completed"><span class="fileInfoItem">速度：{{ props.formatedAverageSpeed }}
</span><span class="fileInfoItem">已上傳：{{(parseFloat(props.formatedSize) * props.progress).toFixed(1)}}/{{ props.formatedSize }}
</span><span class="fileInfoItem">剩余時間：{{ props.formatedTimeRemaining }}
</span></p><p class="fileInfoBoxSuccess" v-else>上傳成功
</p></div></template></uploader-file></li><div class="no-file" v-if="!uploadFileList.length"><i class="icon icon-empty-file"></i> 暫無待上傳文件
</div></ul></div></uploader-list></uploader></div>
</template><script>
import SparkMD5 from "spark-md5";
const FILE_UPLOAD_ID_KEY = "file_upload_id";
const CHUNK_SIZE = 20 * 1024 * 1024;
export default {data() {return {options: {target: "http://127.0.0.1:8025/api/upload",testChunks: true,uploadMethod: "post",chunkSize: CHUNK_SIZE,simultaneousUploads: 3,checkChunkUploadedByResponse: (chunk, message) => {let messageObj = JSON.parse(message);let dataObj = messageObj.data;if (dataObj.uploaded !== undefined) {return dataObj.uploaded;}return (dataObj.uploadedChunks || []).indexOf(chunk.offset + 1) >= 0;},parseTimeRemaining: function (timeRemaining, parsedTimeRemaining) {return parsedTimeRemaining.replace(/\syears?/, "年").replace(/\days?/, "天").replace(/\shours?/, "小時").replace(/\sminutes?/, "分鐘").replace(/\sseconds?/, "秒");},},fileStatusTextObj: {success: "上傳成功",error: "上傳錯誤",uploading: "正在上傳",paused: "停止上傳",waiting: "等待中",},uploadIdInfo: null,uploadFileList: [],fileChunkList: [],collapse: true,};},created() {},methods: {onFileAdded(file, event) {this.uploadFileList.push(file);console.log("file :>> ", file);console.log("文件類型：" + file.fileType);console.log("文件大小：" + file.size + "B");console.log("校驗MD5");this.getFileMD5(file, (md5) => {if (md5 != "") {file.uniqueIdentifier = md5;file.resume();}});},onFileSuccess(rootFile, file, response, chunk) {console.log("上傳成功");},onFileError(rootFile, file, message, chunk) {console.log("上傳出錯：" + message);},onFileProgress(rootFile, file, chunk) {console.log(`當前進度：${Math.ceil(file._prevProgress * 100)}%`);},getFileMD5(file, callback) {let spark = new SparkMD5.ArrayBuffer();let fileReader = new FileReader();let blobSlice =File.prototype.slice ||File.prototype.mozSlice ||File.prototype.webkitSlice;let currentChunk = 0;let chunks = Math.ceil(file.size / CHUNK_SIZE);let startTime = new Date().getTime();file.pause();loadNext();fileReader.onload = function (e) {spark.append(e.target.result);if (currentChunk < chunks) {currentChunk++;loadNext();} else {let md5 = spark.end();console.log(`MD5計算完畢：${md5}，耗時：${new Date().getTime() - startTime} ms.`);callback(md5);}};fileReader.onerror = function () {this.$message.error("文件讀取錯誤");file.cancel();};function loadNext() {const start = currentChunk * CHUNK_SIZE;const end =start + CHUNK_SIZE >= file.size ? file.size : start + CHUNK_SIZE;fileReader.readAsArrayBuffer(blobSlice.call(file.file, start, end));}},fileStatusText(status, response) {if (status === "md5") {return "校驗MD5";} else {return this.fileStatusTextObj[status];}},operate() {if (this.collapse === false) {this.collapse = true;} else {this.collapse = false;}},close() {this.uploaderPanelShow = false;},pause(file, id) {console.log("file :>> ", file);if (file.paused) {file.resume();} else {file.pause();}},remove(file) {this.uploadFileList.findIndex((item, index) => {if (item.id === file.id) {this.$nextTick(() => {this.uploadFileList.splice(index, 1);});return;}});},download(file, id) {console.log("file:>> ", file);window.location.href = `http://127.0.0.1:8025/api/download/${file.uniqueIdentifier}/${file.name}`;},},
};
</script><style lang="less" scoped>
.logo {font-family: "Avenir", Helvetica, Arial, sans-serif;-webkit-font-smoothing: antialiased;-moz-osx-font-smoothing: grayscale;text-align: center;color: #2c3e50;margin-top: 60px;
}
.uploader-example {width: 880px;padding: 15px;margin: 40px auto 0;font-size: 12px;box-shadow: 0 0 10px rgba(0, 0, 0, 0.4);
}
.uploader-example .uploader-btn {margin-right: 4px;
}
.uploader-example .uploader-list {max-height: 440px;overflow: auto;overflow-x: hidden;overflow-y: auto;
}#global-uploader {position: fixed;z-index: 20;right: 15px;bottom: 15px;width: 550px;
}.file-panel {background-color: #fff;border: 1px solid #e2e2e2;border-radius: 7px 7px 0 0;box-shadow: 0 0 10px rgba(0, 0, 0, 0.2);
}.file-title {display: flex;height: 60px;line-height: 30px;padding: 0 15px;border-bottom: 1px solid #ddd;
}.file-title {background-color: #e7ecf2;
}
.uploader-file {height: 90px;
}.uploader-file-meta {display: none !important;
}.operate {flex: 1;text-align: right;
}.file-list {position: relative;height: 300px;overflow-x: hidden;overflow-y: auto;background-color: #fff;padding: 0px;margin: 0 auto;transition: all 0.5s;
}.uploader-file-size {width: 15% !important;
}.uploader-file-status {width: 32.5% !important;text-align: center !important;
}li {background-color: #fff;list-style-type: none;
}.no-file {position: absolute;top: 50%;left: 50%;transform: translate(-50%, -50%);font-size: 16px;
}.file-list-title {font-size: 16px;
}.uploader-file-name {width: 36% !important;
}.uploader-file-actions {float: right !important;
}.uploader-list-ul-hidden {height: 0px;
}.filebox {width: 100%;height: 60px;
}
.fileNameBox {width: 85%;margin: 0;padding: 0;font-size: 16px;margin-top: 5px;height: 30px;line-height: 30px;text-align: center;
}
.fileProgressBox {margin: 0;padding: 0;height: 20px;line-height: 20px;margin-top: 5px;margin-left: 10px;width: 100%;
}
/deep/ .el-progress-bar {width: 95%;
}
.progressLength {display: inline-block;line-height: 20px;width: 80%;
}
.progressBtn {margin-top: -5px;position: absolute;display: inline-block;font-size: 36px;margin-left: 10px;cursor: pointer;
}
.statusBtn {right: 90px;color: #ffba00;
}
.statusBtn:hover {color: #ffc833;
}
.cancelBtn {right: 30px;color: #ff4949;
}
.cancelBtn {margin-left: 10px;
}
.cancelBtn:hover {color: #ff6d6d;
}
.downloadBtn {right: 90px;color: #67c23a;
}
.downloadBtn:hover {color: #85ce61;
}
.fileInfoBox {margin: 0;padding: 0;font-size: 16px;width: 100%;height: 30px;line-height: 30px;margin-left: 10px;margin-bottom: 5px;.fileInfoItem {display: inline-block;width: 33%;}
}
.fileInfoBoxSuccess {margin: 0;padding: 0;font-size: 16px;width: 85%;height: 30px;line-height: 30px;margin-bottom: 5px;text-align: center;
}
</style>
自定義uploader2
在自定義uploader1上實現可上傳文件夾
效果如下
代碼如下
https://gitee.com/KT1205529635/simple-uploader/blob/master/vue-uploader-master/src/views/DiyUpload2.vue#
SpringBoot實現后端
源碼鏈接： https://gitee.com/KT1205529635/simple-uploader/tree/master/springboot-upload-master
后端實現簡單粗暴：springboot + jpa + hutool + mysql
主要實現：
get請求接口校驗上傳文件MD5值和文件是否完整post請求接收上傳文件，并且計算分片，寫入合成文件文件完整上傳完成時，往文件存儲表tool_local_storage中加一條該文件的信息get請求接口實現簡單的文件下載
目錄結構如下：
關鍵代碼如下：
sql如下
DROP TABLE IF EXISTS `file_chunk`;
CREATE TABLE `file_chunk` (`id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,`file_name` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '文件名',`chunk_number` int(11) NULL DEFAULT NULL COMMENT '當前分片，從1開始',`chunk_size` float NULL DEFAULT NULL COMMENT '分片大小',`current_chunk_size` float NULL DEFAULT NULL COMMENT '當前分片大小',`total_size` double(20, 0) NULL DEFAULT NULL COMMENT '文件總大小',`total_chunk` int(11) NULL DEFAULT NULL COMMENT '總分片數',`identifier` varchar(45) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '文件標識',`relative_path` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT 'md5校驗碼',`createtime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP,`updatetime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP(0),PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1529 CHARACTER SET = utf8
COLLATE = utf8_general_ci ROW_FORMAT
= Dynamic
;
DROP TABLE IF EXISTS `tool_local_storage`;
CREATE TABLE `tool_local_storage` (`id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'ID',`real_name` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '文件真實的名稱',`name` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '文件名',`suffix` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '后綴',`path` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '路徑',`type` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '類型',`size` varchar(100) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '大小',`identifier` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT 'md5校驗碼\r\n',`create_by` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '創建者',`update_by` varchar(255) CHARACTER SET utf8
COLLATE utf8_general_ci
NULL DEFAULT NULL COMMENT '更新者',`createtime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP,`updatetime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP(0),PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 3360 CHARACTER SET = utf8
COLLATE = utf8_general_ci
COMMENT = '文件存儲' ROW_FORMAT
= Compact
;
controller實現
package cn.kt.springbootuploadmaster.controller;import cn.kt.springbootuploadmaster.domin.FileChunkParam;
import cn.kt.springbootuploadmaster.domin.ResultVO;
import cn.kt.springbootuploadmaster.service.FileChunkService;
import cn.kt.springbootuploadmaster.service.FileService;
import cn.kt.springbootuploadmaster.service.LocalStorageService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@RestController
@Slf4j
@RequestMapping("/api")
public class FileUploadController {@Autowiredprivate FileService fileService
;@Autowiredprivate FileChunkService fileChunkService
;@Autowiredprivate LocalStorageService localStorageService
;@GetMapping("/upload")public ResultVO<Map<String, Object>> checkUpload(FileChunkParam param
) {log
.info("文件MD5:" + param
.getIdentifier());List<FileChunkParam> list
= fileChunkService
.findByMd5(param
.getIdentifier());Map<String, Object> data
= new HashMap<>(1);if (list
.size() == 0) {data
.put("uploaded", false);return new ResultVO<>(200, "上傳成功", data
);}if (list
.get(0).getTotalChunks() == 1) {data
.put("uploaded", true);data
.put("url", "");return new ResultVO<Map<String, Object>>(200, "上傳成功", data
);}int[] uploadedFiles
= new int[list
.size()];int index
= 0;for (FileChunkParam fileChunkItem
: list
) {uploadedFiles
[index
] = fileChunkItem
.getChunkNumber();index
++;}data
.put("uploadedChunks", uploadedFiles
);return new ResultVO<Map<String, Object>>(200, "上傳成功", data
);}@PostMapping("/upload")public ResultVO chunkUpload(FileChunkParam param
) {log
.info("上傳文件：{}", param
);boolean flag
= fileService
.uploadFile(param
);if (!flag
) {return new ResultVO(211, "上傳失敗");}return new ResultVO(200, "上傳成功");}@GetMapping(value
= "/download/{md5}/{name}")public void downloadbyname(HttpServletRequest request
, HttpServletResponse response
, @PathVariable String name
, @PathVariable String md5
) throws IOException {localStorageService
.downloadByName(name
, md5
, request
, response
);}}
FileService實現
FileService.java
package cn.kt.springbootuploadmaster.service;import cn.kt.springbootuploadmaster.domin.FileChunkParam;
public interface FileService {boolean uploadFile(FileChunkParam param
);
}
FileServiceImpl.java
package cn.kt.springbootuploadmaster.service.impl;import cn.kt.springbootuploadmaster.domin.FileChunkParam;
import cn.kt.springbootuploadmaster.enums.MessageEnum;
import cn.kt.springbootuploadmaster.exception.BusinessException;
import cn.kt.springbootuploadmaster.repository.LocalStorageRepository;
import cn.kt.springbootuploadmaster.service.FileChunkService;
import cn.kt.springbootuploadmaster.service.FileService;
import cn.kt.springbootuploadmaster.service.LocalStorageService;
import cn.kt.springbootuploadmaster.utils.FileUtil;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import sun.misc.Cleaner;import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.lang.reflect.Method;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.security.AccessController;
import java.security.PrivilegedAction;
@Service("fileService")
@Slf4j
public class FileServiceImpl implements FileService {public static final long DEFAULT_CHUNK_SIZE
= 20 * 1024 * 1024;@Value("${file.BASE_FILE_SAVE_PATH}")private String BASE_FILE_SAVE_PATH
;@Autowiredprivate FileChunkService fileChunkService
;@Autowiredprivate LocalStorageService localStorageService
;@Overridepublic boolean uploadFile(FileChunkParam param
) {if (null == param
.getFile()) {throw new BusinessException(MessageEnum.UPLOAD_FILE_NOT_NULL
);}File savePath
= new File(BASE_FILE_SAVE_PATH
);if (!savePath
.exists()) {boolean flag
= savePath
.mkdirs();if (!flag
) {log
.error("保存目錄創建失敗");return false;}}String fullFileName
= savePath
+ File.separator
+ param
.getFilename();if (param
.getTotalChunks() == 1) {return uploadSingleFile(fullFileName
, param
);}boolean flag
= uploadFileByRandomAccessFile(fullFileName
, param
);if (!flag
) {return false;}fileChunkService
.saveFileChunk(param
);return true;}private boolean uploadFileByRandomAccessFile(String resultFileName
, FileChunkParam param
) {try (RandomAccessFile randomAccessFile
= new RandomAccessFile(resultFileName
, "rw")) {long chunkSize
= param
.getChunkSize() == 0L ? DEFAULT_CHUNK_SIZE
: param
.getChunkSize().longValue();long offset
= chunkSize
* (param
.getChunkNumber() - 1);randomAccessFile
.seek(offset
);randomAccessFile
.write(param
.getFile().getBytes());} catch (IOException e
) {log
.error("文件上傳失敗：" + e
);return false;}return true;}private boolean uploadFileByMappedByteBuffer(String resultFileName
, FileChunkParam param
) {try (RandomAccessFile randomAccessFile
= new RandomAccessFile(resultFileName
, "rw");FileChannel fileChannel
= randomAccessFile
.getChannel()) {long chunkSize
= param
.getChunkSize() == 0L ? DEFAULT_CHUNK_SIZE
: param
.getChunkSize().longValue();long offset
= chunkSize
* (param
.getChunkNumber() - 1);byte[] fileBytes
= param
.getFile().getBytes();MappedByteBuffer mappedByteBuffer
= fileChannel
.map(FileChannel.MapMode.READ_WRITE
, offset
, fileBytes
.length
);mappedByteBuffer
.put(fileBytes
);unmap(mappedByteBuffer
);} catch (IOException e
) {log
.error("文件上傳失敗：" + e
);return false;}return true;}private boolean uploadSingleFile(String resultFileName
, FileChunkParam param
) {File saveFile
= new File(resultFileName
);try {param
.getFile().transferTo(saveFile
);localStorageService
.saveLocalStorage(param
);} catch (IOException e
) {log
.error("文件上傳失敗：" + e
);return false;}return true;}public static void unmap(final MappedByteBuffer mappedByteBuffer
) {try {if (mappedByteBuffer
== null) {return;}mappedByteBuffer
.force();AccessController.doPrivileged((PrivilegedAction<Object>) () -> {try {Method getCleanerMethod
= mappedByteBuffer
.getClass().getMethod("cleaner");getCleanerMethod
.setAccessible(true);Cleaner cleaner
=(Cleaner) getCleanerMethod
.invoke(mappedByteBuffer
, new Object[0]);cleaner
.clean();} catch (Exception e
) {log
.error("MappedByteBuffer 釋放失敗：" + e
);}System.out
.println("clean MappedByteBuffer completed");return null;});} catch (Exception e
) {log
.error("unmap error:" + e
);}}
}
其他實現的細節可自己查看源碼，也可以根據自己的想法在這個demo中進行拓展。理清楚其中的大文件傳輸、秒傳、斷點續傳后，自己開發一個小網盤也不是什么難事了 _
源碼下載
https://gitee.com/KT1205529635/simple-uploader
總結
以上是生活随笔為你收集整理的Spring学习笔记（三十六）——SpringBoot 实现大文件分片上传、断点续传及秒传的全部內容，希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。
