远程文件下载/小电影下载
場景:某些網(wǎng)站只提供了在線預(yù)覽,或你充值了1天會員、但想把所有電影都看完 -> OK,咱們下載下來慢慢看
分析:目標(biāo)網(wǎng)站的電影是分塊加載的,一次加載3-4個文件,播放完畢再發(fā)起請求獲取后面的資源,舉例如下:
http://IP:PORT/1jxxl/JXXL846JDG/JXXL846JDG0.ts
http://IP:PORT/1jxxl/JXXL846JDG/JXXL846JDG1.ts
.......
http://IP:PORT/1jxxl/JXXL846JDG/JXXL846JDG218.ts
http://IP:PORT/1jxxl/JXXL846JDG/JXXL846JDG219.ts
這里可以看到,電影的資源加載是有規(guī)律可循的,都是后面的數(shù)字累加,對此我只能呵呵呵了...
可以理解為閑得無聊吧,寫了個下載電影的程序
基本思路:取得電影分塊文件其中一個下載鏈接,存入text文件中,根據(jù)后面資源文件有跡可循的加載規(guī)律,自動切換下載地址,并將目標(biāo)文件下載到本地
親測功能正常...直接上代碼了:
import java.io.*; import java.net.HttpURLConnection; import java.net.ProtocolException; import java.net.URL; import java.nio.file.Files; import java.nio.file.Paths; import java.util.List; import java.util.concurrent.*; import java.util.concurrent.atomic.AtomicInteger;/**** 小電影下載* @date 2019/9/15 16:27* @author wei.heng*/ public class FilmDownload {public static void main(String[] args) {// 下載到哪個目錄下final String root = "G:/haha";String path = "C:/Users/Thinkpad/Desktop/test.txt";String prefixAlia = "American_";List<String> pathList = getFilePath(path);for (String p : pathList) {System.out.println(p);ThreadPoolExecutor executor = new ThreadPoolExecutor(4, 8, 60L, TimeUnit.SECONDS, new LinkedBlockingQueue<>(50), new NameTreadFactory());executor.execute(() -> downloadTs(prefixAlia, root, p));}}static class NameTreadFactory implements ThreadFactory {private final AtomicInteger mThreadNum = new AtomicInteger(1);@Overridepublic Thread newThread(Runnable r) {Thread t = new Thread(r, "my-thread-" + mThreadNum.getAndIncrement());System.out.println(t.getName() + " has been created");return t;}}private static List<String> getFilePath(String filePath){List<String> list = null;try {list = Files.readAllLines(Paths.get(filePath));} catch (IOException e) {e.printStackTrace();}return list;}/*** 按文件名建立目錄,下載遠程文件(網(wǎng)上這些個電影全是分塊加載的,后續(xù)下載完還要做合并)* @param dirAlias 目錄別名(下載的數(shù)據(jù)太多了,我按電影類型分類吧)* @param targetPath 輸出的目標(biāo)文件目錄* @param path 遠程文件地址* @date 2019/9/15 15:06* @author wei.heng*/private static void downloadTs(String dirAlias, String targetPath, String path) {final String fileSeperator = "/";int speratorIndex = path.lastIndexOf(fileSeperator);// 獲取遠程文件名String filename = path.substring(speratorIndex + 1);// 遠程文件目錄String remoteDir = path.substring(0, speratorIndex);// 獲取遠程文件目錄名稱String dir = remoteDir.substring(remoteDir.lastIndexOf(fileSeperator) + 1);// 文件太多,前綴加個別名吧dir = dirAlias + dir;// 分塊文件名稱String dynamicFileName = filename.substring(0,filename.length() - 4);String suffix = ".ts";BufferedInputStream bis = null;HttpURLConnection conn = null;// 這里隨意給了個值,保證文件能夠全部讀取完就成int filePartSize = 3000;for (int partIndex = 0; partIndex < filePartSize; partIndex++) {try {System.out.println("partIndexIndex:" + partIndex);// 檢查文件是否已存在,避免浪費資源,重復(fù)請求// 文件名 = 本地目錄(入?yún)?#xff09; + 文件名命名的目錄(程序常見) + 分塊文件名稱(dynamicFileName + partIndex + suffix)String outputFile = targetPath + fileSeperator + dir + fileSeperator + dynamicFileName + partIndex + suffix;File f = new File(outputFile);if(f.exists()){System.out.println("文件已存在:" + f);continue;}String targetUrl = remoteDir + fileSeperator + dynamicFileName + partIndex + suffix;URL url = new URL(targetUrl);conn = (HttpURLConnection)url.openConnection();// 設(shè)置通用的請求屬性setRequestProperties(conn);// 建立連接conn.connect();// 獲取響應(yīng)碼int httpResult = conn.getResponseCode();System.out.println("httpResult::"+httpResult);if(httpResult==HttpURLConnection.HTTP_OK){// 創(chuàng)建文件目錄File direction = new File(targetPath + fileSeperator + dir);if(!direction.exists()){if(!direction.mkdirs()){throw new RuntimeException("創(chuàng)建文件目錄失敗");}}Files.copy(conn.getInputStream(), Paths.get(outputFile));System.out.println("real length::"+conn.getContentLength());} else if(httpResult==HttpURLConnection.HTTP_NOT_FOUND){break;}} catch (IOException e) {e.printStackTrace();} finally {releaseSource(bis, conn);}}}private static void releaseSource(BufferedInputStream bis, HttpURLConnection conn) {if(bis != null){try {bis.close();} catch (IOException e) {e.printStackTrace();}}if(conn != null){conn.disconnect();}}private static void setRequestProperties(HttpURLConnection conn) throws ProtocolException {// 使用 URL 連接進行輸出conn.setDoOutput(true);// 使用 URL 連接進行輸入conn.setDoInput(true);// 忽略緩存conn.setUseCaches(false);conn.setRequestMethod("GET");conn.setRequestProperty("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");conn.setRequestProperty("Accept-Encoding", "gzip, deflate");conn.setRequestProperty("connection", "Keep-Alive");conn.setRequestProperty("user-agent","Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0");conn.setRequestProperty("Host", "video7.achxc.com:8083");}}到這里基本就OK了...
是不是覺得少了點什么??
1、爬蟲什么的,Java也可以寫,請求網(wǎng)站地址頁面解析獲取對應(yīng)title或src就好了
2、后面我會再貼一篇文件合并的文章(將下載后的電影分塊文件合并成一個)...其實就是分塊文件合并為一個...
總結(jié)
以上是生活随笔為你收集整理的远程文件下载/小电影下载的全部內(nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: “所选的用户密钥未在远程主机上注册,请再
- 下一篇: wsl2 局域网访问_超轻巧局域网传输神