delphi 通过TNetHTTPClient解析抖音无水印高清视频原理及解决X-Bogus签名验证2023-5-1
一、雜談
????????最近有很多熱心網友反饋抖音去水印又不行了,之前是時不時被blocked,現在直接連內容都沒有了,返回直接就是空了,我們今天簡要給大家分析一下請求過程,附上delphi 源碼,及生成簽名驗證,成功請求到json數據的解決方法。
二、請求過程分析
我們還是先獲取一個抖音鏈接
https://v.douyin.com/A2VSVxc/
通過訪問重定向
https://www.douyin.com/video/7065264218437717285
然后提取到其中的視頻ID
7065264218437717285
如果是之前,我們會直接GET請求
https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=7065264218437717285
然后就能得到響應內容了。
但是這種方法已經失效了,今天我們會講解如何在增加一些請求頭參數以及X-Bogus后,可以仍然獲取到JSON格式的數據。如:
{"aweme_detail":{"anchors":null,"authentication_token":"......
.........}
可以看到,獲取到的aweme_detail json數據和以前一樣。
三、URL參數X-Bogus
X-Bogus你可以理解為是一個根據視頻ID及user-agent通過JS生成的用戶信息參數,它可以用于校驗。
詳細的一篇分析可以參考Freebuf上的《【JS 逆向百例】某音 X-Bogus 逆向分析,JSVMP 純算法還原》。
下面是完整的delphi 源碼解析類,主要流程如下:
1.傳入抖音分享鏈接:
https://v.douyin.com/A2VSVxc/
重定向得到:
https://www.douyin.com/video/7065264218437717285
2.提取到其中的視頻ID:
7065264218437717285
3.無水印視頻接口不變:
https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=7065264218437717285
4.(增加步驟4)根據X-Bogus 算法,傳入url鏈接及USER_AGENT數據,生成一個形如:
https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=7065264218437717285&X-Bogus=DFSzswSL2MtANHxFtG3DB09WcBjv
一個攜帶X-Bogus簽名驗證字段的請求鏈接。使用這個鏈接發送GET請求,就能得到aweme_detail 的json 數據了。不信大家可以試試。不過,這個鏈接是不能
在瀏覽器直接訪問的,還必須加上cookie,refer等請求頭數據,詳情看下面的Tdouyin解析類。
5.關于高清無水印視頻鏈接的獲取方法
從"aweme_detail" ?json數據解析出視頻的Uri項,帶入高清視頻接口:
https://aweme.snssdk.com/aweme/v1/play/?video_id=v0200fg10000c86doo3c77uai4m711qg&ratio=1080p&line=0
執行重定向getRedirectedUrl()得到高清無水印鏈接:
https://v95-p-cold.douyinvod.com/9f8215c6204afafffee302e612317776/64201324/video/tos/cn/tos-cn-ve-15c001-alinc2/35721d123b6243cca42398b0c5243c32/?a=1128&ch=0&cr=0&dr=0&cd=0%7C0%7C0%7C0&cv=1&br=2209&bt=2209&cs=0&ds=4&ft=bvjWJkQQqUsmfd4ZFo0OW_EklpPiXnlFZMVJEEy8kdbPD-I&mime_type=video_mp4&qs=0&rc=NDU3Omc3aDY8ZGc7OTkzOUBpajQ6N2Q6ZnJnOzMzNGkzM0BiXjE0NTQvXmExNTVeNTU2YSNuLmpmcjQwbDNgLS1kLS9zcw%3D%3D&l=20230326163724C8959375177E24BE6CEE&btag=a8000
詳細步驟看以下TDouyin解析類,關鍵代碼處都有注解:
1.解析類:
unit uDouyin;interface useswindows,classes,System.Net.URLClient, System.Net.HttpClient, System.Net.HttpClientComponent,System.SysUtils,strutils,uLog,System.RegularExpressions,uFuncs,system.JSON,uConfig,uVideoInfo,uDownVideo; constwm_user=$0400;wm_downfile=wm_user+100+1; //消息參數;//USER_AGENT標識客戶端的類型,這兒是電腦瀏覽器端。USER_AGENT:string='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36';//USER_AGENT標識客戶端的類型,這兒是手機APP端。USER_AGENT_PHONE:string='Mozilla/5.0 (iPhone; CPU iPhone OS 15_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6 Mobile/15E148 Safari/604.1';USER_AGENT_PHONE_2:string='TikTok 26.2.0 rv:262018 (iPhone; iOS 14.4.2; en_US) Cronet';//無水印視頻接口,跟以前一樣。DOUYIN_API_URL:string='https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=' ;DOUYIN_API_URL_2:string='https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=%s&aid=1128&version_name=23.5.0&device_platform=android&os_version=2333' ;//高清無水印視頻接口:DOUYIN_API_URL_1080 = 'https://aweme.snssdk.com/aweme/v1/play/?video_id=%s&ratio=1080p&line=0';typeTDouyin=class(TThread) //支持多線程下載;privateFId:cardinal; //線程標識;Furl:string; //分享的鏈接;;FRedirectedUrl:string; //重定向后的鏈接;Fvideourl:string; //解析后得到的無水印視頻鏈接;FvideoId:string; //視頻id 如:7065264218437717285FvideoTitle:string; //視頻標題Fnickname:string; //作者昵稱FcoverUrl:string; //視頻封面鏈接Fmsg:string; //線程消息Fsavedir:string; //保存視頻文件及封面圖片的目錄Furl_1080:string; //高清視頻鏈接Furi_1080:string; //高清視頻uri參數 不懂的+v:metabycfFphotos:string;class var Fcookie: string; //cookie參數 ,可從瀏覽器獲取 靜態類成員class var Fform: HWND; //接收消息的窗體句柄 類成員procedure SetId(id:cardinal); //設置線程idprocedure SetSaveDir(dir:string); //設置保存目錄class procedure SetForm(const hForm: HWND); static; //設置窗體句柄 靜態方法class procedure SetCookie(const cookie: string); static; //設置cookie 靜態方法protectedprocedure Execute; override;publicconstructor Create(id:cardinal;url:string);destructor Destroy;property id:cardinal read FId write SetId; //id屬性property url:string read Furl; //分享鏈接 屬性property msg:string read Fmsg; //線程消息 屬性property videourl:string read Fvideourl; //無水印視頻鏈接 屬性property videoTitle:string read FvideoTitle; //視頻標題 屬性property nickname:string read Fnickname; //用戶昵稱property RedirectedUrl:string read FRedirectedUrl; //重定向鏈接 屬性property videoId:string read FvideoId; //視頻id 屬性 如:7065264218437717285property coverUrl:string read FcoverUrl; //封面鏈接 屬性property url_1080:string read Furl_1080; //高清視頻鏈接 屬性property photos:string read Fphotos;property savedir:string read Fsavedir write setSaveDir; //保存目錄 屬性function getRedirectedUrl(url:string):string;overload; //獲取重定向鏈接function getRedirectedUrl(url,refer,user_agent:string):string;overload; //獲取重定向鏈接function getVideoId(txt:string):string; //解析出視頻id 如:7065264218437717285function getVideoUrl():string; //解析無水印視頻地址,封面鏈接,視頻標題 工作流程方法在這兒:function parseJson(jo:string):string; //解析aweme_detail json數據class property form: HWND read Fform write SetForm; //窗體句柄 類屬性class property cookie: string read Fcookie write SetCookie; //cookie 類屬性function getPostResult(data:string):string; //post 請求function getRequestResult2(apiurl:string;Cookie:string):string; //GET 請求function getBogusUrl(url:string):string; X-Bogus 算法 不明白的+v:metabycfend; implementation//解析無水印視頻地址,封面鏈接,視頻標題 工作流程方法在這兒: function TDouyin.getVideoUrl():string; varapiurl,apiurl2,jo:string;i:integer;video:TvideoInfo;down:TdownVideo; beginresult:='';FcoverUrl:='';FvideoUrl:='';Fphotos:=''; try//第一步:執行重定向,從而獲取到視頻id//如:https://www.douyin.com/video/7065264218437717285FRedirectedUrl:=getRedirectedUrl(Furl,Furl,USER_AGENT);log('FRedirectedUrl='+FRedirectedUrl); //日志記錄if(FRedirectedUrl)='' then exit;//第二步:分析出視頻id,如:7065264218437717285FvideoId:=getVideoId(FRedirectedUrl);log('FvideoId='+FvideoId); //日志記錄if(FvideoId)='' then exit;//apiurl:=DOUYIN_API_URL+FvideoId; //視頻接口apiurl:=format(DOUYIN_API_URL_2,[FvideoId]); //視頻接口//第三步:計算X-Bogus驗證,加到視頻接口上。得到新的請求鏈接 多了這一步驟。//如:https://www.douyin.com/aweme/v1/web/aweme/detail/?aweme_id=7065264218437717285&X-Bogus=DFSzswSL2MtANHxFtG3DB09WcBjv//不明白的+v:metabycfapiurl2:=getBogusUrl(apiurl); //具有X-Bogus驗證的視頻接口 多了這一步驟。log(apiurl2); //日志記錄if(apiurl2='')then begin log('apiurl2=k');exit;end;//第四步:發送GET請求,帶上cookie,refer參數;到這一步,已經能拿到"aweme_detail" json數據了。jo:=getRequestResult2(apiurl2,Fcookie);Fmsg:=jo;log(jo); //日志記錄if(pos('aweme_detail',jo)<=0)then beginlog('aweme_detail=k');exit;end;if(pos('"aweme_detail":null',jo)>0)then exit;//第五步: 解析 "aweme_detail" json數據parseJson(jo);//第六步: 1解析 圖文if(Fphotos<>'')thenbeginvideo:=TvideoInfo.Create(Fvideotitle,coverUrl,'',Fphotos);down:=TdownVideo.Create(Fid,video,Fsavedir,false);//down.form:=Fform;//down.cookie:=Fcookie;//if(DEBUG=true)then downvideo.process elsedown.Start;exit;end;//第六步: 2解析 高清視頻地址if(Furi_1080<>'')then //Furi_1080為視頻 uribeginFurl_1080:=format(DOUYIN_API_URL_1080,[Furi_1080]);log(Furl_1080); //日志記錄Furl_1080:=getRedirectedUrl(Furl_1080,FRedirectedUrl, USER_AGENT_PHONE); //重定向log('Furl_1080='+Furl_1080); //日志記錄end;//第七步: 啟動下載線程,下載視頻文件和封面圖片。if(Fvideotitle<>'')and(Furl_1080<>'')and(FcoverUrl<>'')thenbeginvideo:=TvideoInfo.Create(Fvideotitle,coverUrl,Furl_1080,'');down:=TdownVideo.Create(Fid,video,Fsavedir,false);//down.form:=Fform;//down.cookie:=Fcookie;down.Start;end; finally//第八步: 發送解析完成消息。Fmsg:='complete';SendMessage(Fform,wm_downfile,2,integer(self)); end;end;//第四步:發送GET請求,帶上cookie,refer參數;到這一步,已經能拿到"aweme_detail" json數據了。 function TDouyin.getRequestResult2(apiurl:string;Cookie:string):string; varclient: TNetHTTPClient;ss: TStringStream;s,id:string;AResponse:IHTTPResponse;i:integer; begin tryclient := TNetHTTPClient.Create(nil);SS := TStringStream.Create('', TEncoding.UTF8);ss.Clear;with client dobeginConnectionTimeout := 10000; // 10秒ResponseTimeout := 10000; // 10秒AcceptCharSet := 'utf-8';UserAgent := USER_AGENT; //1 USER_AGENT USER_AGENT_PHONE_2client.AllowCookies:=true;client.HandleRedirects:=true;Accept:='application/json'; //'*/*'client.ContentType:='application/json'; //2client.AcceptLanguage:='zh-CN';client.CustomHeaders['Cookie'] := cookie;client.CustomHeaders['Referer'] := Furl;tryAResponse:=Get(apiurl, ss);result:=ss.DataString;excepton E: Exception doLog(e.Message);end;end; finallyss.Free;client.Free; end;end;//第五步: 解析 "aweme_detail" json數據 :分為視頻和圖文兩類 function TDouyin.parseJson(jo:string):string; varjson,jroot,jvideo,j1,j2: TJSONObject;arr,arr1:TJSONARRAY;uri,aweme_type,photo:string;i:integer; beginresult:=''; tryjson := TJSONObject.ParseJSONValue(jo) as TJSONObject;if json = nil then exit;jroot:=json.GetValue('aweme_detail') as TJSONObject;FvideoTitle:=trim(jroot.GetValue('desc').Value);aweme_type:=jroot.GetValue('aweme_type').Value;if(aweme_type='68')then //圖文beginarr:=jroot.GetValue('images') as TJSONARRAY;for I := 0 to arr.Size-1 dobeginj1:=arr.Get(i) as TJSONObject;arr1:=j1.GetValue('url_list') as TJSONARRAY;photo:=arr1.Items[0].Value;Fphotos:=Fphotos+photo+#13#10;end;result:='#100#'+Fphotos+'#'+FcoverUrl+'#'+FvideoTitle;exit;end;jvideo:=jroot.GetValue('video') as TJSONObject;j1:=jvideo.GetValue('cover') as TJSONObject; //cover origin_coverarr:=j1.GetValue('url_list') as TJSONARRAY;FcoverUrl:=arr[0].Value;j1:=jvideo.GetValue('play_addr') as TJSONObject;arr:=j1.GetValue('url_list') as TJSONARRAY;FvideoUrl:=arr[0].Value;FvideoUrl:=stringreplace(FvideoUrl,'playwm','play',[rfReplaceAll]);Furi_1080:=j1.GetValue('uri').Value;result:='#100#'+FvideoUrl+'#'+FcoverUrl+'#'+FvideoTitle;finallyif json <> nil then json.Free; end; end;//第二步:分析出視頻id,如:7065264218437717285 function TDouyin.getVideoId(txt:string):string; varm:TMatch;i:integer; beginresult:='';m := TRegEx.Match(txt,'/video/([^/?]+)/');if(m.Groups[1].Success=false) or (length(m.Groups[1].Value)<>19)then exit;result:=m.Groups[1].Value;end;//X-Bogus 算法 不明白的+v:metabycf function TDouyin.getBogusUrl(url:string):string; varjson:TJSONObject;data:string; beginresult:=''; tryjson:=TJSONObject.Create;json.AddPair('url',url);json.AddPair('user_agent',USER_AGENT);data:=json.ToString;log(data);data:=getPostResult(data);log(data);if(data='')then exit;json:=TJSONObject.ParseJSONValue(data) as TJSONObject;result:=json.GetValue('param').Value; finallyjson.Free; end; end;//第一步:執行重定向,從而獲取到視頻id function TDouyin.getRedirectedUrl(url,refer,user_agent:string):string; varclient: TNetHTTPClient;ss: TStringStream;s,id:string;AResponse:IHTTPResponse;i:integer; begin tryclient := TNetHTTPClient.Create(nil);SS := TStringStream.Create('', TEncoding.UTF8);ss.Clear;with client dobeginConnectionTimeout := 2000; // 2秒ResponseTimeout := 2000; // 10秒AcceptCharSet := 'utf-8';UserAgent := user_agent;client.AllowCookies:=true;client.HandleRedirects:=false;Accept:='*/*';client.CustomHeaders['Referer'] := refer;tryAResponse:=Get(url, ss);Log('getRedirectedUrl AResponse='+ss.DataString);s:=AResponse.HeaderValue['Location'];if(s='')then exit;result:=s;excepton E: Exception doLog(e.Message);end;end; finallyss.Free;client.Free; end; end;constructor TDouyin.Create(id:cardinal;url:string); begin//inherited;//FreeOnTerminate := True;inherited Create(True);FId:=id;Furl:=url; //分享鏈接Furi_1080:=''; //視頻uriFphotos:=''; end; destructor TDouyin.Destroy; begininherited Destroy; end;//工作線程 procedure TDouyin.Execute; begin trygetVideoUrl(); finallyend; end;//------------------------------------------屬性方法-------------------------------------procedure TDouyin.SetId(Id:cardinal);beginFId:=Id;end;class procedure TDouyin.SetForm(const hForm: HWND);beginFform:=hForm;end;procedure TDouyin.SetSaveDir(dir:string);beginFsavedir:=dir;end;class procedure TDouyin.SetCookie(const cookie: string);beginFcookie:=cookie;end;end.2、視頻信息
unit uVideoInfo;interface typeTVideoInfo=classprivateFtitle:string; //標題FcoverUrl:string; //封面地址FvideoUrl:string; //視頻地址Fphotos:string; //圖片地址procedure SetTitle(title:string);publicproperty title:string read Ftitle write Settitle;property coverUrl:string read FcoverUrl;property videoUrl:string read FvideoUrl;property photos:string read Fphotos;constructor Create(title,coverUrl,videoUrl,photos:string);end; implementation constructor TVideoInfo.Create(title,coverUrl,videoUrl,photos:string); beginFtitle:=title;FcoverUrl:=coverUrl;FvideoUrl:=videoUrl;Fphotos:=photos; end; procedure TVideoInfo.Settitle(title:string); beginFtitle:=title; end; end.3、下載
unit uDownVideo;interface useswindows,classes,System.Net.URLClient, System.Net.HttpClient, System.Net.HttpClientComponent,System.SysUtils,strutils,uLog,System.RegularExpressions,uFuncs,system.JSON,uConfig,uVideoInfo,WinInet,urlmon,shlobj,ioutils; constwm_user=$0400;wm_downfile=wm_user+100+1;USER_AGENT:string='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36'; typeTDownVideo=class(TThread)privateFId:cardinal;Fvideo:TvideoInfo;Fsavedir:string;FvideoFilename:string;FcoverFilename:string;FphotoFilename:string; //圖片文件名Fmsg:string;FSerial:boolean; //是否在文件名前面加入序號;Fsuccess:boolean;Frefer:string;class var Fcookie: string;procedure SetId(id:cardinal);procedure SetSavedir(dir:string);procedure SetRefer(refer:string);class var Fform: HWND;class procedure SetForm(const hForm: HWND); static;class procedure SetCookie(const cookie: string); static;function DownloadFile(SourceFile, DestFile: string): Boolean;protectedprocedure Execute; override;publicconstructor Create(id:cardinal;video:TvideoInfo;savedir:string;bSerial:boolean);destructor Destroy;property id:cardinal read FId write SetId;property savedir:string read Fsavedir write SetSavedir;property video:TvideoInfo read Fvideo;property msg:string read Fmsg;property videoFilename:string read FvideoFilename;property coverFilename:string read FcoverFilename;property photoFilename:string read FphotoFilename;property serial:boolean read Fserial;property success:boolean read Fsuccess;property refer:string read Frefer write SetRefer;class property cookie: string read Fcookie write SetCookie;class property form: HWND read Fform write SetForm;function formatFilename(caption:string):string;function formatDir(dir:string):string;function GetValidName(s:string):string;procedure downloadFileLog(SourceFile, DestFile: string);procedure process();end; implementation //bSerial為是否給文件名加上序號 constructor TDownVideo.Create(id:cardinal;video:TvideoInfo;savedir:string;bSerial:boolean); begin//inherited;//FreeOnTerminate := True;inherited Create(True);FId:=id;Fvideo:=video;Fsavedir:=formatdir(savedir);Fmsg:='';FvideoFilename:='';FcoverFilename:='';FphotoFilename:='';Fsuccess:=false;Fserial:=bSerial;if(not directoryexists(Fsavedir))thenforcedirectories(Fsavedir); end; destructor TDownVideo.Destroy; begininherited Destroy; end; //下載流程 procedure TDownVideo.process(); vardir,title,photoUrl,photoname:string;b:boolean;photolist:tstrings;i:integer; beginphotolist:=nil; trytitle:=formatfilename(trim(video.title));if(Fvideo.photos='')then //下載視頻beginif(Fserial=true)thenbeginFcoverFilename:=Fsavedir+'\'+inttostr(Fid)+'.'+title+'.webp';FvideoFilename:=Fsavedir+'\'+inttostr(Fid)+'.'+title+'.mp4';end else beginFcoverFilename:=Fsavedir+'\'+title+'.webp';FvideoFilename:=Fsavedir+'\'+title+'.mp4';end;if(video.coverUrl<>'')then downloadFileLog(Fvideo.coverUrl,FcoverFilename);if(video.videoUrl<>'')then downloadFileLog(Fvideo.videoUrl,FvideoFilename);end else begin //下載圖片photolist:=tstringlist.Create;photolist.Text:=video.photos;if(Fserial=true)thendir:=Fsavedir+'\'+inttostr(Fid)+'.'+titleelsedir:=Fsavedir+'\'+title;forcedirectories(dir);if(Fvideo.coverUrl<>'')thenbeginFcoverFilename:=dir+'\0.封面 '+title+'.webp';downloadFileLog(Fvideo.coverUrl,FcoverFilename);end;for I := 0 to photolist.Count-1 dobeginphotoUrl:=photolist[i];if(trim(photoUrl)='')then continue;photoname:=dir+'\'+inttostr(i+1)+'.'+title+'.webp';downloadFileLog(photoUrl,photoname);FphotoFilename:=FphotoFilename+photoname+#13#10;end;end;finallyFmsg:='complete';if(photolist<>nil)then photolist.Free;SendMessage(Fform,wm_downfile,1,integer(self)); end; end; //線程中執行下載 procedure TDownVideo.Execute; beginprocess(); end; function TDownVideo.GetValidName(s:string):string; varc:char;txt:string; begintxt:=s;for c in TPath.GetInvalidFileNameChars() dobegintxt:=stringreplace(txt,c,'',[rfReplaceAll]);end;result:=txt; end; //去除文件名中的非法字符 function TDownVideo.formatFilename(caption:string):string; vars:string; begins:=caption;if(length(s)>72)then s:=leftstr(s,72);result:=GetValidName(s); end; //去除路徑中的非法字符 function TDownVideo.formatDir(dir:string):string; varcaption:string; begincaption:=trim(extractfilename(dir));if(length(caption)>72)then caption:=leftstr(caption,72);caption:=GetValidName(caption);result:=extractfilepath(dir)+caption; end; //下載文件 procedure TDownVideo.downloadFileLog(SourceFile, DestFile: string); beginif(fileexists(DestFile))then deletefile(DestFile);if(Downloadfile(SourceFile,DestFile))thenbeginlog('成功:'+DestFile);Fsuccess:=true;end else beginlog('失敗:'+DestFile+' '+SourceFile);Fsuccess:=false;end; end; //下載文件 function TDownVideo.DownloadFile(SourceFile, DestFile: string): Boolean; begin tryDeleteUrlCacheEntry(pchar(SourceFile));Result := UrlDownloadToFile(nil, PChar(SourceFile), PChar(DestFile), 0, nil) = 0; exceptResult := False; end; end;//------------------------------------------屬性方法-------------------------------------procedure TDownVideo.SetSavedir(dir:string);beginFsavedir:=dir;end;procedure TDownVideo.SetRefer(refer:string);beginFrefer:=refer;end;class procedure TDownVideo.SetCookie(const cookie: string);beginFcookie:=cookie;end;procedure TDownVideo.SetId(Id:cardinal);beginFId:=Id;end;class procedure TDownVideo.SetForm(const hForm: HWND);beginFform:=hForm;end;end.需要技術支持及成品的+v:metabycf
總結
以上是生活随笔為你收集整理的delphi 通过TNetHTTPClient解析抖音无水印高清视频原理及解决X-Bogus签名验证2023-5-1的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 聊聊Web App、Hybrid App
- 下一篇: 3D渲染引擎介绍