引用處：
【問(wèn)題解決】HttpClient解析服務(wù)器返回的response出現(xiàn)亂碼

問(wèn)題場(chǎng)景
最近在用httpClient做網(wǎng)絡(luò)爬蟲(chóng)的時(shí)候，遇到了一個(gè)不大不小的問(wèn)題，當(dāng)使用HttpGet向指定網(wǎng)址發(fā)送請(qǐng)求后，接收到的Response無(wú)法正常解析，出現(xiàn) 口口？？這樣的亂碼，編碼也考慮到了中文編碼，具體代碼如下：

//處理邏輯 HttpResponse response = HttpUtils.doGet(baseUrl + title + postUrl, headers); InputStream is = getInputStreamFromResponse(response); responseText = Utils.getStringFromInputStream(in); result = EncodeUtils.unicdoeToGB2312(responseText);//上面使用到的函數(shù)public static HttpResponse doGet(String url, Map<String, String> headers) {HttpClient client = createHttpClient();HttpGet getMethod = new HttpGet(url);HttpResponse response = null;response = client.execute(getMethod);return response;}public static String getStringFromStream(InputStream in) {StringBuilder buffer = new StringBuilder();BufferedReader reader = null;reader = new BufferedReader(new InputStreamReader(in, "UTF-8"));String line = null;while ((line = reader.readLine()) != null) {buffer.append(line + "\n");}reader.close();return buffer.toString();}

解析到的結(jié)果如下圖所示，一團(tuán)亂麻：

解決方案
上面的代碼基本邏輯是沒(méi)有問(wèn)題的，也考慮到了中文的編碼，但是卻有一個(gè)很隱秘的陷阱在里面，一般的網(wǎng)站都先將網(wǎng)頁(yè)壓縮后再傳回給瀏覽器，減少傳輸時(shí)間，如下圖所示：

而上面的處理邏輯則沒(méi)有考慮到Response的inputStream是經(jīng)過(guò)壓縮的，需要使用對(duì)應(yīng)的數(shù)據(jù)流對(duì)象處理，圖中使用的content-encoding是gzip格式，則需要使用GZIPInputStream對(duì)其進(jìn)行處理，只需要對(duì)上文中的函數(shù)public static String getStringFromStream(InputStream in)改進(jìn)即可，如下所示：

public static String getStringFromResponse(HttpResponse response) {if (response == null) {return null;}String responseText = "";InputStream in = getInputStreamFromResponse(response);Header[] headers = response.getHeaders("Content-Encoding");for(Header h : headers){if(h.getValue().indexOf("gzip") > -1){//For GZip responsetry{GZIPInputStream gzin = new GZIPInputStream(is);InputStreamReader isr = new InputStreamReader(gzin,"utf-8");responseText = Utils.getStringFromInputStreamReader(isr);}catch (IOException exception){exception.printStackTrace();}break;}}responseText = Utils.getStringFromStream(in);return responseText;}

最終得到的結(jié)果就是人能夠看懂的了：

問(wèn)題原因
在分析服務(wù)器返回response時(shí)，只注意到content-type是text/html，表明我們可以用文本解析的方式來(lái)獲取response的內(nèi)容，如果content-type是excel，則表明我們可以用excel軟件來(lái)讀取內(nèi)容。content-type表明的是內(nèi)容是以何種格式組織的。但是我卻忽略了content-encoding這個(gè)字段，content-encoding字段表明的是服務(wù)器以何種方式來(lái)對(duì)傳輸?shù)膬?nèi)容進(jìn)行額外編碼，例如壓縮，如果content-encoding是gzip，則表明服務(wù)器是以Gzip格式壓縮數(shù)據(jù)，而數(shù)據(jù)本身的格式可以是文本，也可以是視頻等。兩者需要區(qū)分對(duì)待。但是需要注意的是，有的服務(wù)器雖然返回的是gzip的content-encoding，而實(shí)際上卻并沒(méi)有對(duì)內(nèi)容進(jìn)行g(shù)zip編碼，所以有可能會(huì)出現(xiàn)gzip解碼失敗。

（官方原文解釋）
RFC 2616 for HTTP 1.1 specifies how web servers must indicate encoding transformations using the Content-Encoding header. Although on the surface, Content-Encoding (e.g., gzip, deflate, compress) and Content-Type(e.g., x-application/x-gzip) sound similar, they are, in fact, two distinct pieces of information. Whereas servers use Content-Type to specify the data type of the entity body, which can be useful for client applications that want to open the content with the appropriate application, Content-Encoding is used solely to specify any additional encoding done by the server before the content was transmitted to the client. Although the HTTP RFC outlines these rules pretty clearly, some web sites respond with “gzip” as the Content-Encoding even though the server has not gzipped the content.
Our testing has shown this problem to be limited to some sites that serve Unix/Linux style “tarball” files. Tarballs are gzip compressed archives files. By setting the Content-Encoding header to “gzip” on a tarball, the server is specifying that it has additionally gzipped the gzipped file. This, of course, is unlikely but not impossible or non-compliant.
Therein lies the problem. A server responding with content-encoding, such as “gzip,” is specifying the necessary mechanism that the client needs in order to decompress the content. If the server did not actually encode the content as specified, then the client’s decompression would fail.

總結(jié)

以上是生活随笔為你收集整理的HttpClient解析服务器返回的response出现乱码的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。