python提交post请求payload webkit_python爬虫实现POST request payload形式的请求
1. 背景
最近在爬取某個(gè)站點(diǎn)時(shí)，發(fā)現(xiàn)在POST數(shù)據(jù)時(shí)，使用的數(shù)據(jù)格式是request payload，有別于之前常見(jiàn)的 POST數(shù)據(jù)格式(Form data)。而使用Form data數(shù)據(jù)的提交方式時(shí)，無(wú)法提交成功。
1.1. Http請(qǐng)求中Form Data 和 Request Payload的區(qū)別
AJAX Post請(qǐng)求中常用的兩種傳參數(shù)的形式：form data 和 request payload
1.1.1. Form data
get請(qǐng)求的時(shí)候，我們的參數(shù)直接反映在url里面，形式為key1=value1&key2=value2形式，比如：
http://news.baidu.com/ns?word=NBA&tn=news&from=news&cl=2&rn=20&ct=1
而如果是post請(qǐng)求，那么表單參數(shù)是在請(qǐng)求體中，也是以key1=value1&key2=value2的形式在請(qǐng)求體中。通過(guò)chrome的開(kāi)發(fā)者工具可以看到，如下：
RequestURL:http://127.0.0.1:8080/test/test.do
Request Method:POST
Status Code:200 OK
Request Headers
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:zh-CN,zh;q=0.8,en;q=0.6
AlexaToolbar-ALX_NS_PH:AlexaToolbar/alxg-3.2
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:25
Content-Type:application/x-www-form-urlencoded
Cookie:JSESSIONID=74AC93F9F572980B6FC10474CD8EDD8D
Host:127.0.0.1:8080
Origin:http://127.0.0.1:8080
Referer:http://127.0.0.1:8080/test/index.jsp
User-Agent:Mozilla/5.0 (Windows NT 6.1)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36
Form Data
name:mikan
address:street
Response Headers
Content-Length:2
Date:Sun, 11 May 2014 11:05:33 GMT
Server:Apache-Coyote/1.1
這里要注意post請(qǐng)求的Content-Type為application/x-www-form-urlencoded(默認(rèn)的)，參數(shù)是在請(qǐng)求體中，即上面請(qǐng)求中的Form Data。
前端代碼：提交數(shù)據(jù)
xhr.setRequestHeader("Content-type","application/x-www-form-urlencoded");
xhr.send("name=foo&value=bar");
后端代碼：接收提交的數(shù)據(jù)。在servlet中，可以通過(guò)request.getParameter(name)的形式來(lái)獲取表單參數(shù)。
/**
* 獲取httpRequest的參數(shù)
*
* @param request
* @param name
* @return
*/
protected String getParameterValue(HttpServletRequest request, String name) {
return StringUtils.trimToEmpty(request.getParameter(name));
}
1.1.2. Request payload
如果使用原生AJAX POST請(qǐng)求的話(huà)，那么請(qǐng)求在chrome的開(kāi)發(fā)者工具的表現(xiàn)如下，主要是參數(shù)在
Remote Address:192.168.234.240:80
Request URL:http://tuanbeta3.XXX.com/qimage/upload.htm
Request Method:POST
Status Code:200 OK
Request Headers
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip,deflate,sdch
Accept-Language:zh-CN,zh;q=0.8,en;q=0.6
Connection:keep-alive
Content-Length:151
Content-Type:application/json;charset=UTF-8
Cookie:JSESSIONID=E08388788943A651924CA0A10C7ACAD0
Host:tuanbeta3.XXX.com
Origin:http://tuanbeta3.XXX.com
Referer:http://tuanbeta3.XXX.com/qimage/customerlist.htm?menu=19
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
X-Requested-With:XMLHttpRequest
Request Payload
[{widthEncode:NNNcaXN, heightEncode:NNNN5NN, displayUrl:201409/03/66I5P266rtT86oKq6,…}]
Response Headers
Connection:keep-alive
Content-Encoding:gzip
Content-Type:application/json;charset=UTF-8
Date:Thu, 04 Sep 2014 06:49:44 GMT
Server:nginx/1.4.7
Transfer-Encoding:chunked
Vary:Accept-Encoding
總結(jié)
以上是生活随笔為你收集整理的python提交post请求payload webkit_python爬虫实现POST request payload形式的请求的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。
- 上一篇： 怎样导入python colorama模
- 下一篇： ubuntu 升级python3.8_U