我有一臺本地計算機(local_user@local_machine)。hadoop文件系統位于不同的服務器(some_user@another_server)上。hadoop服務器中的一個用戶名為target_user。如何從local_user@local_machine訪問target_user中的文件？更準確地說，假設在some_user@another_server上的HDFS中存在一個文件/user/target_user/test.txt。從local_user@local_machine訪問/user/target_user/test.txt時，我應該使用什么正確的文件路徑？在

{hdi}本身可以訪問文件。但是我無法使用我編寫的從HDFS讀寫的python腳本從本地機器訪問文件(需要3個參數-本地文件路徑、遠程文件路徑和讀或寫)，很可能是因為我沒有給出正確的路徑。在

我試過以下方法，但都不管用：

$ #local_user@local_machine

$ python3 rw_hdfs.py ./to_local_test.txt /user/target_user/test.txt read

$ python3 rw_hdfs.py ./to_local_test.txt some_user@another_server/user/target_user/test.txt read

所有這些都給出了完全相同的錯誤：Traceback (most recent call last):

File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 377, in _make_request

httplib_response = conn.getresponse(buffering=True)

TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 560, in urlopen

body=body, headers=headers)

File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 379, in _make_request

httplib_response = conn.getresponse()

File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse

response.begin()

File "/usr/lib/python3.5/http/client.py", line 297, in begin

version, status, reason = self._read_status()

File "/usr/lib/python3.5/http/client.py", line 279, in _read_status

raise BadStatusLine(line)

http.client.BadStatusLine:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/usr/lib/python3/dist-packages/requests/adapters.py", line 376, in send

timeout=timeout

File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 610, in urlopen

_stacktrace=sys.exc_info()[2])

File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 247, in increment

raise six.reraise(type(error), error, _stacktrace)

File "/usr/lib/python3/dist-packages/six.py", line 685, in reraise

raise value.with_traceback(tb)

File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 560, in urlopen

body=body, headers=headers)

File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 379, in _make_request

httplib_response = conn.getresponse()

File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse

response.begin()

File "/usr/lib/python3.5/http/client.py", line 297, in begin

version, status, reason = self._read_status()

File "/usr/lib/python3.5/http/client.py", line 279, in _read_status

raise BadStatusLine(line)

requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine('\x15\x03\x03\x00\x02\x02\n',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "python_hdfs.py", line 63, in

status, name, nnaddress= check_node_status(node)

File "python_hdfs.py", line 18, in check_node_status

request = requests.get("%s/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus"%name,verify=False).json()

File "/usr/lib/python3/dist-packages/requests/api.py", line 67, in get

return request('get', url, params=params, **kwargs)

File "/usr/lib/python3/dist-packages/requests/api.py", line 53, in request

return session.request(method=method, url=url, **kwargs)

File "/usr/lib/python3/dist-packages/requests/sessions.py", line 468, in request

resp = self.send(prep, **send_kwargs)

File "/usr/lib/python3/dist-packages/requests/sessions.py", line 576, in send

r = adapter.send(request, **kwargs)

File "/usr/lib/python3/dist-packages/requests/adapters.py", line 426, in send

raise ConnectionError(err, request=request)

requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('\x15\x03\x03\x00\x02\x02\n',))

總結

以上是生活随笔為你收集整理的文件系统位于服务器,如何访问hadoop文件系统上的文件，该文件系统位于与本地计算机不同的服务器上？...的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。