Hive多字符分割列

1 簡介

此手冊應用于hive建表時指定列按照多字段分割的情景。

2 準備

2.1 環境說明

hive版本: 1.1.0-cdh5.4.7

3 使用

3.1 使用說明

方法一)通過org.apache.hadoop.hive.contrib.serde2.RegexSerDe格式的serde。

1)? 建表語句

#指定以^|~作為分隔符

CREATE TABlE tableex3(id STRING, name STRING)

ROW FORMAT SERDE'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'

WITH SERDEPROPERTIES (

"input.regex" = "^(.*)\\^\\|~(.*)$"

)

STORED AS TEXTFILE;

2)? 準備數據

1^|~wee

2^|~do

we^|~xml

%^|~we

3)? 轉載數據

load data local inpath '/var/lib/hadoop-hdfs/tee.txt'into table tableex3;

?

4)? 驗證:

select * from tableex3;

+--------------+----------------+--+

| tableex3.id?| tableex3.name? |

+--------------+----------------+--+

| 1???????????| wee??????????? |

| 2???????????| do??????????? ?|

| we??????????| xml??????????? |

| %???????????| we???????????? |

| NULL????????| NULL?????????? |

+--------------+----------------+--+

方法二)通過org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe格式的serde。

#指定以^|~作為分隔符

CREATE TABLE multi_delim (col1 STRING, col2 STRING,Col3STRING) ROW FORMAT SERDE'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES("field.delim"="^|~");

cat /var/lib/hadoop-hdfs/tee3.txt

1^|~wee^|~hi

2^|~do^|~where

we^|~xml^|~rice

%^|~we^|~^|

load data local inpath '/var/lib/hadoop-hdfs/tee.txt'into table tableex3;

?

select * from multi_delim;

+-------------------+-------------------+-------------------+--+

| multi_delim.col1? | multi_delim.col2? | multi_delim.col3? |

+-------------------+-------------------+-------------------+--+

| 1???????????????? | wee?????????????? | hi??????????????? |

| 2???????????????? | do??????????????? | where???????????? |

| we??????????????? | xml?????????????? | rice????????????? |

| %???????????????? | we?????????? ?????| ^|??????????????? |

|?????????????????? | NULL????????????? | NULL????????????? |

+-------------------+-------------------+-------------------+--+

3.2 使用問題

暫無.

5 總結

目前impala尚不支持多字段分割

Not Support for multiple-character string as the fielddelimite

https://issues.cloudera.org/browse/IMPALA-2428

總結

以上是生活随笔為你收集整理的Hive多字符列分割的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。