Hadoop系列之Hive(數據倉庫)安裝配置

1.在NameNode安裝

cd /root/soft

tar zxvf apache-hive-0.13.1-bin.tar.gz

mv apache-hive-0.13.1-bin /usr/local/hadoop/hive

2. 配置環境變量(每個節點都需要增加)

打開/etc/profile

#添加以下內容：

export HIVE_HOME=/usr/local/hadoop/hive

export PATH=$HIVE_HOME/bin:$PATH

#環境變量生效

source /etc/profile

3.安裝mysql數據庫環境

請參照http://azhuang.blog.51cto.com/9176790/1551549

數據庫安裝成功后,一定要建立號權限及創建hive數據庫。操作如下

grant all privileges on hive.* to hive@'192.168.3.%' identified by '123';

create database hive character set latin1;? #UTF-8編碼hive會報錯,所以需要修改編碼為latin1

4. 配置Hive

cd /usr/local/hadoop/hive/conf/

cp hive-default.xml.template hive-site.xml

#vim hive-site.xml(修改之間配置)

javax.jdo.option.ConnectionURL

jdbc:mysql://192.168.3.10:3306/hive?characterEncoding=latin1

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

javax.jdo.option.ConnectionUserName

hive

javax.jdo.option.ConnectionPassword

123

#以上四項分別是：

數據庫連接，數據庫驅動名，用戶名，密碼。

5.把mySQL的JDBC驅動包復制到Hive的lib目錄下

cp /root/soft/mysql-connector-java-commercial-5.1.30-bin.jar /usr/local/hadoop/hive/lib/

6.復制Hive到所有DataNode節點

scp -r /usr/local/hadoop/hive root@192.168.3.11:/usr/local/hadoop/

scp -r /usr/local/hadoop/hive root@192.168.3.12:/usr/local/hadoop/

7.簡單測試

//查看當前數據表

hive> show tables;

OK

Time taken: 1.069 seconds

//從本地文件系統中導入數據到Hive表

#創建student.txt測試文本,字段之間用tab鍵分割

[root@hdfs-master soft]# cat /root/soft/student.txt

1?? ?aa?? ?10?? ?121221

2?? ?bb?? ?20?? ?0990

3?? ?cc?? ?30?? ?120120

#創建student測試表

hive> create table student(id int, name string, age int, tel string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'? STORED AS TEXTFILE;

OK

Time taken: 0.043 seconds

#再次查看當前數據表及結構

hive> show tables;

OK

student

hive> desc student;

OK

id???????????????? ??? ?int

name?????????????? ??? ?string

age??????????????? ??? ?int

tel??????????????? ??? ?string

Time taken: 0.103 seconds, Fetched: 4 row(s)

#把/root/soft/student.txt本地記錄導入到student表

hive> load data local inpath '/root/soft/student.txt'? into table student;

Copying data from file:/root/soft/student.txt

Copying file: file:/root/soft/student.txt

Loading data to table default.student

Table default.student stats: [numFiles=1, numRows=0, totalSize=43, rawDataSize=0]

OK

Time taken: 0.376 seconds

#查看student表，如果有記錄表示本地插入數據成功.

hive> select * from student;

OK

1?? ?aa?? ?10?? ?121221

2?? ?bb?? ?20?? ?0990

3?? ?cc?? ?30?? ?120120

Time taken: 0.066 seconds, Fetched: 3 row(s)

//HDFS上導入數據到Hive表

#上傳本地文件到hdfs

[root@hdfs-master soft]# hdfs dfs -put /root/soft/student.txt /hive

[root@hdfs-master soft]# hdfs dfs -cat /hive/student.txt

1?? ?aa?? ?10?? ?121221

2?? ?bb?? ?20?? ?0990

3?? ?cc?? ?30?? ?120120

#從hdfs導入到hive數據

hive> load data inpath '/hive/student.txt' into table student;

Loading data to table default.student

Table default.student stats: [numFiles=2, numRows=0, totalSize=86, rawDataSize=0]

OK

Time taken: 1.389 seconds

hive> select * from student;

OK

1?? ?aa?? ?10?? ?121221

2?? ?bb?? ?20?? ?0990

3?? ?cc?? ?30?? ?120120

1?? ?aa?? ?10?? ?121221

2?? ?bb?? ?20?? ?0990

3?? ?cc?? ?30?? ?120120

Time taken: 0.049 seconds, Fetched: 6 row(s)

總結

以上是生活随笔為你收集整理的hadoop安装hive及配置mysql_Hadoop系列之Hive(数据仓库)安装配置的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。