未被external修飾的是內(nèi)部表(managed table)，被external修飾的為外部表(external table)；

區(qū)別：

內(nèi)部表數(shù)據(jù)由Hive自身管理，外部表數(shù)據(jù)由HDFS管理；

內(nèi)部表數(shù)據(jù)存儲的位置是hive.metastore.warehouse.dir(默認(rèn)：/user/hive/warehouse)，外部表數(shù)據(jù)的存儲位置由自己制定；

刪除內(nèi)部表會直接刪除元數(shù)據(jù)(metadata)及存儲數(shù)據(jù)；刪除外部表僅僅會刪除元數(shù)據(jù)，HDFS上的文件并不會被刪除；

對內(nèi)部表的修改會將修改直接同步給元數(shù)據(jù)，而對外部表的表結(jié)構(gòu)和分區(qū)進(jìn)行修改，則需要修復(fù)(MSCK REPAIR TABLE table_name;)

如下，進(jìn)行試驗進(jìn)行理解

試驗理解

創(chuàng)建內(nèi)部表t1

create table t1(

id????? int

,name??? string

,hobby?? array

,add???? map

)

row format delimited

fields terminated by ','

collection items terminated by '-'

map keys terminated by ':'

;

2. 查看表的描述：desc t1;

裝載數(shù)據(jù)(t1)

注：一般很少用insert (不是insert overwrite)語句，因為就算就算插入一條數(shù)據(jù)，也會調(diào)用MapReduce，這里我們選擇Load Data的方式。

LOAD DATA [LOCAL] INPATH 'filepath' [OVERWRITE] INTO TABLE tablename [PARTITION (partcol1=val1, partcol2=val2 ...)]

然后上載

load data local inpath '/home/hadoop/Desktop/data' overwrite into table t1;

別忘記寫文件名/data,筆者第一次忘記寫，把整個Desktop上傳了，一查全是null和亂碼。。。。

查看表內(nèi)容：

select * from t1;

創(chuàng)建一個外部表t2

create external table t2(

id????? int

,name??? string

,hobby?? array

,add???? map

)

row format delimited

fields terminated by ','

collection items terminated by '-'

map keys terminated by ':'

location '/user/t2'

;

裝載數(shù)據(jù)(t2)

load data local inpath '/home/hadoop/Desktop/data' overwrite into table t2;

查看文件位置

我們在NameNode:50070/explorer.html#/user/目錄下，可以看到t2文件

t1在哪呢？在我們之前配置的默認(rèn)路徑里

同樣我們可以通過命令行獲得兩者的位置信息：

desc formatted table_name;

這里寫圖片描述

這里寫圖片描述

注：圖中managed table就是內(nèi)部表，而external table就是外部表。

分別刪除內(nèi)部表和外部表

下面分別刪除內(nèi)部表和外部表，查看區(qū)別

觀察HDFS上的文件

發(fā)現(xiàn)t1已經(jīng)不存在了

這里寫圖片描述

但是t2仍然存在

這里寫圖片描述

因而外部表僅僅刪除元數(shù)據(jù)

重新創(chuàng)建外部表t2

create external table t2(

id????? int

,name??? string

,hobby?? array

,add???? map

)

row format delimited

fields terminated by ','

collection items terminated by '-'

map keys terminated by ':'

location '/user/t2'

;

這里寫圖片描述

不往里面插入數(shù)據(jù)，我們select * 看看結(jié)果

這里寫圖片描述

可見數(shù)據(jù)仍然在！！！

官網(wǎng)介紹

以下是官網(wǎng)中關(guān)于external表的介紹：

A table created without the EXTERNAL clause is called a managed table because Hive manages its data.

Managed and External Tables

By default Hive creates managed tables, where files, metadata and statistics are managed by internal Hive processes. A managed table is stored under the hive.metastore.warehouse.dir path property, by default in a folder path similar to /apps/hive/warehouse/databasename.db/tablename/. The default location can be overridden by the location property during table creation. If a managed table or partition is dropped, the data and metadata associated with that table or partition are deleted. If the PURGE option is not specified, the data is moved to a trash folder for a defined duration.

Use managed tables when Hive should manage the lifecycle of the table, or when generating temporary tables.

An external table describes the metadata / schema on external files. External table files can be accessed and managed by processes outside of Hive. External tables can access data stored in sources such as Azure Storage Volumes (ASV) or remote HDFS locations. If the structure or partitioning of an external table is changed, an MSCK REPAIR TABLE table_name statement can be used to refresh metadata information.

Use external tables when files are already present or in remote locations, and the files should remain even if the table is dropped.

Managed or external tables can be identified using the DESCRIBE FORMATTED table_name command, which will display either MANAGED_TABLE or EXTERNAL_TABLE depending on table type.

Statistics can be managed on internal and external tables and partitions for query optimization.

Hive官網(wǎng)介紹：

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DescribeTable/View/Column

總結(jié)

以上是生活随笔為你收集整理的hive外部表改为内部表_hive内部表外部表介绍的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。