當前位置：首頁 > 运维知识 > 数据库 >内容正文

数据库

用SQL删除重复记录的N种方法

發布時間：2025/3/21 数据库 48 豆豆

生活随笔收集整理的這篇文章主要介紹了用SQL删除重复记录的N种方法小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

例如：?
? id?????????? name???????? value?
? 1?????????????? a???????????????? pp?
? 2?????????????? a???????????????? pp?
? 3?????????????? b???????????????? iii?
? 4?????????????? b???????????????? pp?
? 5?????????????? b???????????????? pp?
? 6?????????????? c???????????????? pp?
? 7?????????????? c???????????????? pp?
? 8?????????????? c???????????????? iii?
? id是主鍵?
? 要求得到這樣的結果?
? id?????????? name???????? value?
? 1?????????????? a???????????????? pp?
? 3?????????????? b???????????????? iii?
? 4?????????????? b???????????????? pp?
? 6?????????????? c???????????????? pp?
? 8?????????????? c???????????????? iii?

方法1
delete?? YourTable??
? where?? [id]?? not?? in?? (?
? select?? max([id])?? from?? YourTable??
? group?? by?? (name?? +?? value))?
?
方法2
delete?? a?
? from?? 表?? a?? left?? join(?
? select?? id=min(id)?? from?? 表?? group?? by?? name,value?
? )b?? on?? a.id=b.id?
? where?? b.id?? is?? null?
?
?
?
查詢及刪除重復記錄的SQL語句
查詢及刪除重復記錄的SQL語句
1、查找表中多余的重復記錄，重復記錄是根據單個字段（peopleId）來判斷
select * from people
where peopleId in (select?? peopleId from?? people group by?? peopleId having count(peopleId) > 1)
2、刪除表中多余的重復記錄，重復記錄是根據單個字段（peopleId）來判斷，只留有rowid最小的記錄
delete from people
where peopleId in (select?? peopleId from people group by?? peopleId?? having count(peopleId) > 1)
and rowid not in (select min(rowid) from?? people group by peopleId having count(peopleId )>1)
3、查找表中多余的重復記錄（多個字段）
select * from vitae a
where (a.peopleId,a.seq) in?? (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
4、刪除表中多余的重復記錄（多個字段），只留有rowid最小的記錄
delete from vitae a
where (a.peopleId,a.seq) in?? (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)

5、查找表中多余的重復記錄（多個字段），不包含rowid最小的記錄
select * from vitae a
where (a.peopleId,a.seq) in?? (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)
(二)
比方說
在A表中存在一個字段“name”，
而且不同記錄之間的“name”值有可能會相同，
現在就是需要查詢出在該表中的各記錄之間，“name”值存在重復的項；
Select Name,Count(*) From A Group By Name Having Count(*) > 1
如果還查性別也相同大則如下:
Select Name,sex,Count(*) From A Group By Name,sex Having Count(*) > 1

(三)
方法一
declare @max integer,@id integer
declare cur_rows cursor local for select 主字段,count(*) from 表名 group by 主字段 having count(*) >； 1
open cur_rows
fetch cur_rows into @id,@max
while @@fetch_status=0
begin
select @max = @max -1
set rowcount @max
delete from 表名 where 主字段 = @id
fetch cur_rows into @id,@max
end
close cur_rows
set rowcount 0 方法二
＂重復記錄＂有兩個意義上的重復記錄，一是完全重復的記錄，也即所有字段均重復的記錄，二是部分關鍵字段重復的記錄，比如Name字段重復，而其他字段不一定重復或都重復可以忽略。
　　1、對于第一種重復，比較容易解決，使用
select distinct * from tableName
　　就可以得到無重復記錄的結果集。
　　如果該表需要刪除重復的記錄（重復記錄保留1條），可以按以下方法刪除
select distinct * into #Tmp from tableName
drop table tableName
select * into tableName from #Tmp
drop table #Tmp
　　發生這種重復的原因是表設計不周產生的，增加唯一索引列即可解決。
　　2、這類重復問題通常要求保留重復記錄中的第一條記錄，操作方法如下
　　假設有重復的字段為Name,Address，要求得到這兩個字段唯一的結果集
select identity(int,1,1) as autoID, * into #Tmp from tableName
select min(autoID) as autoID into #Tmp2 from #Tmp group by Name,autoID
select * from #Tmp where autoID in(select autoID from #tmp2)
　　最后一個select即得到了Name，Address不重復的結果集（但多了一個autoID字段，實際寫時可以寫在select子句中省去此列）
(四)
查詢重復
select * from tablename where id in (
select id from tablename
group by id
having count(id) > 1
)

學習sql有一段時間了，發現在我建了一個用來測試的表（沒有建索引）中出現了許多的重復記錄。后來總結了一些刪除重復記錄的方法，在Oracle中，可以通過唯一rowid實現刪除重復記錄；還可以建臨時表來實現...這個只提到其中的幾種簡單實用的方法，希望可以和大家分享（以表employee為例）。

SQL> desc employee

?Name????????????????????????????????????? Null???? Type
----------------------------------------- -------- ------------------

emp_id??????????????????????????????????????????????? NUMBER(10)
emp_name?????????????????????????????????????????? VARCHAR2(20)

salary???????????????????????????????????????????????? ?NUMBER(10,2)

可以通過下面的語句查詢重復的記錄：

SQL> select * from employee;

??? EMP_ID EMP_NAME????????? ????????????????????????SALARY

---------- ---------------------------------------- ----------

???????? 1 sunshine????????????????????????????????????? 10000

???????? 2 semon??????????????????? ?????????????????????20000

???????? 2 semon???????????????????????????????????????? 20000

???????? 3 xyz?????????????????????????????????????????? 30000

???????? 2 semon???????????????????????????????????????? 20000

SQL> select distinct * from employee;

??? EMP_ID EMP_NAME???????????????????????????????????? SALARY

---------- ---------------------------------------- ----------

???????? 1 sunshine????????????????????????????????????? 10000

???????? 2 semon???????????????????????????????????????? 20000

??? ?????3 xyz???????????????????????????????????????????? 30000

SQL>? select * from employee group by emp_id,emp_name,salary having count (*)>1

??? EMP_ID EMP_NAME???????????????????????????????????? SALARY

---------- ---------------------------------------- ----------

???????? 1 sunshine????????????????????????????????????? 10000

???????? 2 semon????????????????????????????????????????? 20000

SQL> select * from employee e1

where rowid in (select max(rowid) from employe e2
?where e1.emp_id=e2.emp_id and

? e1.emp_name=e2.emp_name and e1.salary=e2.salary);

??? EMP_ID EMP_NAME???????????????????????????????????? SALARY

---------- ---------------------------------------- ----------

???????? 1 sunshine????????????????????????????????????? 10000

???????? 3 xyz????? ?????????????????????????????????????? 30000

???????? 2 semon???????????????????????????????????????? 20000

2. 刪除的幾種方法：

（1）通過建立臨時表來實現

SQL>create table temp_emp as (select distinct * from employee)?

SQL> truncate table employee; (清空employee表的數據）

SQL> insert into employee select * from temp_emp; ?(再將臨時表里的內容插回來）

( 2）通過唯一rowid實現刪除重復記錄.在Oracle中，每一條記錄都有一個rowid，rowid在整個數據庫中是唯一的，rowid確定了每條記錄是在Oracle中的哪一個數據文件、塊、行上。在重復的記錄中，可能所有列的內容都相同，但rowid不會相同，所以只要確定出重復記錄中那些具有最大或最小rowid的就可以了，其余全部刪除。

SQL>delete from employee e2 where rowid not in (
??????? select max(e1.rowid) from employee e1 where

??????? e1.emp_id=e2.emp_id and e1.emp_name=e2.emp_name and e1.salary=e2.salary);--這里用min(rowid)也可以。

SQL>delete from employee e2 where rowid <(
??????? select max(e1.rowid) from employee e1 where
??????? e1.emp_id=e2.emp_id and e1.emp_name=e2.emp_name and

????????????????? e1.salary=e2.salary);

（3）也是通過rowid，但效率更高。

SQL>delete from employee where rowid not in (
??????? select max(t1.rowid) from employee t1 group by

???????? t1.emp_id,t1.emp_name,t1.salary);--這里用min(rowid)也可以。

??? EMP_ID EMP_NAME???????????????????????????????????? SALARY

---------- ---------------------------------------- ----------

???????? 1 sunshine????????????????????????????????????? 10000

???????? 3 xyz???? ??????????????????????????????????????? 30000