用sql删除数据库重复的数据的方法

 

/***********************************************
两个意义上的重复记录:
1.是完全重复的记录，也即所有字段均重复的记录，
2.是部分关键字段重复的记录,比如username字段重复，
  而其他字段不一定重复或都重复可以忽略,这类重复
  问题通常要求保留重复记录中的第一条记录
************************************************/

/*1.数据完全重复(用到了一个临时表#Tmp)*/
CREATE TABLE admin1(
 [username] [nvarchar](50) COLLATE Chinese_PRC_CI_AS NULL,
 [password] [nvarchar](50) COLLATE Chinese_PRC_CI_AS NULL
) ON [PRIMARY]

INSERT INTO admin1(username,password) VALUES('liyan','111')
INSERT INTO admin1(username,password) VALUES('liyan','111')
INSERT INTO admin1(username,password) VALUES('liyan','222')
INSERT INTO admin1(username,password) VALUES('liyan','222')
INSERT INTO admin1(username,password) VALUES('liyan','333')
select * from admin1

select distinct * into #Tmp from admin1
drop table admin1
select * into admin1 from #Tmp
drop table #Tmp
select * from admin1

/*2.某个字段重复(用到了2个临时表:Tmp1,Tmp2)*/
CREATE TABLE [dbo].[admin](
 [id] [int] IDENTITY(1,1) NOT NULL,
 [username] [nvarchar](50) COLLATE Chinese_PRC_CI_AS NULL,
 [password] [nvarchar](50) COLLATE Chinese_PRC_CI_AS NULL,
 CONSTRAINT [PK_admin] PRIMARY KEY CLUSTERED
(
 [id] ASC
)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]

INSERT INTO admin(username,password) VALUES('adminstrator','111')
INSERT INTO admin(username,password) VALUES('adminstrator','111')
INSERT INTO admin(username,password) VALUES('adminstrator','222')
INSERT INTO admin(username,password) VALUES('adminstrator','222')
INSERT INTO admin(username,password) VALUES('adminstrator','333')
INSERT INTO admin(username,password) VALUES('liyan','111')
INSERT INTO admin(username,password) VALUES('liyan','111')
INSERT INTO admin(username,password) VALUES('liyan','222')
INSERT INTO admin(username,password) VALUES('liyan','222')
INSERT INTO admin(username,password) VALUES('liyan','333')

if exists(select * from tempdb..sysobjects where id=object_id('tempdb..#Tmp1')) drop table #Tmp1
select id as autoid, * into #Tmp1 from admin
if exists(select * from tempdb..sysobjects where id=object_id('tempdb..#Tmp2')) drop table #Tmp2
select min(autoid) as autoid into #Tmp2 from #Tmp1 group by username,password
if exists(select * from dbo.sysobjects where id = object_id(N'admin') and OBJECTPROPERTY(id,N'IsUserTable') = 1) drop table admin
select id,username,password into admin from #Tmp1 where autoid in(select autoid from #tmp2)

=====================================================

下面的是引用网上的一篇文章(http://tb.blog.csdn.net/TrackBack.aspx?PostId=1530926)

关于使用sql删除数据库重复的数据的方法整理：

1.数据完全重复

通过一个临时表过渡一下
insert into table1 select distinct field from table
drop table table
insert into table select * from table1

2.某个字段重复

这个在网上有很多相关的解决方法，较常见的有
delete from table where id not in (select min(id) from table group by name)
delete from table where field in (select field from table group by field having count(*) > 1)

上面的方法在删除小数量级的数据时还有用，当一旦处理的数据是几十万或者更多时就出问题了，一般的机器估计一运行就马上给费了。其实稍有点常识的算一算就知道这样的语句会有多大的运算量了，它的运算量至少是以乘方的形式递增的，想想就恐怖。

我在这里主要是要给出对于大数量级的表的重复数据删除的解决方案，其实也很简单，也是利用了一个过渡表来实现
insert in tabletemp select * from table
delete from table as a where a.id > (select min(b.id) from table1 as b where b.field=a.field)
drop table tabletemp
这样利用了数据库的索引的优势，大大的减少运算量

========================================

SQL如何删除重复的数据行- -

delete from table where id in (
 select max(id) from table group by name having count(*)>1
)--删除重复记录中ID最大的一条(如果有2条以上的重复记录则需多次执行)

如果table数据完全一样，可以先将数据导入到一个临时表内
或

delete from table where id  not in (
 select min(id) from table group by name
)--只保留重复记录的第一条(id最小的一条)

太少了..加点其它的内容

CREATE  PROCEDURE 存储过程名    --执行动态SQL语句
(
   @num int
)
 AS
declare @string nvarchar(100)
set @string='SELECT  TOP  '+ CAST (@num as nvarchar) +' * FROM  表名' 
exec (@string)

=====================================

select ID,NAME from house1 where name='中凯' and roomtype='双人间' and startdate>='2007-5-25' and id in(select min(id) from group by name)

name相同时id小的出现

======================

一般就是distinct , group by , #tempTable,

当然借助index会更快些