mysql 查询语句去重 通过sql去重
url 在表中具有唯一性；
由于定时任务的多进程/代码的多线程/定时任务的调度/脚本的执行时间，导致，破坏了唯一性；
SELECT COUNT(1) FROM test_error_tmp
UNION ALL
SELECT COUNT(DISTINCT url) FROM test_error_tmp;
COUNT(1)
40972
40910
差62
SELECT * FROM (
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1;
;
c url
2 b2b.88152.com/show-2677485.html
2 b2b.88152.com/show-2989753.html
2 b2b.88152.com/show-2994375.html
2 b2b.88152.com/show-3071983.html
2 haikou.kuyiso.com/huizhan/29251188x.htm
2 info.123cha.com/show_1007121.html
2 info.b2b168.com/s168-48159518.html
2 info.b2b168.com/s168-58451934.html
2 info.b2b168.com/s168-60677934.html
2 lanzhou.kuyiso.com/canyin/1328.htm
2 site.leshou.com/s/39010445.html
2 site.leshou.com/s/40131383.html
2 site.leshou.com/s/45062793.html
2 site.leshou.com/s/45714581.html
2 site.leshou.com/s/46021681.html
2 site.leshou.com/s/49557679.html
2 site.leshou.com/s/50777239.html
2 site.leshou.com/s/52407278.html
2 wap.ailaba.org/sell/182965618167809.html
2 www.01ny.cn/xinxi/25324778.html
2 www.01ny.cn/xinxi/37941846.html
2 www.05188.com/fenlei/22739365.html
2 www.51sole.com/b2b/pd_26691448.htm
2 www.587766.com/gongqiu/show_2096363.html
2 www.afinance.cn/fenlei/25435008.html
2 www.afinance.cn/fenlei/37074884.html
2 www.afinance.cn/fenlei/38574413.html
2 www.ailaba.org/sell/179521059297846.html
2 www.ailaba.org/sell/183760178499696.html
2 www.bzcm.net/xinxi/29968353.html
2 www.cnhan.com/fenlei/34281043.html
2 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
2 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2 www.hizh.cn/xinxi/10229031.html
2 www.hizh.cn/xinxi/3524483.html
2 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
2 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
2 www.laiwunews.cn/xinxi/13358851.html
2 www.laiwunews.cn/xinxi/18715417.html
2 www.laiwunews.cn/xinxi/18774478.html
2 www.laiwunews.cn/xinxi/18917168.html
2 www.laiwunews.cn/xinxi/19054895.html
2 www.laiwunews.cn/xinxi/20974600.html
2 www.laiwunews.cn/xinxi/20975830.html
2 www.laiwunews.cn/xinxi/21115975.html
2 www.laiwunews.cn/xinxi/21361687.html
2 www.laiwunews.cn/xinxi/21416448.html
2 www.laiwunews.cn/xinxi/21512940.html
2 www.laiwunews.cn/xinxi/25450210.html
2 www.laiwunews.cn/xinxi/27248072.html
2 www.laiwunews.cn/xinxi/27340981.html
2 www.qhnews.com/bianmin/30874042.html
2 www.qhnews.com/bianmin/32388347.html
2 www.qhnews.com/bianmin/32550280.html
2 www.qixiangwang.cn/fenlei/20773619.html
2 www.qiyegu.com/sell/201412/13/deatil_10657611.html
2 www.sooshong.com/chxchx/offerdetail-150433811437054.html
2 www.waaku.com/infoview11788636.html
2 www.yuqinge.com/xinxi/4683460.html
2 www.zznews.cn/xinxi/21212448.html
2 yt.qu114.com/zhuanli/37012081.html
恰好共计62条
现在通过sql删除62条，注意不是62*2条
删除每个url出现晚的数据行
先选出主键id，再执行删除
SELECT id,url FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
;
id url
2204 b2b.88152.com/show-2677485.html
8647 b2b.88152.com/show-2677485.html
2293 b2b.88152.com/show-2989753.html
8426 b2b.88152.com/show-2989753.html
2318 b2b.88152.com/show-2994375.html
8654 b2b.88152.com/show-2994375.html
2220 b2b.88152.com/show-3071983.html
8662 b2b.88152.com/show-3071983.html
1264 haikou.kuyiso.com/huizhan/29251188x.htm
8477 haikou.kuyiso.com/huizhan/29251188x.htm
1935 info.123cha.com/show_1007121.html
8603 info.123cha.com/show_1007121.html
1887 info.b2b168.com/s168-48159518.html
9577 info.b2b168.com/s168-48159518.html
3000 info.b2b168.com/s168-58451934.html
8474 info.b2b168.com/s168-58451934.html
9428 info.b2b168.com/s168-60677934.html
9431 info.b2b168.com/s168-60677934.html
2210 lanzhou.kuyiso.com/canyin/1328.htm
8674 lanzhou.kuyiso.com/canyin/1328.htm
1909 site.leshou.com/s/39010445.html
9076 site.leshou.com/s/39010445.html
1867 site.leshou.com/s/40131383.html
9089 site.leshou.com/s/40131383.html
1885 site.leshou.com/s/45062793.html
9086 site.leshou.com/s/45062793.html
1859 site.leshou.com/s/45714581.html
9090 site.leshou.com/s/45714581.html
1889 site.leshou.com/s/46021681.html
9084 site.leshou.com/s/46021681.html
1900 site.leshou.com/s/49557679.html
9081 site.leshou.com/s/49557679.html
1873 site.leshou.com/s/50777239.html
9088 site.leshou.com/s/50777239.html
1904 site.leshou.com/s/52407278.html
9079 site.leshou.com/s/52407278.html
2992 wap.ailaba.org/sell/182965618167809.html
8475 wap.ailaba.org/sell/182965618167809.html
1610 www.01ny.cn/xinxi/25324778.html
9539 www.01ny.cn/xinxi/25324778.html
9536 www.01ny.cn/xinxi/37941846.html
9543 www.01ny.cn/xinxi/37941846.html
2282 www.05188.com/fenlei/22739365.html
9288 www.05188.com/fenlei/22739365.html
1913 www.51sole.com/b2b/pd_26691448.htm
8449 www.51sole.com/b2b/pd_26691448.htm
1230 www.587766.com/gongqiu/show_2096363.html
8480 www.587766.com/gongqiu/show_2096363.html
2275 www.afinance.cn/fenlei/25435008.html
9304 www.afinance.cn/fenlei/25435008.html
10328 www.afinance.cn/fenlei/37074884.html
10329 www.afinance.cn/fenlei/37074884.html
1917 www.afinance.cn/fenlei/38574413.html
8440 www.afinance.cn/fenlei/38574413.html
1924 www.ailaba.org/sell/179521059297846.html
8421 www.ailaba.org/sell/179521059297846.html
7276 www.ailaba.org/sell/183760178499696.html
9114 www.ailaba.org/sell/183760178499696.html
1222 www.bzcm.net/xinxi/29968353.html
8481 www.bzcm.net/xinxi/29968353.html
2954 www.cnhan.com/fenlei/34281043.html
8476 www.cnhan.com/fenlei/34281043.html
1276 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
8428 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2262 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9414 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9450 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
9452 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2271 www.hizh.cn/xinxi/10229031.html
9311 www.hizh.cn/xinxi/10229031.html
2043 www.hizh.cn/xinxi/3524483.html
9360 www.hizh.cn/xinxi/3524483.html
9566 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9583 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9515 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
9522 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
1293 www.laiwunews.cn/xinxi/13358851.html
8416 www.laiwunews.cn/xinxi/13358851.html
2213 www.laiwunews.cn/xinxi/18715417.html
8670 www.laiwunews.cn/xinxi/18715417.html
3181 www.laiwunews.cn/xinxi/18774478.html
8299 www.laiwunews.cn/xinxi/18774478.html
2259 www.laiwunews.cn/xinxi/18917168.html
9417 www.laiwunews.cn/xinxi/18917168.html
2280 www.laiwunews.cn/xinxi/19054895.html
9293 www.laiwunews.cn/xinxi/19054895.html
1282 www.laiwunews.cn/xinxi/20974600.html
8427 www.laiwunews.cn/xinxi/20974600.html
2217 www.laiwunews.cn/xinxi/20975830.html
8664 www.laiwunews.cn/xinxi/20975830.html
2256 www.laiwunews.cn/xinxi/21115975.html
9422 www.laiwunews.cn/xinxi/21115975.html
2211 www.laiwunews.cn/xinxi/21361687.html
8672 www.laiwunews.cn/xinxi/21361687.html
2214 www.laiwunews.cn/xinxi/21416448.html
8669 www.laiwunews.cn/xinxi/21416448.html
2272 www.laiwunews.cn/xinxi/21512940.html
9403 www.laiwunews.cn/xinxi/21512940.html
2267 www.laiwunews.cn/xinxi/25450210.html
9407 www.laiwunews.cn/xinxi/25450210.html
2290 www.laiwunews.cn/xinxi/27248072.html
9387 www.laiwunews.cn/xinxi/27248072.html
2288 www.laiwunews.cn/xinxi/27340981.html
9390 www.laiwunews.cn/xinxi/27340981.html
3825 www.qhnews.com/bianmin/30874042.html
8303 www.qhnews.com/bianmin/30874042.html
9559 www.qhnews.com/bianmin/32388347.html
9572 www.qhnews.com/bianmin/32388347.html
2541 www.qhnews.com/bianmin/32550280.html
8442 www.qhnews.com/bianmin/32550280.html
2564 www.qixiangwang.cn/fenlei/20773619.html
8288 www.qixiangwang.cn/fenlei/20773619.html
2946 www.qiyegu.com/sell/201412/13/deatil_10657611.html
8478 www.qiyegu.com/sell/201412/13/deatil_10657611.html
1992 www.sooshong.com/chxchx/offerdetail-150433811437054.html
8614 www.sooshong.com/chxchx/offerdetail-150433811437054.html
1299 www.waaku.com/infoview11788636.html
8414 www.waaku.com/infoview11788636.html
2297 www.yuqinge.com/xinxi/4683460.html
9257 www.yuqinge.com/xinxi/4683460.html
2215 www.zznews.cn/xinxi/21212448.html
8666 www.zznews.cn/xinxi/21212448.html
9425 yt.qu114.com/zhuanli/37012081.html
9426 yt.qu114.com/zhuanli/37012081.html
2204 b2b.88152.com/show-2677485.html
8647 b2b.88152.com/show-2677485.html
2293 b2b.88152.com/show-2989753.html
8426 b2b.88152.com/show-2989753.html
2318 b2b.88152.com/show-2994375.html
8654 b2b.88152.com/show-2994375.html
2220 b2b.88152.com/show-3071983.html
8662 b2b.88152.com/show-3071983.html
1264 haikou.kuyiso.com/huizhan/29251188x.htm
8477 haikou.kuyiso.com/huizhan/29251188x.htm
1935 info.123cha.com/show_1007121.html
8603 info.123cha.com/show_1007121.html
1887 info.b2b168.com/s168-48159518.html
9577 info.b2b168.com/s168-48159518.html
3000 info.b2b168.com/s168-58451934.html
8474 info.b2b168.com/s168-58451934.html
9428 info.b2b168.com/s168-60677934.html
9431 info.b2b168.com/s168-60677934.html
2210 lanzhou.kuyiso.com/canyin/1328.htm
8674 lanzhou.kuyiso.com/canyin/1328.htm
1909 site.leshou.com/s/39010445.html
9076 site.leshou.com/s/39010445.html
1867 site.leshou.com/s/40131383.html
9089 site.leshou.com/s/40131383.html
1885 site.leshou.com/s/45062793.html
9086 site.leshou.com/s/45062793.html
1859 site.leshou.com/s/45714581.html
9090 site.leshou.com/s/45714581.html
1889 site.leshou.com/s/46021681.html
9084 site.leshou.com/s/46021681.html
1900 site.leshou.com/s/49557679.html
9081 site.leshou.com/s/49557679.html
1873 site.leshou.com/s/50777239.html
9088 site.leshou.com/s/50777239.html
1904 site.leshou.com/s/52407278.html
9079 site.leshou.com/s/52407278.html
2992 wap.ailaba.org/sell/182965618167809.html
8475 wap.ailaba.org/sell/182965618167809.html
1610 www.01ny.cn/xinxi/25324778.html
9539 www.01ny.cn/xinxi/25324778.html
9536 www.01ny.cn/xinxi/37941846.html
9543 www.01ny.cn/xinxi/37941846.html
2282 www.05188.com/fenlei/22739365.html
9288 www.05188.com/fenlei/22739365.html
1913 www.51sole.com/b2b/pd_26691448.htm
8449 www.51sole.com/b2b/pd_26691448.htm
1230 www.587766.com/gongqiu/show_2096363.html
8480 www.587766.com/gongqiu/show_2096363.html
2275 www.afinance.cn/fenlei/25435008.html
9304 www.afinance.cn/fenlei/25435008.html
10328 www.afinance.cn/fenlei/37074884.html
10329 www.afinance.cn/fenlei/37074884.html
1917 www.afinance.cn/fenlei/38574413.html
8440 www.afinance.cn/fenlei/38574413.html
1924 www.ailaba.org/sell/179521059297846.html
8421 www.ailaba.org/sell/179521059297846.html
7276 www.ailaba.org/sell/183760178499696.html
9114 www.ailaba.org/sell/183760178499696.html
1222 www.bzcm.net/xinxi/29968353.html
8481 www.bzcm.net/xinxi/29968353.html
2954 www.cnhan.com/fenlei/34281043.html
8476 www.cnhan.com/fenlei/34281043.html
1276 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
8428 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2262 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9414 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9450 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
9452 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2271 www.hizh.cn/xinxi/10229031.html
9311 www.hizh.cn/xinxi/10229031.html
2043 www.hizh.cn/xinxi/3524483.html
9360 www.hizh.cn/xinxi/3524483.html
9566 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9583 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9515 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
9522 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
1293 www.laiwunews.cn/xinxi/13358851.html
8416 www.laiwunews.cn/xinxi/13358851.html
2213 www.laiwunews.cn/xinxi/18715417.html
8670 www.laiwunews.cn/xinxi/18715417.html
3181 www.laiwunews.cn/xinxi/18774478.html
8299 www.laiwunews.cn/xinxi/18774478.html
2259 www.laiwunews.cn/xinxi/18917168.html
9417 www.laiwunews.cn/xinxi/18917168.html
2280 www.laiwunews.cn/xinxi/19054895.html
9293 www.laiwunews.cn/xinxi/19054895.html
1282 www.laiwunews.cn/xinxi/20974600.html
8427 www.laiwunews.cn/xinxi/20974600.html
2217 www.laiwunews.cn/xinxi/20975830.html
8664 www.laiwunews.cn/xinxi/20975830.html
2256 www.laiwunews.cn/xinxi/21115975.html
9422 www.laiwunews.cn/xinxi/21115975.html
2211 www.laiwunews.cn/xinxi/21361687.html
8672 www.laiwunews.cn/xinxi/21361687.html
2214 www.laiwunews.cn/xinxi/21416448.html
8669 www.laiwunews.cn/xinxi/21416448.html
2272 www.laiwunews.cn/xinxi/21512940.html
9403 www.laiwunews.cn/xinxi/21512940.html
2267 www.laiwunews.cn/xinxi/25450210.html
9407 www.laiwunews.cn/xinxi/25450210.html
2290 www.laiwunews.cn/xinxi/27248072.html
9387 www.laiwunews.cn/xinxi/27248072.html
2288 www.laiwunews.cn/xinxi/27340981.html
9390 www.laiwunews.cn/xinxi/27340981.html
3825 www.qhnews.com/bianmin/30874042.html
8303 www.qhnews.com/bianmin/30874042.html
9559 www.qhnews.com/bianmin/32388347.html
9572 www.qhnews.com/bianmin/32388347.html
2541 www.qhnews.com/bianmin/32550280.html
8442 www.qhnews.com/bianmin/32550280.html
2564 www.qixiangwang.cn/fenlei/20773619.html
8288 www.qixiangwang.cn/fenlei/20773619.html
2946 www.qiyegu.com/sell/201412/13/deatil_10657611.html
8478 www.qiyegu.com/sell/201412/13/deatil_10657611.html
1992 www.sooshong.com/chxchx/offerdetail-150433811437054.html
8614 www.sooshong.com/chxchx/offerdetail-150433811437054.html
1299 www.waaku.com/infoview11788636.html
8414 www.waaku.com/infoview11788636.html
2297 www.yuqinge.com/xinxi/4683460.html
9257 www.yuqinge.com/xinxi/4683460.html
2215 www.zznews.cn/xinxi/21212448.html
8666 www.zznews.cn/xinxi/21212448.html
9425 yt.qu114.com/zhuanli/37012081.html
9426 yt.qu114.com/zhuanli/37012081.html
SELECT MIN(id),url FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
;
MIN(id) url
2204 b2b.88152.com/show-2677485.html
2293 b2b.88152.com/show-2989753.html
2318 b2b.88152.com/show-2994375.html
2220 b2b.88152.com/show-3071983.html
1264 haikou.kuyiso.com/huizhan/29251188x.htm
1935 info.123cha.com/show_1007121.html
1887 info.b2b168.com/s168-48159518.html
3000 info.b2b168.com/s168-58451934.html
9428 info.b2b168.com/s168-60677934.html
2210 lanzhou.kuyiso.com/canyin/1328.htm
1909 site.leshou.com/s/39010445.html
1867 site.leshou.com/s/40131383.html
1885 site.leshou.com/s/45062793.html
1859 site.leshou.com/s/45714581.html
1889 site.leshou.com/s/46021681.html
1900 site.leshou.com/s/49557679.html
1873 site.leshou.com/s/50777239.html
1904 site.leshou.com/s/52407278.html
2992 wap.ailaba.org/sell/182965618167809.html
1610 www.01ny.cn/xinxi/25324778.html
9536 www.01ny.cn/xinxi/37941846.html
2282 www.05188.com/fenlei/22739365.html
1913 www.51sole.com/b2b/pd_26691448.htm
1230 www.587766.com/gongqiu/show_2096363.html
2275 www.afinance.cn/fenlei/25435008.html
10328 www.afinance.cn/fenlei/37074884.html
1917 www.afinance.cn/fenlei/38574413.html
1924 www.ailaba.org/sell/179521059297846.html
7276 www.ailaba.org/sell/183760178499696.html
1222 www.bzcm.net/xinxi/29968353.html
2954 www.cnhan.com/fenlei/34281043.html
1276 www.cntrades.com/b2b/hnszbjh/sell/itemid-85394203.html
2262 www.gtobal.com/?m=trade&c=detail&a=preview&id=4930157402&type=2
9450 www.gtobal.com/?m=trade&c=detail&a=preview&id=5060488002&type=2
2271 www.hizh.cn/xinxi/10229031.html
2043 www.hizh.cn/xinxi/3524483.html
9566 www.kuyiso.com/html/101/ershoushebei/18348452x.htm
9515 www.kuyiso.com/html/21/zhiyepx/29045579x.htm
1293 www.laiwunews.cn/xinxi/13358851.html
2213 www.laiwunews.cn/xinxi/18715417.html
3181 www.laiwunews.cn/xinxi/18774478.html
2259 www.laiwunews.cn/xinxi/18917168.html
2280 www.laiwunews.cn/xinxi/19054895.html
1282 www.laiwunews.cn/xinxi/20974600.html
2217 www.laiwunews.cn/xinxi/20975830.html
2256 www.laiwunews.cn/xinxi/21115975.html
2211 www.laiwunews.cn/xinxi/21361687.html
2214 www.laiwunews.cn/xinxi/21416448.html
2272 www.laiwunews.cn/xinxi/21512940.html
2267 www.laiwunews.cn/xinxi/25450210.html
2290 www.laiwunews.cn/xinxi/27248072.html
2288 www.laiwunews.cn/xinxi/27340981.html
3825 www.qhnews.com/bianmin/30874042.html
9559 www.qhnews.com/bianmin/32388347.html
2541 www.qhnews.com/bianmin/32550280.html
2564 www.qixiangwang.cn/fenlei/20773619.html
2946 www.qiyegu.com/sell/201412/13/deatil_10657611.html
1992 www.sooshong.com/chxchx/offerdetail-150433811437054.html
1299 www.waaku.com/infoview11788636.html
2297 www.yuqinge.com/xinxi/4683460.html
2215 www.zznews.cn/xinxi/21212448.html
9425 yt.qu114.com/zhuanli/37012081.html
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
;
id
8647
8426
8654
8662
8477
8603
9577
8474
9431
8674
9076
9089
9086
9090
9084
9081
9088
9079
8475
9539
9543
9288
8449
8480
9304
10329
8440
8421
9114
8481
8476
8428
9414
9452
9311
9360
9583
9522
8416
8670
8299
9417
9293
8427
8664
9422
8672
8669
9403
9407
9387
9390
8303
9572
8442
8288
8478
8614
8414
9257
8666
9426
[SQL]DELETE FROM test_error_tmp WHERE id IN (
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
);
[Err] 1093 - You can't specify target table 'test_error_tmp' for update in FROM clause
SELECT * FROM
(
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
) AS tab
id
8647
8426
8654
8662
8477
8603
9577
8474
9431
8674
9076
9089
9086
9090
9084
9081
9088
9079
8475
9539
9543
9288
8449
8480
9304
10329
8440
8421
9114
8481
8476
8428
9414
9452
9311
9360
9583
9522
8416
8670
8299
9417
9293
8427
8664
9422
8672
8669
9403
9407
9387
9390
8303
9572
8442
8288
8478
8614
8414
9257
8666
9426
[SQL]DELETE FROM test_error_tmp WHERE id IN
(
SELECT * FROM
(
SELECT id FROM test_error_tmp
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error_tmp WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error_tmp GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
) AS tab
);
受影响的行: 62
时间: 0.403s
校验
SELECT COUNT(1) FROM test_error_tmp
UNION ALL
SELECT COUNT(DISTINCT url) FROM test_error_tmp;
COUNT(1)
40946
40946
1:1
完成。
该表在读写中，进行的删除。
SELECT COUNT(1) FROM test_error
UNION ALL
SELECT COUNT(DISTINCT url) FROM test_error;
DELETE FROM test_error WHERE id IN
(
SELECT * FROM
(
SELECT id FROM test_error
WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error GROUP BY url
) AS tab
WHERE c>1
)
AND id NOT IN (
SELECT MIN(id) FROM test_error WHERE url IN
(
SELECT url FROM
(
SELECT COUNT(1) AS c,url FROM test_error GROUP BY url
) AS tab
WHERE c>1
) GROUP BY url
)
) AS tab
) AND payoff_status=0;
mysql 查询语句去重 通过sql去重的更多相关文章
- MySQL查询语句执行过程及性能优化（JOIN/ORDER BY)-图
http://blog.csdn.net/iefreer/article/details/12622097 MySQL查询语句执行过程及性能优化-查询过程及优化方法(JOIN/ORDER BY) 标签 ...
- MYSQL查询语句大全集锦
MYSQL查询语句大全集锦 1:使用SHOW语句找出在服务器上当前存在什么数据库: mysql> SHOW DATABASES; 2:2.创建一个数据库MYSQLDATA mysql> C ...
- MySQL查询语句执行过程及性能优化-查询过程及优化方法（JOIN/ORDER BY）
在上一篇文章MySQL查询语句执行过程及性能优化-基本概念和EXPLAIN语句简介中介绍了EXPLAIN语句,并举了一个慢查询例子:
- [转]MySQL查询语句执行过程详解
Mysql查询语句执行原理 数据库查询语句如何执行?语法分析:首先进行语法分析,对使用sql表示的查询进行语法分析,生成查询语法分析树.语义检查:检查sql中所涉及的对象以及是否在数据库中存在,用户是 ...
- php面试专题---MYSQL查询语句优化
php面试专题---MYSQL查询语句优化 一.总结 一句话总结: mysql的性能优化包罗甚广: 索引优化,查询优化,查询缓存,服务器设置优化,操作系统和硬件优化,应用层面优化(web服务器,缓存) ...
- Mysql查询语句执行过程
Mysql查询语句执行过程 Mysql分为server层和存储引擎两部分,或许可以再加一层连接层 连接层(器) Mysql使用的是典型的C/S架构.连接器通过典型的TCP握手完成连接. 需要注 ...
- 在magento里把查询语句转换称sql语句
在magento里把查询语句转换称sql语句 $order->getSelectsql(true); order为对象.
- mysql查询语句，通过limit来限制查询的行数。
mysql查询语句,通过limit来限制查询的行数. 例如: select name from usertb where age > 20 limit 0, 1; //限制从第一条开始,显示1条 ...
- MySQL查询语句执行过程及性能优化-基本概念和EXPLAIN语句简介
网站或服务的性能关键点很大程度在于数据库的设计(假设你选择了合适的语言开发框架)以及如何查询数据上. 我们知道MySQL的性能优化方法,一般有建立索引.规避复杂联合查询.设置冗余字段.建立中间表.查询 ...
- mysql查询语句集
1. mysql 查询出某字段的值不为空的语句 1.不为空 select * from table where id <> ""; select * from tabl ...
随机推荐
- POJ 4118 开餐馆
Description 北大信息学院的同学小明毕业之后打算创业开餐馆.现在共有n 个地点可供选择.小明打算从中选择合适的位置开设一些餐馆.这 n 个地点排列在同一条直线上.我们用一个整数序列m1, m ...
- Python编译错误总结
1.TypeError: object() takes no parameters 如果你出现了这个报错,请检查你的__init__函数名或者其定义有没有写错.一般是把__init__写成了__int ...
- jmeter-如何进行参数化-循环读取参数
在进行测试的时候,测试数据是一项重要的准备工作,每次迭代的数据当不一样的时候,需要进行参数化,从参数化的文件中来读取测试数据. 本经验主要介绍的是用Csv Data配置元件来进行参数化. 方法/步骤 ...
- HDU 5076 Memory
Memory Time Limit: 4000ms Memory Limit: 262144KB This problem will be judged on HDU. Original ID: 50 ...
- Leetcode 297.二叉树的序列化和反序列化
二叉树地序列化和反序列化 序列化是将一个数据结构或者对象转换为连续的比特位的操作,进而可以将转换后的数据存储在一个文件或者内存中,同时也可以通过网络传输到另一个计算机环境,采取相反方式重构得到原数据. ...
- Python 和 Flask实现RESTful services
使用Flask建立web services超级简单. 当然,也有很多Flask extensions可以帮助建立RESTful services,但是这个例实在太简单了,不需要使用任何扩展. 这个we ...
- 69. JPA实体Bean的生命周期【从零开始学Spring Boot】
[从零开始学习Spirng Boot-常见异常汇总] 在使用JPA的时候,如果对bean的生命周期不了解的话,难免会碰到一些问题,比如:InvalidDataAccessApiUsageExcepti ...
- Hadoop JVM调整解决 MapReduce 作业超时问题
摘要:由于业务需要,在mapreduce汇总时需要关联两个基础表,一个60M左右,不影响mr运行,另一个表,大小约为380M,行数为1700万行左右,在默认配置下,一旦加载这个数据就会在reduce阶 ...
- BZOJ1573: [Usaco2009 Open]牛绣花cowemb
求半径d<=50000的圆(不含边界)内n<=50000条直线有多少交点,给直线的解析式. 一开始就想,如果能求出直线交点与原点距离<d的条件,那么从中不重复地筛选即可.然而两个kx ...
- Android广播Broadcast
Android Broadcast简单认识 Broadcast是应用程序间传输信息的一种机制,BroadcastReceiver是对发送出来的广播(Broadcast)进行过滤并接收相应的一类组件. ...