爬虫——python——百度地图经纬度查询——经纬度查看地点地名——利用百度API获取地名经纬度——爬取所有的中国地址


import requests
address = '40.8587960,86.866991'
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
使用python爬虫

import requests
address = '40.8587960,86.866991'
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str(lng) + '|' + str(lat)+ '|' + str(formatted_address) + '|' + str(business) + '|'+ str(city) + '|'+ str(direction) + '|'+ str(distance) + '|'+ str(district) + '|'+ str(province) + '|'+ str(street) + '|'+ str(street_number) + '|'+ str(cityCode)
print(s)
解析数据

http://api.map.baidu.com/lbsapi/getpoint/index.html
#鸡头1
#129.000000,54.000000 #锚点(鸡头)
#115.000000,54.0000000
#115.000000,42.0000000 #北京
#129.000000,42.000000 #沈阳
#['54.0000000', '42.000000', '129.000000', '115.000000'],#鸡头1 #鸡头2
#129.000000,42.000000 #沈阳
#135.200000,42.000000
#129.000000,50.000000
#135.200000,50.000000
#['50.000000','42.000000','135.000000','129.000000'],#鸡头2
#鸡头补充
#115.000000,42.0000000 #北京
#105.0000000,42.000000 #锚点(银川上面)
#105.0000000,45.500000 #乌兰巴托
#115.000000,45.500000
#['45.500000', '42.000000', '115.000000', '105.000000'],#鸡头补充 #鸡屁股上补充
#105.0000000,42.000000 #锚点(银川上面)
#91.500000,45.500000 #乌鲁木齐
#105.0000000,45.500000 #乌兰巴托
#91.500000,42.000000 #锚点(鸡屁股)
#['45.500000', '42.000000', '105.0000000', '91.500000'],#鸡屁股上补充
#鸡屁股上
#91.500000,42.000000 #锚点(鸡屁股)
#79.800000,42.000000 #啊拉木图
#79.800000,49.200000
#91.500000,49.200000
#['49.200000', '42.000000', '91.500000', '79.800000'],#鸡屁股上
#鸡屁股下
#79.800000,42.000000 #啊拉木图
#79.800000,30.0000000 #巴雷利
#73.400000,42.000000
#73.400000,30.000000
#['42.000000', '30.000000', '79.800000', '73.400000'],#鸡屁股下
#鸡下补充
#79.800000,30.0000000 #巴雷利
#97.300000,30.000000 #拉萨昌都(锚点)
#97.300000,26.800000
#79.800000,26.800000
#['42.000000', '30.000000', '97.300000', '79.800000'],#鸡下补充
#鸡下
#97.300000,30.000000 #拉萨昌都(锚点)
#105.000000,30.000000 #(成都重庆贵阳)
#105.000000,21.000000 #河内
#97.300000,21.000000 #曼德勒
#['30.000000', '21.000000', '105.000000', '97.300000'],#鸡下 #鸡胸
#105.0000000,42.000000 #锚点(银川上面)
#105.0000000,21.000000 #河内
#129.000000,42.000000 #沈阳
#129.000000,21.000000 #台北(海)
#['42.000000', '21.000000', '129.000000', '105.000000'],#鸡胸 #鸡身
#105.000000,42.000000 #锚点(银川上面)
#79.800000,42.000000 #啊拉木图
#79.800000,30.000000 #巴雷利
#105.000000,30.000000 #(成都重庆贵阳)
#['42.000000', '30.000000', '105.000000', '79.800000'],#鸡身
#鸡脚 (南海)
#105.000000,21.000000 #河内
#129.000000,21.000000 #台北(海)
#105.0000000,3.000000 #新加坡
#129.000000,3.000000 #海(苏拉威西海)
#['21.000000', '3.000000', '129.000000', '105.000000'],
经纬度分析

import requests
from decimal import Decimal
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address)#传入经度,纬度,获取地址地名
slon -= precision #['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal('42.000000')#经度longitude开始
elon = Decimal('30.000000')#经度结束
slat=Decimal('105.000000')# 纬度latitude开始
elat=Decimal('79.800000')#纬度结束
precision = Decimal('5.0001000')#精度 0.0001000==1000米
while slat>=elat:
slat -= precision
longitude(slon,precision,elon,slat)
根据精度获取多个地名

import requests
from decimal import Decimal
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address)#传入经度,纬度,获取地址地名
slon -= precision LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 (南海)
]
precision = Decimal('10.0001000') # 精度 0.0001000==1000米
for single_list in LogLatList:
slons,elons,slats,elats=single_list
#['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons)#经度longitude开始
elon = Decimal(elons)#经度结束
slat=Decimal(slats)# 纬度latitude开始
elat=Decimal(elats)#纬度结束 while slat>=elat:
slat -= precision
longitude(slon,precision,elon,slat)
批量获取数据

import requests
from decimal import Decimal
import threading , time
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address)#传入经度,纬度,获取地址地名
slon -= precision class Thre(threading.Thread):#继承线程中的类
#Thre(single_list,precision,ctime)
def __init__(self,single_list,precision,ctime):
super(Thre,self).__init__()#重新写父类,解决多继承问题
self.precision=precision#精度
self.single_list=single_list#单列表
self.ctime=ctime#当前时间
def run(self):
print('执行线程开始时间:', self.ctime, '执行的列表:',self.single_list)
slons, elons, slats, elats = self.single_list
# ['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons) # 经度longitude开始
elon = Decimal(elons) # 经度结束
slat = Decimal(slats) # 纬度latitude开始
elat = Decimal(elats) # 纬度结束
while slat >= elat:
slat -= self.precision
longitude(slon,self.precision, elon, slat)
else:
end_time=time.time()-self.ctime
print('执行线程所用时间:',end_time, '执行的列表:',self.single_list) def main():
LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 (南海)
]
precision = Decimal('20.0001000') # 精度 0.0001000==1000米 thre_list=[] #线程列表
for single_list in LogLatList:
ctime=time.time()#当前时间
temp=Thre(single_list,precision,ctime)#实例化类
thre_list.append(temp)#线程列表
for thre in thre_list:#线程列表
thre.start()#执行单个线程 if __name__ == '__main__':
main()
多线程获取数据

import requests
from decimal import Decimal
import threading , time
def obtain(address,file_name):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s)
if cityCode==0:
#print('外国')
pass
else:
#print(s)
fname=str(file_name)+'list'
with open(fname, 'a+', encoding=('utf-8')) as f:
f.write(s+'\n') def longitude(slon,precision,elon,slat,file_name):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的,经度,纬度,
obtain(address,file_name)#传入经度,纬度,获取地址地名
slon -= precision class Thre(threading.Thread):#继承线程中的类
#Thre(single_list,precision,ctime)
#temp = Thre(single_list, precision, ctime, file_name) # 实例化类
def __init__(self,single_list,precision,ctime,file_name):
super(Thre,self).__init__()#重新写父类,解决多继承问题
self.precision=precision#精度
self.single_list=single_list#单列表
self.ctime=ctime#当前时间
self.file_name=file_name#文件名
def run(self):
print('执行线程开始时间:', self.ctime, '执行的列表:',self.single_list)
slons, elons, slats, elats = self.single_list
# ['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons) # 经度longitude开始
elon = Decimal(elons) # 经度结束
slat = Decimal(slats) # 纬度latitude开始
elat = Decimal(elats) # 纬度结束
while slat >= elat:
slat -= self.precision
longitude(slon,self.precision, elon, slat,self.file_name)
else:
end_time=time.time()-self.ctime
print('执行线程所用时间:',end_time, '执行的列表:',self.single_list) def main():
LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 (南海)
]
# ——测试使用——开始————
precision = Decimal('1.0001000') # 精度 0.0001000==1000米
# ——测试使用——结束———— #——实际获取时替换——精度自己控制————建议是 0.0001000 ————开始————
#precision = Decimal('0.0001000') # 精度 0.0001000==1000米
# ——实际获取时替换——精度自己控制————建议是 0.0001000 ————结束———— file_name=0 thre_list=[] #线程列表
for single_list in LogLatList:
ctime=time.time()#当前时间
temp=Thre(single_list,precision,ctime,file_name)#实例化类
file_name+=1
thre_list.append(temp)#线程列表
for thre in thre_list:#线程列表
thre.start()#执行单个线程 if __name__ == '__main__':
main()
写入到文件

爬虫——python——百度地图经纬度查询——经纬度查看地点地名——利用百度API获取地名经纬度——爬取所有的中国地址的更多相关文章
- Android百度地图开发-第一篇:申请、搭建百度地图
一.前言 这是第一篇关于Android使用百度地图的学习记录,主要记录: 1.在百度地图开发者平台上申请API Key. 2.在自己的应用中加入百度地图的Android版SDK. 3.在自己的应用中显 ...
- 百度地图JavaScript API获取用户当前经纬度和详细地理位置,反之通过详细地理位置获取当前经纬度
前言: 前端时间刚好使用了百度地图的js api定位获取用户当前经纬度并获取当前详细位置和通过当前用户详细地理位置换取用户当前经纬度坐标的功能,为了方便下次找起来方便一些自己在这里记录一下,希望也能够 ...
- python爬虫学习(二):定向爬虫例子-->使用BeautifulSoup爬取"软科中国最好大学排名-生源质量排名2018",并把结果写进txt文件
在正式爬取之前,先做一个试验,看一下爬取的数据对象的类型是如何转换为列表的: 写一个html文档: x.html<html><head><title>This is ...
- Python 从底层结构聊 Beautiful Soup 4(内置豆瓣最新电影排行榜爬取案例)
1. 前言 什么是 Beautiful Soup 4 ? Beautiful Soup 4(简称 BS4,后面的 4 表示最新版本)是一个 Python 第三方库,具有解析 HTML 页面的功能,爬虫 ...
- 针对源代码和检查元素不一致的网页爬虫——利用Selenium、PhantomJS、bs4爬取12306的列车途径站信息
整个程序的核心难点在于上次豆瓣爬虫针对的是静态网页,源代码和检查元素内容相同:而在12306的查找搜索过程中,其网页发生变化(出现了查找到的数据),这个过程是动态的,使得我们在审查元素中能一一对应看到 ...
- 简单又强大的pandas爬虫 利用pandas库的read_html()方法爬取网页表格型数据
文章目录 一.简介 二.原理 三.爬取实战 实例1 实例2 一.简介 一般的爬虫套路无非是发送请求.获取响应.解析网页.提取数据.保存数据等步骤.构造请求主要用到requests库,定位提取数据用的比 ...
- Python 爬取图书图片和地址
#-*- coding:utf-8 -*- import xlwt import urllib import re def getHtml(url): page = urllib.urlopen(ur ...
- Python反爬:利用js逆向和woff文件爬取猫眼电影评分信息
首先:看看运行结果效果如何! 1. 实现思路 小编基本实现思路如下: 利用js逆向模拟请求得到电影评分的页面(就是猫眼电影的评分信息并不是我们上述看到的那个页面上,应该它的实现是在一个页面上插入另外一 ...
- 记录开发基于百度地图API实现在地图上绘制轨迹并拾取轨迹对应经纬度的工具说明
前言: 最近一直在做数据可视化方面的工作,其中平面可视化没什么难度,毕竟已经有很多成熟的可供使用的框架,比如百度的echart.js,highcharts.js等.还有就是3D可视化了,整体来说难度也 ...
随机推荐
- R:ggplot2数据可视化——进阶(3)
Part 3: Top 50 ggplot2 Visualizations - The Master List, 结合进阶1.2内容构建图形 有效的图形是: 不扭曲事实 传递正确的信息 简洁优雅 美观 ...
- 天道神诀---防火墙以及selinux(上篇)
Linux防火墙 linux6.x 防火墙会影响通信,默认是拒绝所有. [root@redhat6 sysconfig]# chkconfig iptables --listiptables ...
- 2018-8-10-WPF-使用-Direct2D1-画图-绘制基本图形
title author date CreateTime categories WPF 使用 Direct2D1 画图 绘制基本图形 lindexi 2018-08-10 19:16:53 +0800 ...
- opencv3.1.0 在控制台程序中报错:winnt.h(6464): error C2872: ACCESS_MASK: 不明确的
在winnt.h里面有一个cv的命名空间,同样定义了一个ACCESS_MASK,跟opencv的cv::ACCESS_MASK发生了冲突!!! 该冲突在MFC中没有出现,在控制台程序中才会报错!对于o ...
- 【JZOJ6293】迷宫
description analysis 有没有想起[\(NOIP2018\)]保卫王国? 设\(tr[t][x][y]\)表示线段树上的\(t\)节点代表的区间,从最左边列的\(x\)行到最右边列\ ...
- thinkphp 闭包支持
闭包定义 我们可以使用闭包的方式定义一些特殊需求的路由,而不需要执行控制器的操作方法了,例如: 'URL_ROUTE_RULES'=>array( 'test' => function() ...
- csps模拟86异或,取石子,优化题解
题面:https://www.cnblogs.com/Juve/articles/11736440.html 异或: 考试时只想出了暴力 我们可以对于二进制下每一位w,求出[l,r]中有几个数在这一位 ...
- 0922CSP-S模拟测试赛后总结
连发三篇爆炸实录我的心态竟然还这么好…… 昨天题目的D2.稍难. 这也不是我连续拿倒数第一的理由. T1不会.赛时硬写了一个30分的三次方暴力.还有一个地方写挂了.如果不是数据足够水我就爆零了. 也就 ...
- 组合数学——cf991E
/* 如果有某一位,那么这一位必须存在 枚举所有情况,计算每种情况时0额外另算 */ #include<bits/stdc++.h> using namespace std; #defin ...
- hdu多校第三场 1006 (hdu6608) Fansblog Miller-Rabin素性检测
题意: 给你一个1e9-1e14的质数P,让你找出这个质数的前一个质数Q,然后计算Q!mod P 题解: 1e14的数据范围pass掉一切素数筛法,考虑Miller-Rabin算法. 米勒拉宾算法是一 ...