爬虫——python——百度地图经纬度查询——经纬度查看地点地名——利用百度API获取地名经纬度——爬取所有的中国地址
import requests
address = '40.8587960,86.866991'
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
使用python爬虫
import requests
address = '40.8587960,86.866991'
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str(lng) + '|' + str(lat)+ '|' + str(formatted_address) + '|' + str(business) + '|'+ str(city) + '|'+ str(direction) + '|'+ str(distance) + '|'+ str(district) + '|'+ str(province) + '|'+ str(street) + '|'+ str(street_number) + '|'+ str(cityCode)
print(s)
解析数据
http://api.map.baidu.com/lbsapi/getpoint/index.html
#鸡头1
#129.000000,54.000000 #锚点（鸡头）
#115.000000,54.0000000
#115.000000,42.0000000 #北京
#129.000000,42.000000 #沈阳
#['54.0000000', '42.000000', '129.000000', '115.000000'],#鸡头1 #鸡头2
#129.000000,42.000000 #沈阳
#135.200000,42.000000
#129.000000,50.000000
#135.200000,50.000000
#['50.000000','42.000000','135.000000','129.000000'],#鸡头2
#鸡头补充
#115.000000,42.0000000 #北京
#105.0000000,42.000000 #锚点（银川上面）
#105.0000000,45.500000 #乌兰巴托
#115.000000,45.500000
#['45.500000', '42.000000', '115.000000', '105.000000'],#鸡头补充 #鸡屁股上补充
#105.0000000,42.000000 #锚点（银川上面）
#91.500000,45.500000 #乌鲁木齐
#105.0000000,45.500000 #乌兰巴托
#91.500000,42.000000 #锚点（鸡屁股）
#['45.500000', '42.000000', '105.0000000', '91.500000'],#鸡屁股上补充
#鸡屁股上
#91.500000,42.000000 #锚点（鸡屁股）
#79.800000,42.000000 #啊拉木图
#79.800000,49.200000
#91.500000,49.200000
#['49.200000', '42.000000', '91.500000', '79.800000'],#鸡屁股上
#鸡屁股下
#79.800000,42.000000 #啊拉木图
#79.800000,30.0000000 #巴雷利
#73.400000,42.000000
#73.400000,30.000000
#['42.000000', '30.000000', '79.800000', '73.400000'],#鸡屁股下
#鸡下补充
#79.800000,30.0000000 #巴雷利
#97.300000,30.000000 #拉萨昌都（锚点）
#97.300000,26.800000
#79.800000,26.800000
#['42.000000', '30.000000', '97.300000', '79.800000'],#鸡下补充
#鸡下
#97.300000,30.000000 #拉萨昌都（锚点）
#105.000000,30.000000 #（成都重庆贵阳）
#105.000000,21.000000 #河内
#97.300000,21.000000 #曼德勒
#['30.000000', '21.000000', '105.000000', '97.300000'],#鸡下 #鸡胸
#105.0000000,42.000000 #锚点（银川上面）
#105.0000000,21.000000 #河内
#129.000000,42.000000 #沈阳
#129.000000,21.000000 #台北（海）
#['42.000000', '21.000000', '129.000000', '105.000000'],#鸡胸 #鸡身
#105.000000,42.000000 #锚点（银川上面）
#79.800000,42.000000 #啊拉木图
#79.800000,30.000000 #巴雷利
#105.000000,30.000000 #（成都重庆贵阳）
#['42.000000', '30.000000', '105.000000', '79.800000'],#鸡身
#鸡脚 （南海）
#105.000000,21.000000 #河内
#129.000000,21.000000 #台北（海）
#105.0000000,3.000000 #新加坡
#129.000000,3.000000 #海（苏拉威西海）
#['21.000000', '3.000000', '129.000000', '105.000000'],
经纬度分析
import requests
from decimal import Decimal
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的，经度，纬度，
obtain(address)#传入经度，纬度，获取地址地名
slon -= precision #['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal('42.000000')#经度longitude开始
elon = Decimal('30.000000')#经度结束
slat=Decimal('105.000000')# 纬度latitude开始
elat=Decimal('79.800000')#纬度结束
precision = Decimal('5.0001000')#精度 0.0001000==1000米
while slat>=elat:
slat -= precision
longitude(slon,precision,elon,slat)
根据精度获取多个地名
import requests
from decimal import Decimal
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的，经度，纬度，
obtain(address)#传入经度，纬度，获取地址地名
slon -= precision LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 （南海）
]
precision = Decimal('10.0001000') # 精度 0.0001000==1000米
for single_list in LogLatList:
slons,elons,slats,elats=single_list
#['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons)#经度longitude开始
elon = Decimal(elons)#经度结束
slat=Decimal(slats)# 纬度latitude开始
elat=Decimal(elats)#纬度结束 while slat>=elat:
slat -= precision
longitude(slon,precision,elon,slat)
批量获取数据
import requests
from decimal import Decimal
import threading , time
def obtain(address):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s) def longitude(slon,precision,elon,slat):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的，经度，纬度，
obtain(address)#传入经度，纬度，获取地址地名
slon -= precision class Thre(threading.Thread):#继承线程中的类
#Thre(single_list,precision,ctime)
def __init__(self,single_list,precision,ctime):
super(Thre,self).__init__()#重新写父类，解决多继承问题
self.precision=precision#精度
self.single_list=single_list#单列表
self.ctime=ctime#当前时间
def run(self):
print('执行线程开始时间:', self.ctime, '执行的列表:',self.single_list)
slons, elons, slats, elats = self.single_list
# ['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons) # 经度longitude开始
elon = Decimal(elons) # 经度结束
slat = Decimal(slats) # 纬度latitude开始
elat = Decimal(elats) # 纬度结束
while slat >= elat:
slat -= self.precision
longitude(slon,self.precision, elon, slat)
else:
end_time=time.time()-self.ctime
print('执行线程所用时间:',end_time, '执行的列表:',self.single_list) def main():
LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 （南海）
]
precision = Decimal('20.0001000') # 精度 0.0001000==1000米 thre_list=[] #线程列表
for single_list in LogLatList:
ctime=time.time()#当前时间
temp=Thre(single_list,precision,ctime)#实例化类
thre_list.append(temp)#线程列表
for thre in thre_list:#线程列表
thre.start()#执行单个线程 if __name__ == '__main__':
main()
多线程获取数据
import requests
from decimal import Decimal
import threading , time
def obtain(address,file_name):
url = 'http://api.map.baidu.com/geocoder?output=json&key=f247cdb592eb43ebac6ccd27f796e2d2&location=' + str(address)
response = requests.get(url)
answer = response.json()
print('得到反解数据', answer)
lng = answer['result']['location']['lng']
lat = answer['result']['location']['lat']
formatted_address = answer['result']['formatted_address']
business = answer['result']['business']
city = answer['result']['addressComponent']['city']
direction = answer['result']['addressComponent']['city']
distance = answer['result']['addressComponent']['direction']
district = answer['result']['addressComponent']['district']
province = answer['result']['addressComponent']['province']
street = answer['result']['addressComponent']['street']
street_number = answer['result']['addressComponent']['street_number']
cityCode = answer['result']['cityCode']
s = str('%6f' % lng) + '|' + str('%6f' % lat) + '|' + str(formatted_address) + '|' + str(business) + '|' + str(
city) + '|' + str(direction) + '|' + str(distance) + '|' + str(district) + '|' + str(province) + '|' + str(
street) + '|' + str(street_number) + '|' + str(cityCode)
print(s)
if cityCode==0:
#print('外国')
pass
else:
#print(s)
fname=str(file_name)+'list'
with open(fname, 'a+', encoding=('utf-8')) as f:
f.write(s+'\n') def longitude(slon,precision,elon,slat,file_name):
while slon >= elon:
address = '%s,%s' % (slon,slat) # 请求用的，经度，纬度，
obtain(address,file_name)#传入经度，纬度，获取地址地名
slon -= precision class Thre(threading.Thread):#继承线程中的类
#Thre(single_list,precision,ctime)
#temp = Thre(single_list, precision, ctime, file_name) # 实例化类
def __init__(self,single_list,precision,ctime,file_name):
super(Thre,self).__init__()#重新写父类，解决多继承问题
self.precision=precision#精度
self.single_list=single_list#单列表
self.ctime=ctime#当前时间
self.file_name=file_name#文件名
def run(self):
print('执行线程开始时间:', self.ctime, '执行的列表:',self.single_list)
slons, elons, slats, elats = self.single_list
# ['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
slon = Decimal(slons) # 经度longitude开始
elon = Decimal(elons) # 经度结束
slat = Decimal(slats) # 纬度latitude开始
elat = Decimal(elats) # 纬度结束
while slat >= elat:
slat -= self.precision
longitude(slon,self.precision, elon, slat,self.file_name)
else:
end_time=time.time()-self.ctime
print('执行线程所用时间:',end_time, '执行的列表:',self.single_list) def main():
LogLatList=[
['42.000000', '30.000000', '105.000000', '79.800000'], # 鸡身
['42.000000', '21.000000', '129.000000', '105.000000'], # 鸡胸
['50.000000', '42.000000', '135.000000','129.000000'], #鸡头2
['54.000000', '42.000000', '129.000000', '115.000000'], #鸡头1
['45.500000', '42.000000', '115.000000', '105.000000'], #鸡头补充
['49.200000', '42.000000', '91.500000', '79.800000'], # 鸡屁股上
['42.000000', '30.000000', '79.800000', '73.400000'], # 鸡屁股下
['45.500000', '42.000000', '105.000000', '91.500000'], #鸡屁股上补充
['30.000000', '21.000000', '105.000000', '97.300000'], # 鸡下
['42.000000', '30.000000', '97.300000', '79.800000'], #鸡下补充
['21.000000', '3.000000', '129.000000', '105.000000'], # 鸡脚 （南海）
]
# ——测试使用——开始————
precision = Decimal('1.0001000') # 精度 0.0001000==1000米
# ——测试使用——结束———— #——实际获取时替换——精度自己控制————建议是 0.0001000 ————开始————
#precision = Decimal('0.0001000') # 精度 0.0001000==1000米
# ——实际获取时替换——精度自己控制————建议是 0.0001000 ————结束———— file_name=0 thre_list=[] #线程列表
for single_list in LogLatList:
ctime=time.time()#当前时间
temp=Thre(single_list,precision,ctime,file_name)#实例化类
file_name+=1
thre_list.append(temp)#线程列表
for thre in thre_list:#线程列表
thre.start()#执行单个线程 if __name__ == '__main__':
main()
写入到文件
爬虫——python——百度地图经纬度查询——经纬度查看地点地名——利用百度API获取地名经纬度——爬取所有的中国地址的更多相关文章
- Android百度地图开发-第一篇：申请、搭建百度地图
一.前言 这是第一篇关于Android使用百度地图的学习记录,主要记录: 1.在百度地图开发者平台上申请API Key. 2.在自己的应用中加入百度地图的Android版SDK. 3.在自己的应用中显 ...
- 百度地图JavaScript API获取用户当前经纬度和详细地理位置，反之通过详细地理位置获取当前经纬度
前言: 前端时间刚好使用了百度地图的js api定位获取用户当前经纬度并获取当前详细位置和通过当前用户详细地理位置换取用户当前经纬度坐标的功能,为了方便下次找起来方便一些自己在这里记录一下,希望也能够 ...
- python爬虫学习(二)：定向爬虫例子-->使用BeautifulSoup爬取"软科中国最好大学排名-生源质量排名2018"，并把结果写进txt文件
在正式爬取之前,先做一个试验,看一下爬取的数据对象的类型是如何转换为列表的: 写一个html文档: x.html<html><head><title>This is ...
- Python 从底层结构聊 Beautiful Soup 4（内置豆瓣最新电影排行榜爬取案例）
1. 前言 什么是 Beautiful Soup 4 ? Beautiful Soup 4(简称 BS4,后面的 4 表示最新版本)是一个 Python 第三方库,具有解析 HTML 页面的功能,爬虫 ...
- 针对源代码和检查元素不一致的网页爬虫——利用Selenium、PhantomJS、bs4爬取12306的列车途径站信息
整个程序的核心难点在于上次豆瓣爬虫针对的是静态网页,源代码和检查元素内容相同:而在12306的查找搜索过程中,其网页发生变化(出现了查找到的数据),这个过程是动态的,使得我们在审查元素中能一一对应看到 ...
- 简单又强大的pandas爬虫 利用pandas库的read_html()方法爬取网页表格型数据
文章目录 一.简介 二.原理 三.爬取实战 实例1 实例2 一.简介 一般的爬虫套路无非是发送请求.获取响应.解析网页.提取数据.保存数据等步骤.构造请求主要用到requests库,定位提取数据用的比 ...
- Python 爬取图书图片和地址
#-*- coding:utf-8 -*- import xlwt import urllib import re def getHtml(url): page = urllib.urlopen(ur ...
- Python反爬：利用js逆向和woff文件爬取猫眼电影评分信息
首先:看看运行结果效果如何! 1. 实现思路 小编基本实现思路如下: 利用js逆向模拟请求得到电影评分的页面(就是猫眼电影的评分信息并不是我们上述看到的那个页面上,应该它的实现是在一个页面上插入另外一 ...
- 记录开发基于百度地图API实现在地图上绘制轨迹并拾取轨迹对应经纬度的工具说明
前言: 最近一直在做数据可视化方面的工作,其中平面可视化没什么难度,毕竟已经有很多成熟的可供使用的框架,比如百度的echart.js,highcharts.js等.还有就是3D可视化了,整体来说难度也 ...
随机推荐
- unittest(2)
测试用例执行顺序 1.setUp和tearDown相关 setUp:表示前置条件,它在每一个用例执行之前必须会执行一次 setUp可以理解为我们需要自动化测试时,需要打开网页窗口,输入对 ...
- Nginx常用功能配置二
Nginx常用功能配置二 Nginx location匹配设置 location作用:可以根据用户请求的URI来执行不同的应用,根据用户请求的网站的地址URL匹配. location语法: locat ...
- java Twain 直接打印/界面打印
这两天,在搞归档系统.需要用到Twain协议来驱动扫描仪. 找了两天,java的twain操作资料真的不多.而且我还是要找直接打印的功能. 后来只能静下心来看类库和源码.最后搞定他. 打印方式分为3种 ...
- PHP算法之按奇偶排序数组
给定一个非负整数数组 A,返回一个数组,在该数组中, A 的所有偶数元素之后跟着所有奇数元素. 你可以返回满足此条件的任何数组作为答案. 示例: 输入:[3,1,2,4]输出:[2,4,3,1]输出 ...
- Git 获取项目git clone
git clone 克隆项目 git clone 实际上是一个封装了其他几个命令的命令. 它创建了一个新目录,切换到新的目录,然后 git init 来初始化一个空的 Git 仓库, 然后为你指定的 ...
- es5 JSON对象
1. JSON.stringify(obj/arr) js对象(数组)转换为json对象(数组) 2. JSON.parse(json) json对象(数组)转换为js对象(数组) <!DOCT ...
- 93. 复原IP地址
题目描述: 给定一个只包含数字的字符串,复原它并返回所有可能的 IP 地址格式. 示例: 输入: "25525511135" 输出: ["255.255.11.135&q ...
- [JZOJ6299] 2019.08.12【NOIP提高组A】工厂
题目 题目大意 工厂内每个人只会操作一些机器. 他们会以随机的顺序来,每次选任意一台机器来操作. 一台机器只能由一个工人来操作. 可以花费一的代价来使某个工人学会一种机器. 问花费最少的代价,使得在所 ...
- yield支持的协程
#_author:来童星#date:2019/12/12def consumer(name): print("--->start...") while True: new_b ...
- php curl的正确使用方法
在做一个读取远程抓取数据并显示的demo的时候,遇到了以下几个问题: 1.用的curl变量进行了多定义 2.抓取远程数据时没有返回正确的json数据 没有返回正确的json数据不是因为网站提供的接口问 ...