Use API to retrieve data from internet

Reference: Working with APIs

Many big companies and organizations provide API for us to retrieve data, and we use  requests library to get data with Json format from them.

1: Type of requests

There are many types of request for getting data, We can use a simple GET request to retrieve information from the OpenNotify API. OpenNotify has several API endpoints. An endpoint is a server route that is used to retrieve different data from the API. For example, the /comments endpoint on the Reddit API might retrieve information about comments, whereas the/users endpoint might retrieve data about users.

For example:

# Make a get request to get the latest position of the international space station from the opennotify api.
response = requests.get("http://api.open-notify.org/iss-now.json")
status_code = response.status_code

And you can see a listing of all the endpoints on OpenNotify here.

2: Status codes

The request we just made had a status code of 200. Status codes are returned with every request that is made to a web server. Status codes indicate information about what happened with a request. Here are some codes that are relevant toGET requests:

• 200 -- everything went okay, and the result has been returned (if any)
• 301 -- the server is redirecting you to a different endpoint. This can happen when a company switches domain names, or an endpoint name is changed.
• 401 -- the server thinks you're not authenticated. This happens when you don't send the right credentials to access an API (we'll talk about this in a later mission).
• 400 -- the server thinks you made a bad request. This can happen when you don't send along the right data, among other things.
• 403 -- the resource you're trying to access is forbidden -- you don't have the right permissions to see it.
• 404 -- the resource you tried to access wasn't found on the server.

 For example:

# Make a get request to get the latest position of the international space station from the opennotify api.
response = requests.get("http://api.open-notify.org/iss-pass.json")
status_code = response.status_code

3: Query parameters

Look at documentation for the OpenNotify API, we see that the ISS Pass endpoint requires two parameters.

The ISS Pass endpoint returns when the ISS will next pass over a given location on earth. In order to compute this, we need to pass the coordinates of the location to the API. We do this by passing two parameters -- latitude and longitude.

We can make a dictionary with these parameters, and then pass them into the function.

We can also do the same thing directly by adding the query parameters to the url, like this: http://api.open-notify.org/iss-pass.json?lat=40.71&lon=-74.

For example we send San the coordinates of Francisco as parameters to the API:

response = requests.get("http://api.open-notify.org/iss-pass.json", params={"lat":37.78, "lon":-122.41})
content = response.content

And the content is:

b'{\n  "message": "success", \n  "request": {\n    "altitude": 100, \n    "datetime": 1441417753, \n    "latitude": 37.78, \n    "longitude": -122.41, \n    "passes": 5\n  }, \n  "response": [\n    {\n      "duration": 369, \n      "risetime": 1441456672\n    }, \n    {\n      "duration": 626, \n      "risetime": 1441462284\n    }, \n    {\n      "duration": 581, \n      "risetime": 1441468104\n    }, \n    {\n      "duration": 482, \n      "risetime": 1441474000\n    }, \n    {\n      "duration": 509, \n      "risetime": 1441479853\n    }\n  ]\n}'

The json library has two main methods:

• dumps -- Takes in a Python object, and converts it to a string.
• loads -- Takes a json string, and converts it to a Python object.

 We can use this 2 functions to convert normal parameters between json format.

To see more details about the parameters format JSON, please click the link: JavaScript Object Notation

4: Getting JSON from a request

We can get the content of a response as a Python object by using the .json() method on the response.

Get the duration value of the first pass of the ISS over San Francisco, and assign the value the first_pass_duration.

# Make the same request we did 2 screens ago.
parameters = {"lat": 37.78, "lon": -122.41}
response = requests.get("http://api.open-notify.org/iss-pass.json", params=parameters)

# Get the response data as a python object.  Verify that it's a dictionary.
data = response.json()
print(type(data))
print(data)
firt_pass_duration = data["reponse"][0]["duration"]

As we can see just now, we can transform the json format data from the response to dictionary for processing.

5: Content type

The server doesn't just send a status code and the data when it generates a response. It also sends metadata containing information on how the data was generated and how to decode it. This is stored in the response headers. We can access this with the .headers property of a response.

The headers will be shown as a dictionary. Within the headers, content-type is the most important key for now. It tells us the format of the response, and how to decode it. For the OpenNotify API, the format is json, which is why we could decode it with json earlier.

Exercise:

OpenNotify has a API endpoint, astros.json. It tells we how many people are currently in space.

We can see the details here.

Here is the sample code:

import requests
parameters = {"lat":39.92889, "lon":116.38833}
response = requests.get("http://api.open-notify.org/astros.json", params=parameters)
data = response.json()
#print(data)
in_space_count = data['number']
print(in_space_count)