Link Analysis_2_Application

US Cities Distribution Network

1.1 Task Description

Nodes: Cities with attributes (1) location, (2) population;

Edges: Connections between cities with weight attribute the cost of traveling;

%matplotlib notebook
import networkx as nx
import matplotlib.pyplot as plt
G = nx.read_gpickle('major_us_cities')

G.nodes(data=True)

[('El Paso, TX', {'location': (-106, 31), 'population': 674433}),
 ('Long Beach, CA', {'location': (-118, 33), 'population': 469428}),
 ('Dallas, TX', {'location': (-96, 32), 'population': 1257676}),
 ('Oakland, CA', {'location': (-122, 37), 'population': 406253}),
 ('Albuquerque, NM', {'location': (-106, 35), 'population': 556495}),
 ('Baltimore, MD', {'location': (-76, 39), 'population': 622104}),
 ('Raleigh, NC', {'location': (-78, 35), 'population': 431746}),
 ('Mesa, AZ', {'location': (-111, 33), 'population': 457587}),
 ('Arlington, TX', {'location': (-97, 32), 'population': 379577}),
 ('Sacramento, CA', {'location': (-121, 38), 'population': 479686}),
 ('Wichita, KS', {'location': (-97, 37), 'population': 386552}),
 ('Tucson, AZ', {'location': (-110, 32), 'population': 526116}),
 ('Cleveland, OH', {'location': (-81, 41), 'population': 390113}),
 ('Louisville/Jefferson County, KY',
  {'location': (-85, 38), 'population': 609893}),
 ('San Jose, CA', {'location': (-121, 37), 'population': 998537}),
 ('Oklahoma City, OK', {'location': (-97, 35), 'population': 610613}),
 ('Atlanta, GA', {'location': (-84, 33), 'population': 447841}),
 ('New Orleans, LA', {'location': (-90, 29), 'population': 378715}),
 ('Miami, FL', {'location': (-80, 25), 'population': 417650}),
 ('Fresno, CA', {'location': (-119, 36), 'population': 509924}),
 ('Philadelphia, PA', {'location': (-75, 39), 'population': 1553165}),
 ('Houston, TX', {'location': (-95, 29), 'population': 2195914}),
 ('Boston, MA', {'location': (-71, 42), 'population': 645966}),
 ('Kansas City, MO', {'location': (-94, 39), 'population': 467007}),
 ('San Diego, CA', {'location': (-117, 32), 'population': 1355896}),
 ('Chicago, IL', {'location': (-87, 41), 'population': 2718782}),
 ('Charlotte, NC', {'location': (-80, 35), 'population': 792862}),
 ('Washington D.C.', {'location': (-77, 38), 'population': 646449}),
 ('San Antonio, TX', {'location': (-98, 29), 'population': 1409019}),
 ('Phoenix, AZ', {'location': (-112, 33), 'population': 1513367}),
 ('San Francisco, CA', {'location': (-122, 37), 'population': 837442}),
 ('Memphis, TN', {'location': (-90, 35), 'population': 653450}),
 ('Los Angeles, CA', {'location': (-118, 34), 'population': 3884307}),
 ('New York, NY', {'location': (-74, 40), 'population': 8405837}),
 ('Denver, CO', {'location': (-104, 39), 'population': 649495}),
 ('Omaha, NE', {'location': (-95, 41), 'population': 434353}),
 ('Seattle, WA', {'location': (-122, 47), 'population': 652405}),
 ('Portland, OR', {'location': (-122, 45), 'population': 609456}),
 ('Tulsa, OK', {'location': (-95, 36), 'population': 398121}),
 ('Austin, TX', {'location': (-97, 30), 'population': 885400}),
 ('Minneapolis, MN', {'location': (-93, 44), 'population': 400070}),
 ('Colorado Springs, CO', {'location': (-104, 38), 'population': 439886}),
 ('Fort Worth, TX', {'location': (-97, 32), 'population': 792727}),
 ('Indianapolis, IN', {'location': (-86, 39), 'population': 843393}),
 ('Las Vegas, NV', {'location': (-115, 36), 'population': 603488}),
 ('Detroit, MI', {'location': (-83, 42), 'population': 688701}),
 ('Nashville-Davidson, TN', {'location': (-86, 36), 'population': 634464}),
 ('Milwaukee, WI', {'location': (-87, 43), 'population': 599164}),
 ('Columbus, OH', {'location': (-82, 39), 'population': 822553}),
 ('Virginia Beach, VA', {'location': (-75, 36), 'population': 448479}),
 ('Jacksonville, FL', {'location': (-81, 30), 'population': 842583})]

1.2 Create Layouts for Plotting

Dictionary for node positioning methods:

[x for x in nx.__dir__() if x.endswith('_layout')]

['circular_layout',
 'random_layout',
 'shell_layout',
 'spring_layout',
 'spectral_layout',
 'fruchterman_reingold_layout']

1.2.1 Spring Layout (default) Node Positioning: (1) As few crossing edges as possible; (2) Keep edge length similar.

plt.figure(figsize=(10,9))
nx.draw_networkx(G)

1.2.2 Random Layout

plt.figure(figsize=(10,9))
pos = nx.random_layout(G)
nx.draw_networkx(G, pos)

1.2.3 Cicular Layout

plt.figure(figsize=(10,9))
pos = nx.circular_layout(G)
nx.draw_networkx(G, pos)

1.2.4 Custom Layout

plt.figure(figsize=(10,7))
pos = nx.get_node_attributes(G, 'location')
nx.draw_networkx(G, pos)

plt.figure(figsize=(10,7))
nx.draw_networkx(G, pos, alpha=0.7, with_labels=False, edge_color='.4')
plt.axis('off')
plt.tight_layout();

Set size of nodes based on population, multiply pop with small number so plots won't be large.

Get weights of transportation costs and pass it to edges.

plt.figure(figsize=(10,7))
node_color = [G.degree(v) for v in G]
node_size = [0.0005 * nx.get_node_attributes(G, 'population')[v] for v in G]
edge_width = [0.0015*G[u][v]['weight'] for u,v in G.edges()]
nx.draw_networkx(G, pos, node_size=node_size,
                 node_color=node_color, alpha=0.7, with_labels=False,
                 width=edge_width, edge_color='.4', cmap=plt.cm.Blues)
plt.axis('off')
plt.tight_layout();

Display the most expensive costs, i.e., separately add specific labels and edges.

plt.figure(figsize=(10,7))
node_color = [G.degree(v) for v in G]
node_size = [0.0005*nx.get_node_attributes(G, 'population')[v] for v in G]
edge_width = [0.0015*G[u][v]['weight'] for u,v in G.edges()]
nx.draw_networkx(G, pos, node_size=node_size,
                 node_color=node_color, alpha=0.7, with_labels=False,
                 width=edge_width, edge_color='.4', cmap=plt.cm.Blues)
greater_than_770 = [x for x in G.edges(data=True) if x[2]['weight']>770]
nx.draw_networkx_edges(G, pos, edgelist=greater_than_770, edge_color='r', alpha=0.4, width=6)
nx.draw_networkx_labels(G, pos, labels={'Los Angeles, CA': 'LA', 'New York, NY': 'NYC'}, font_size=18, font_color='w')
plt.axis('off')
plt.tight_layout();

1.3 Degree Distribution

Probability distributions over entire network

# function degree() returns a dictionary with keys being nodes and
# values being degrees of nodes
degrees = G.degree()
degree_values = sorted(set(degrees.values()))
histogram = [list(degrees.values()).count(i)/float(nx.number_of_nodes(G)) for i in degree_values]
import matplotlib.pyplot as plt
plt.bar(degree_values, histogram)
plt.xlabel('Degree')
plt.ylabel('Fraction of Nodes')
plt.show()

1.4 Extracting Attributes

1.4.1 Node-based Method

Transform into DataFrame columns, initialize the dataframe, using the nodes as the index:

df = pd.DataFrame(index = G.nodes())

df['location'] = pd.Series(nx.get_node_attributes(G, 'location'))
df['population'] = pd.Series(nx.get_node_attributes(G, 'population'))
df.head()

Add features:

df['clustering'] = pd.Series(nx.clustering(G))
df['degree'] = pd.Series(G.degree())
df

1.4.2 Edge-based Features

Initialize the DataFrame, using the edges as the index:

G.edges(data=True)
df = pd.DataFrame(index=G.edges())
df['weight'] = pd.Series(nx.get_edge_attributes(G, 'weight'))
df

df['preferential attachment'] = [i[2] for i in nx.preferential_attachment(G, df.index)]
df['Common Neighbors'] = df.index.map(lambda city: len(list(nx.common_neighbors(G, city[0], city[1]))))
df