--- title: Merge Data keywords: fastai sidebar: home_sidebar summary: "This notebook was made to demonstrate how to merge datasets by matching a single columns values from two datasets. We add columns of data from a foreign dataset into the ACS data we downloaded in our last tutorial." description: "This notebook was made to demonstrate how to merge datasets by matching a single columns values from two datasets. We add columns of data from a foreign dataset into the ACS data we downloaded in our last tutorial." ---
{% raw %}
{% endraw %}

This Coding Notebook is the second in a series.

An Interactive version can be found here Open In Colab

This colab and more can be found at https://github.com/BNIA/colabs

  • Content covered in previous tutorials will be used in later tutorials.

  • New code and or information should have explanations and or descriptions attached.

  • Concepts or code covered in previous tutorials will be used without being explaining in entirety.

  • If content can not be found in the current tutorial and is not covered in previous tutorials, please let me know.

  • This notebook has been optimized for Google Colabs ran on a Chrome Browser.

  • Statements found in the index page on view expressed, responsibility, errors and ommissions, use at risk, and licensing extend throughout the tutorial.

About this Tutorial:

Whats Inside?

The Tutorial

In this notebook, the basics of how to perform a merge is introduced

  • We will merge two datasets
  • We will merge two datasets using a crosswalk

Objectives

By the end of this tutorial users should have an understanding of:

  • How dataset merges are performed
  • The types different union approaches a merge can take
  • the 'mergeData' function, and how to use it in the future

Guided Walkthrough

SETUP

{% raw %}
{% endraw %}

(Optional) Local File Access

{% raw %}
# (Optional) Run this cell to gain access to Google Drive (Colabs only) 
from google.colab import drive

# Colabs operates in a virtualized enviornment
# Colabs default directory is at ~/content.
# We mount Drive into a temporary folder at '~/content/drive' 

drive.mount('/content/drive')
{% endraw %} {% raw %}
# Once connected, I navigate to where the data lives in my drive folder.

!cd ./drive/'My Drive'/colabs/DATA
{% endraw %}

Retrieve Datasets

Our example will merge two simple datasets; pulling CSA names using tract ID's.

The First dataset will be obtained from the Census' ACS 5-year serveys.

Functions used to obtain this data were obtained from Tutorial 0) ACS: Explore and Download.

The Second dataset will be obtained using using a CSV from a publicly accessible link

Get the Principal dataset.

We will use the function we created in our last tutorial to download the data!

{% raw %}
# Our download function will use Baltimore City's tract, county and state as internal paramters
# Change these values in the cell below using different geographic reference codes will change those parameters
tract = '*'
county = '510'
state = '24'

# Specify the download parameters the function will receieve here
tableId = 'B19001'
year = '17'
saveAcs = False
{% endraw %} {% raw %}
df = retrieve_acs_data(state, county, tract, tableId, year, saveAcs)
df.head()
Number of Columns 17
B19001_001E_Total B19001_002E_Total_Less_than_$10_000 B19001_003E_Total_$10_000_to_$14_999 B19001_004E_Total_$15_000_to_$19_999 B19001_005E_Total_$20_000_to_$24_999 B19001_006E_Total_$25_000_to_$29_999 B19001_007E_Total_$30_000_to_$34_999 B19001_008E_Total_$35_000_to_$39_999 B19001_009E_Total_$40_000_to_$44_999 B19001_010E_Total_$45_000_to_$49_999 B19001_011E_Total_$50_000_to_$59_999 B19001_012E_Total_$60_000_to_$74_999 B19001_013E_Total_$75_000_to_$99_999 B19001_014E_Total_$100_000_to_$124_999 B19001_015E_Total_$125_000_to_$149_999 B19001_016E_Total_$150_000_to_$199_999 B19001_017E_Total_$200_000_or_more state county tract
NAME
Census Tract 1901 796 237 76 85 38 79 43 36 35 15 43 45 39 5 0 6 14 24 510 190100
Census Tract 1902 695 63 87 93 6 58 30 14 29 23 38 113 70 6 32 11 22 24 510 190200
Census Tract 2201 2208 137 229 124 52 78 87 50 80 13 217 66 159 205 167 146 398 24 510 220100
Census Tract 2303 632 3 20 0 39 7 0 29 8 9 44 29 98 111 63 94 78 24 510 230300
Census Tract 2502.07 836 102 28 101 64 104 76 41 40 47 72 28 60 19 27 15 12 24 510 250207
{% endraw %}

Get the Secondary Dataset

{% raw %}
# Get the Second dataset. 
# Our Example dataset contains Polygon Geometry information. 
# We want to merge this over to our principle dataset. we will grab it by matching on either CSA or Tract

# The url listed below is public.

print('Crosswalk Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv')
print('Boundaries Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv')

inFile = input("\n Please enter the location of your file : \n" )

crosswalk = pd.read_csv( inFile )
crosswalk.head()
Crosswalk Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
Boundaries Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv

 Please enter the location of your file : 
https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
TRACT2010 GEOID2010 CSA2010
0 10100 24510010100 Canton
1 10200 24510010200 Patterson Park North & East
2 10300 24510010300 Canton
3 10400 24510010400 Canton
4 10500 24510010500 Fells Point
{% endraw %}

Perform Merge & Save

{% raw %}
# This little piece of code does nothing important but serves as a 
# friendly reminder of the 4 basic join types.

# Left - returns all left records, only includes the right record if it has a match
# Right - Returns all right records, only includes the left record if it has a match 
# Full - Returns all records regardless of keys matching
# Inner - Returns only records where a key match

from PIL import Image
import requests
from io import BytesIO
url = "https://docs.trifacta.com/download/attachments/123830435/JoinVennDiagram.png"
response = requests.get(url)
img = Image.open(BytesIO(response.content))
size = 328, 328
img.thumbnail(size, Image.ANTIALIAS)
img
{% endraw %} {% raw %}
# Get Columns from both datasets to match on
# You can get these values from the column values above.

# Our Examples will work with the prompted values

print( 'Princpal Columns ' + str(df.columns) + '')
left_on = input("Left on principal column: ('tract') \n" )

print( 'Crosswalk Columns ' + str(crosswalk.columns) + '')
right_on = input("Right on crosswalk column: ('TRACT2010', or, 'TRACTCE10') \n" )

# Specify how the merge will be performed
Princpal Columns Index(['B19001_001E_Total', 'B19001_002E_Total_Less_than_$10_000',
       'B19001_003E_Total_$10_000_to_$14_999',
       'B19001_004E_Total_$15_000_to_$19_999',
       'B19001_005E_Total_$20_000_to_$24_999',
       'B19001_006E_Total_$25_000_to_$29_999',
       'B19001_007E_Total_$30_000_to_$34_999',
       'B19001_008E_Total_$35_000_to_$39_999',
       'B19001_009E_Total_$40_000_to_$44_999',
       'B19001_010E_Total_$45_000_to_$49_999',
       'B19001_011E_Total_$50_000_to_$59_999',
       'B19001_012E_Total_$60_000_to_$74_999',
       'B19001_013E_Total_$75_000_to_$99_999',
       'B19001_014E_Total_$100_000_to_$124_999',
       'B19001_015E_Total_$125_000_to_$149_999',
       'B19001_016E_Total_$150_000_to_$199_999',
       'B19001_017E_Total_$200_000_or_more', 'state', 'county', 'tract'],
      dtype='object')
Left on principal column: ('tract') 
tract
Crosswalk Columns Index(['TRACT2010', 'GEOID2010', 'CSA2010'], dtype='object')
Right on crosswalk column: ('TRACT2010', or, 'TRACTCE10') 
TRACT2010
{% endraw %} {% raw %}
# We will perform a left merge in this example.
# It will return our Principal dataset with columns from the second dataset appended to records where their specified columns match.

how = input("How: (‘left’, ‘right’, ‘outer’, ‘inner’) " )
How: (‘left’, ‘right’, ‘outer’, ‘inner’) left
{% endraw %} {% raw %}
# Actually perfrom the merge

merged_df = pd.merge(df, crosswalk, left_on=left_on, right_on=right_on, how=how)
merged_df = merged_df.drop(left_on, axis=1)
merged_df
B19001_001E_Total B19001_002E_Total_Less_than_$10_000 B19001_003E_Total_$10_000_to_$14_999 B19001_004E_Total_$15_000_to_$19_999 B19001_005E_Total_$20_000_to_$24_999 B19001_006E_Total_$25_000_to_$29_999 B19001_007E_Total_$30_000_to_$34_999 B19001_008E_Total_$35_000_to_$39_999 B19001_009E_Total_$40_000_to_$44_999 B19001_010E_Total_$45_000_to_$49_999 B19001_011E_Total_$50_000_to_$59_999 B19001_012E_Total_$60_000_to_$74_999 B19001_013E_Total_$75_000_to_$99_999 B19001_014E_Total_$100_000_to_$124_999 B19001_015E_Total_$125_000_to_$149_999 B19001_016E_Total_$150_000_to_$199_999 B19001_017E_Total_$200_000_or_more state county TRACT2010 GEOID2010 CSA2010
0 796 237 76 85 38 79 43 36 35 15 43 45 39 5 0 6 14 24 510 190100 24510190100 Southwest Baltimore
1 695 63 87 93 6 58 30 14 29 23 38 113 70 6 32 11 22 24 510 190200 24510190200 Southwest Baltimore
2 2208 137 229 124 52 78 87 50 80 13 217 66 159 205 167 146 398 24 510 220100 24510220100 Inner Harbor/Federal Hill
3 632 3 20 0 39 7 0 29 8 9 44 29 98 111 63 94 78 24 510 230300 24510230300 South Baltimore
4 836 102 28 101 64 104 76 41 40 47 72 28 60 19 27 15 12 24 510 250207 24510250207 Cherry Hill
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
196 1219 84 41 42 72 39 94 69 38 41 87 124 107 40 91 93 157 24 510 272005 24510272005 Cross-Country/Cheswolde
197 883 78 27 31 3 33 31 30 43 23 52 96 133 65 67 53 118 24 510 120201 24510120201 Greater Charles Village/Barclay
198 1835 155 109 136 106 198 94 130 118 83 71 162 135 180 0 43 115 24 510 272004 24510272004 Cross-Country/Cheswolde
199 1679 347 165 130 125 64 78 80 25 50 168 99 166 70 54 29 29 24 510 272006 24510272006 Glen-Fallstaff
200 239791 29106 15759 13992 11786 12818 12304 10610 10261 9641 18773 21563 25273 16094 10011 10712 11088 24 510 10000 0 Baltimore City

201 rows × 22 columns

{% endraw %}

As you can see, our Census data will now have a CSA appended to it.

{% raw %}
# Save Data to User Specified File

import csv

outFile = input("Please enter the new Filename to save the data to ('acs_csa_merge_test': " )
merged_df.to_csv(outFile+'.csv', quoting=csv.QUOTE_ALL) 
{% endraw %}

Final Result

{% raw %}
# Import needed modules
import os, sys, csv, pandas as pd

flag = input("Enter a URL? If not ACS data will be used. (Y/N):  " )
if (flag == 'y' or flag == 'Y'):
  df = pd.read_csv( input("Please enter the location of your Principal file: " ) )
else:
  tract = input("Please enter tract id (*): " )
  county = input("Please enter county id (510): " )
  state = input("Please enter state id (24): " )
  tableId = input("Please enter acs table id (B19001): " ) 
  year = input("Please enter acs year (18): " )
  saveAcs = input("Save ACS? (True/False): " )
  df = retrieve_acs_data(state, county, tract, tableId, year, saveAcs)

print( 'Principal Columns ' + str(df.columns))

print('Crosswalk Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv')
print('Boundaries Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv')

crosswalk = pd.read_csv( input("Please enter the location of your crosswalk file: " ) )
print( 'Crosswalk Columns ' + str(crosswalk.columns) + '\n')

left_on = input("Left on: " )
right_on = input("Right on: " )
how = input("How: (‘left’, ‘right’, ‘outer’, ‘inner’) " )

merged_df = pd.merge(df, crosswalk, left_on=left_on, right_on=right_on, how=how)
merged_df = merged_df.drop(left_on, axis=1)

# Save the data
# Save the data
saveFile = input("Save File ('Y' or 'N'): ")
if saveFile == 'Y' or saveFile == 'y':
  outFile = input("Saved Filename (Do not include the file extension ): ")
  merged_df.to_csv(outFile+'.csv', quoting=csv.QUOTE_ALL);
Enter a URL? If not ACS data will be used. (Y/N):  N
Please enter tract id (*): *
Please enter county id (510): 510
Please enter state id (24): 24
Please enter acs table id (B19001): B19001
Please enter acs year (18): 18
Save ACS? (True/False): False
Number of Columns 17
Principal Columns Index(['B19001_001E_Total', 'B19001_002E_Total_Less_than_$10,000',
       'B19001_003E_Total_$10,000_to_$14,999',
       'B19001_004E_Total_$15,000_to_$19,999',
       'B19001_005E_Total_$20,000_to_$24,999',
       'B19001_006E_Total_$25,000_to_$29,999',
       'B19001_007E_Total_$30,000_to_$34,999',
       'B19001_008E_Total_$35,000_to_$39,999',
       'B19001_009E_Total_$40,000_to_$44,999',
       'B19001_010E_Total_$45,000_to_$49,999',
       'B19001_011E_Total_$50,000_to_$59,999',
       'B19001_012E_Total_$60,000_to_$74,999',
       'B19001_013E_Total_$75,000_to_$99,999',
       'B19001_014E_Total_$100,000_to_$124,999',
       'B19001_015E_Total_$125,000_to_$149,999',
       'B19001_016E_Total_$150,000_to_$199,999',
       'B19001_017E_Total_$200,000_or_more', 'state', 'county', 'tract'],
      dtype='object')
Crosswalk Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
Boundaries Example: https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8xXdUaT17jkdK0MWTJpg3GOy6jMWeaXTlguXNjCSb8Vr_FanSZQRaTU-m811fQz4kyMFK5wcahMNY/pub?gid=886223646&single=true&output=csv
Please enter the location of your crosswalk file: https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
Crosswalk Columns Index(['TRACT2010', 'GEOID2010', 'CSA2010'], dtype='object')

Left on: tract
Right on: TRACT2010
How: (‘left’, ‘right’, ‘outer’, ‘inner’) left
Save File ('Y' or 'N'): N
{% endraw %}

Advanced

Intro

The following Python function is a bulked out version of the previous notes.

  • It contains everything from the tutorial plus more.
  • It can be imported and used in future projects or stand alone.

Description: add columns of data from a foreign dataset into a primary dataset along set parameters.

Purpose: Makes Merging datasets simple

Services

  • Merge two datasets without a crosswalk
  • Merge two datasets with a crosswalk
{% raw %}
{% endraw %} {% raw %}

mergeDatasets[source]

mergeDatasets(left_ds=False, right_ds=False, crosswalk_ds=False, use_crosswalk=True, left_col=False, right_col=False, crosswalk_left_col=False, crosswalk_right_col=False, merge_how=False, interactive=True)

{% endraw %}

Function Explanation

Input(s):

  • Dataset url
  • Crosswalk Url
  • Right On
  • Left On
  • How
  • New Filename

Output: File

How it works:

  • Read in datasets
  • Perform Merge

  • If the 'how' parameter is equal to ['left', 'right', 'outer', 'inner']

    • then a merge will be performed.
  • If a column name is provided in the 'how' parameter
    • then that single column will be pulled from the right dataset as a new column in the left_ds.

Function Diagrams

{% raw %}
#@title Run: Diagram the mergeDatasets()

%%html
<img src="https://charleskarpati.com/images/class_diagram_merge_datasets.png">
{% endraw %} {% raw %}
#@title Run: mergeDatasets Flow Chart

%%html
<img src="https://charleskarpati.com/images/flow_chart_merge_datasets.png">
{% endraw %} {% raw %}
#@title Run: Gannt Chart  mergeDatasets()

%%html
<img src="https://charleskarpati.com/images/gannt_chart_merge_datasets.png">
{% endraw %} {% raw %}
#@title Run: Sequence Diagram  mergeDatasets()

%%html
<img src="https://charleskarpati.com/images/sequence_diagram_merge_datasets.png">
{% endraw %}

Function Examples

Interactive Example 1

{% raw %}
# Table: FDIC Baltimore Banks
# Columns: Bank Name, Address(es), Census Tract
left_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vSHFrRSHva1f82ZQ7Uxwf3A1phqljj1oa2duGlZDM1vLtrm1GI5yHmpVX2ilTfMHQ/pub?gid=601362340&single=true&output=csv'
left_col = 'Census Tract'

# Table: Crosswalk Census Communities
# 'TRACT2010', 'GEOID2010', 'CSA2010'
right_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv'
right_col='TRACT2010'

merge_how = 'outer'
interactive = True
use_crosswalk = True

merged_df = mergeDatasets( left_ds=left_ds, left_col=left_col, 
              right_ds=right_ds, right_col=right_col, 
              merge_how='left', interactive =True, use_crosswalk=use_crosswalk )
 Handling Left Dataset
Please provide a new dataset URL: https://docs.google.com/spreadsheets/d/e/2PACX-1vSHFrRSHva1f82ZQ7Uxwf3A1phqljj1oa2duGlZDM1vLtrm1GI5yHmpVX2ilTfMHQ/pub?gid=601362340&single=true&output=csv
Please provide a new dataset URL: https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv
Valid Column Not Given
Index(['TRACT2010', 'GEOID2010', 'CSA2010'], dtype='object')
   TRACT2010    GEOID2010                      CSA2010
0  10100      24510010100  Canton                     
1  10200      24510010200  Patterson Park North & East
2  10300      24510010300  Canton                     
3  10400      24510010400  Canton                     
4  10500      24510010500  Fells Point                
Please provide a dataset column name from the list above.
Column Name: TRACT2010
Left Dataset and Columns are Valid

 Handling Right Dataset
Right Dataset and Columns are Valid

 Checking the merge_how Parameter
merge_how operator is Valid left

 Checking the Crosswalk Parameter
Are you using a crosswalk? 'True' or 'False': False

 Ensuring Left->Right compatability
PERFORMING MERGE LEFT->RIGHT
left_col TRACT2010 right_col TRACT2010 how left
/usr/local/lib/python3.6/dist-packages/pandas/core/ops/__init__.py:1115: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  result = method(y)
{% endraw %} {% raw %}
merged_df.head()
TRACT2010 GEOID2010_x CSA2010_x GEOID2010_y CSA2010_y
0 10100 24510010100 Canton 24510010100 Canton
1 10200 24510010200 Patterson Park North & East 24510010200 Patterson Park North & East
2 10300 24510010300 Canton 24510010300 Canton
3 10400 24510010400 Canton 24510010400 Canton
4 10500 24510010500 Fells Point 24510010500 Fells Point
{% endraw %}

Example 1.5 ) Get CSA and Geometry with a Crosswalk.

{% raw %}
# Primary Table
# Description: I created a public dataset from a google xlsx sheet 'Bank Addresses and Census Tract' from a workbook of the same name.
# Table: FDIC Baltimore Banks
# Columns: Bank Name, Address(es), Census Tract
left_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vSHFrRSHva1f82ZQ7Uxwf3A1phqljj1oa2duGlZDM1vLtrm1GI5yHmpVX2ilTfMHQ/pub?gid=601362340&single=true&output=csv'
left_col = 'Census Tract'

# Alternate Primary Table
# Description: Same workbook, different Sheet: 'Branches per tract' 
# Columns: Census Tract, Number branches per tract
# left_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vSHFrRSHva1f82ZQ7Uxwf3A1phqljj1oa2duGlZDM1vLtrm1GI5yHmpVX2ilTfMHQ/pub?gid=1698745725&single=true&output=csv'
# lef_col = 'Number branches per tract'

# Crosswalk Table
# Table: Crosswalk Census Communities
# 'TRACT2010', 'GEOID2010', 'CSA2010'
crosswalk_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv'
use_crosswalk = True
crosswalk_left_col = 'TRACT2010'
crosswalk_right_col = 'GEOID2010'

# Secondary Table
# Table: Baltimore Boundaries
# 'TRACTCE10', 'GEOID10', 'CSA', 'NAME10', 'Tract', 'geometry'
right_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTPKW6YOHPFvkw3FM3m5y67-Aa5ZlrM0Ee1Fb57wlGuldr99sEvVWnkej30FXhSb3j8o9gr8izq2ZRP/pub?output=csv'
right_col ='GEOID10'

merge_how = 'geometry'
interactive = True
merge_how = 'outer'

merged_df = mergeDatasets( left_ds=left_ds, left_col=left_col, 
              use_crosswalk=use_crosswalk, crosswalk_ds=crosswalk_ds,
              crosswalk_left_col = crosswalk_left_col, crosswalk_right_col = crosswalk_right_col,
              right_ds=right_ds, right_col=right_col, 
              merge_how=merge_how, interactive = interactive )

merged_df.head()
{% endraw %}

Here we can save the data so that it may be used in later tutorials.

{% raw %}
string = 'test_save_data_with_geom_and_csa'
merged_df.to_csv(string+'.csv', encoding="utf-8", index=False, quoting=csv.QUOTE_ALL)
{% endraw %}

Download data by:

  • Clicking the 'Files' tab in the left hand menu of this screen. Locate your file within the file explorer that appears directly under the 'Files' tab button once clicked. Right click the file in the file explorer and select the 'download' option from the dropdown.

In the next tutorial you will learn how to load this data as a geospatial dataset so that it may be mapped and mapping functionalities may be applied to it.

You can upload this data into the next tutorial in one of two ways.

1)

  • uploading the saved file to google Drive and connecting to your drive path

OR.

2)

  • 'by first downloading the dataset as directed above, and then navigating to the next tutorial. Go to their page and:
  • Uploading data using an file 'upload' button accessible within the 'Files' tab in the left hand menu of this screen. The next tutorial will teach you how to load this data so that it may be mapped.

Interactive Example 2

{% raw %}
# When the prompts come up input the values not included from Interactive Example 1 and you will get the same output.
# This is to demonstrate that not all parameters must be known prior to executing the function.

mergeDatasets( left_ds=left_ds, left_col=left_col, right_ds=right_ds, interactive =True )
{% endraw %} {% raw %}
mergedDataset = mergeDatasets( left_ds=left_ds, left_col=left_col, use_crosswalk=use_crosswalk, right_ds=right_ds, right_col=right_col, merge_how = merge_how, interactive = interactive )
{% endraw %} {% raw %}
mergedDataset.dtypes
{% endraw %}

Interactive Run Alone

{% raw %}
mergeDatasets()
{% endraw %}

Preconfigured Example 1

{% raw %}
# Census Crosswalk
# 'TRACT2010', 'GEOID2010', 'CSA2010'
left_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vREwwa_s8Ix39OYGnnS_wA8flOoEkU7reIV4o3ZhlwYhLXhpNEvnOia_uHUDBvnFptkLLHHlaQNvsQE/pub?output=csv'

# Baltimore Boundaries
# 'TRACTCE10', 'GEOID10', 'CSA', 'NAME10', 'Tract', 'geometry'
right_ds = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTPKW6YOHPFvkw3FM3m5y67-Aa5ZlrM0Ee1Fb57wlGuldr99sEvVWnkej30FXhSb3j8o9gr8izq2ZRP/pub?output=csv'
# The Left DS Cols will map to the first three Right DS Cols listed
left_col = 'GEOID2010'
right_col = 'GEOID10'
merge_how = 'outer'
interactive = True
{% endraw %} {% raw %}
mergeDatasets( left_ds=left_ds, left_col=left_col, right_ds=right_ds, right_col=right_col, merge_how = merge_how, interactive = interactive )
{% endraw %}