Modules¶
Data Processing contains 5 modules: 1. Authentication 2. Extract 3. Transform 4. Load 5. Helpers
Authentication¶
Fill in the config.ini.example with the proper user credentials for the project. Rename this file to config.ini. Do not store passwords in this file Use .gitignore to ignore .ini files to prevent uploading them to github.
Extract¶
Functions to extract data from the Objectstore where raw files are stored, from API’s or from the data catalog.
Transform Geospatial¶
Functions to spatial transform and enrich datasets.
Load¶
Common functions to load data into the Objectstore, CKAN (our public data.amsterdam.nl) or PostgreSQL.