Data-as-a-platform API Proof of Concept

This proof of concept will explore migrating the DaaP API to a single python container organised around a python framework (FastAPI).

Getting started

Dependencies

• This project requires python 3.11 and Postgres.
• To run postgres locally via docker, run docker-compose up -d
• Run poetry install to install python dependencies
• Run poetry run alembic upgrade head to create the development database tables

Configuration

Any environment variables set in the .env file will overwrite the configuration.

e.g. to disable Azure Active Directory authentication:

echo AUTH_ENABLED=False >> .env

Starting the API

• Run poetry run uvicorn daap_api.main:app --reload
• The API is served at http://localhost:8000/
• Example endpoint: http://localhost:8000/v1/data-products
• The docs are served at http://localhost:8000/docs or http://localhost:8000/redoc

Running the tests

• poetry run pytest
• poetry run pytest tests/unit to just run the unit tests

Opening a shell

• Python: poetry run python -i -m daap_api.main
• PSQL: docker-compose exec -u postgres postgres_db psql daap_api_dev

Manage database migrations

Schema migrations for the metadata store are written as alembic migrations. These are a set of operations such as op.create_table and op.drop_table, which correspond to the SQL required to upgrade/downgrade the schema to/from a new version.

For more info see: https://alembic.sqlalchemy.org/en/latest/tutorial.html#tutorial

Useful commands:

• poetry run alembic upgrade head - apply all database migrations to the development database
• poetry run alembic revision -m 'Add a column' - create a candidate migration file in /migrations
• poetry run alembic downgrade -1 - revert the most recent migration
• poetry run alembic downgrade base - reset database to the starting state
• poetry run alembic current - show the databases's current version

Architecture notes

See docs/architecture.md

Visualising the data model