Skip to content
iamonahPublic

About

A Go-based background job processing system exploring AWS S3, SQS, worker concurrency, and reliable job processing with the transactional outbox pattern.

Resources

Stars

3 stars

Watchers

0 watching

Forks

Latest commit

 

History

71 Commits

Folders and files

Repository files navigation

Drift

An asynchronous API built with Go to explore Amazon SQS and background job processing.

Architecture

Drift infrastructure

What I'm Exploring

  • Asynchronous API design
  • Amazon SQS
  • Background workers
  • SQS visibility timeout and retries
  • Message processing and acknowledgement
  • Idempotent job processing
  • PostgreSQL for job state
  • S3 for generated reports

Stack

  • Go
  • Amazon SQS
  • PostgreSQL
  • Amazon S3

Built primarily as a hands-on project for understanding Amazon SQS and asynchronous job processing.

Configuration

The application reads its settings from app.yaml. This file contains the database connection settings, server settings, and AWS configuration.

docker compose up -d driftdb
make run

For integration tests, start the isolated database and run the tests:

make test-db
make test

Keep production credentials out of a committed app.yaml; use a deployment- specific configuration file or a secret-management approach before deploying.

About

A Go-based background job processing system exploring AWS S3, SQS, worker concurrency, and reliable job processing with the transactional outbox pattern.

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages