Back to Patterns

Lightweight Message Queue

A first-class, SQL-native message queue that leverages surveilr's ingestion and watch capabilities. Treat inbound messages as durable resources in uniform_resource, expose queue semantics via SQL views, and optionally add an SQS-compatible API facade for standard client interoperability. The goal is not to compete with high-throughput MQ systems — it's to provide a lightweight, auditable queue that's 'good enough' for many regulated operations.

Key Benefits

  • No external MQ cluster to manage — just surveilr and SQLite
  • Queue history becomes evidence — fully auditable and SQL-queryable
  • Ubiquitous ingress paths: file drops, SFTP, S3, IMAP/SMTP
  • CloudEvents, MQTT, and SQS payload compatibility
  • Visibility timeouts, acknowledgements, retries, and DLQ built-in

Use Cases

  • Compliance evidence ingestion and routing
  • Continuous control checks and scan outputs
  • Workflow orchestration in regulated environments
  • Queue-with-a-database patterns for SQL introspection
  • Environments where only file drops and email are available

Core Features

SQS-compatible API

File & IMAP ingress

SQL-queryable queue log

Dead-letter queues