Skip to content

Data Layer

Three managed data services run on DigitalOcean, all within the same private network as the backend services.

PostgreSQL — Primary Database

Provider: DigitalOcean Managed PostgreSQL
Access: Private VPC — not exposed to the public internet

The single source of truth for all application data.

Key Tables (high level)

TableDescription
accountsCustomer and agent company records
usersIndividual user accounts linked to an account
campaignsBulk send campaigns — message, contact list, send_at, status
messagesEvery outbound SMS — status, timestamps, cost
dlrsRaw DLR events received from Celcom Africa
contactsContact records per account
contact_listsNamed contact groups
contact_validation_jobsAsync validation job status per list upload
sender_idsRegistered and approved sender IDs
creditsCredit balance ledger per account
top_upsM-Pesa top-up transactions
invoicesBilling invoices
pricing_tiersPer-account rate tables (direct customers and agents)
agent_accountsAgent ↔ sub-customer relationships
agent_customer_pricingPer-customer rates set by the agent for their sub-customers
mno_prefixesMNO prefix registry per country — used for phone number validation
content_blocklistWords and patterns checked by the SMS Content Worker

Message Status Lifecycle

Message status:

submitted → queued → sent → delivered
                          → failed
                          → expired

Campaign status:

submitted → pending_content_check → approved ──► queued → sending → completed

                                    send_at in future

                                             └──► scheduled ──► queued (when due)
                                                             └──► cancelled
                                  → rejected (content check failed)
  • submitted — API accepted the request
  • pending_content_check — bulk campaign awaiting async content check
  • approved — content check passed; routed to queued or scheduled
  • scheduled — future send_at set; held in PostgreSQL, nothing in RabbitMQ yet
  • queued — jobs published to sms.normal, SMS Worker processing
  • sending — partially sent (large campaigns)
  • completed — all messages submitted to Celcom
  • rejected — content check failed; customer must review
  • cancelled — customer cancelled before send_at; credits refunded

Redis — Cache & Session Store

Provider: DigitalOcean Managed Redis
Access: Private VPC

PurposeKey Pattern
JWT refresh token storesession:{userId}:{tokenId}
Rate limitingratelimit:{accountId}:{window}
Credit balance cachebalance:{accountId}
Sender ID lookup cachesenderid:{accountId}
Idempotency keysidem:{requestId}
Content check result cachecontent:{hash}

Credit balance and sender ID lookups are cached with short TTLs (≤ 30s) to reduce database load during high-volume sends. The authoritative value is always in PostgreSQL.

Content check results are cached by message body hash — if the same message body is submitted again it skips the full blocklist scan.


RabbitMQ — Message Queue

Provider: DigitalOcean Managed RabbitMQ
Access: Private VPC

Queues

QueuePublisherConsumerPurpose
sms.priorityMain APISMS WorkerOTP / transactional sends
sms.normalMain API / SchedulerSMS WorkerStandard sends and due campaigns
sms.contentMain APISMS Content WorkerBulk campaign content checks
dlr.inboundDLR Webhook ServiceDLR Processor WorkerIncoming delivery reports
contacts.validateMain APIPhone Validation WorkerContact list validation jobs

Message Durability

All queues are durable. Messages are persisted to disk so they survive a RabbitMQ restart. Workers acknowledge messages only after completing their work — failures cause RabbitMQ to redeliver to another instance.

Backpressure

If Celcom Africa is slow or unreachable, messages accumulate in sms.normal and sms.priority. The Main API continues accepting new submissions (returns 202 Accepted) while the worker retries. This decouples API throughput from upstream aggregator latency.

If the DLR Processor Worker is slow (e.g. DB contention), DLR events accumulate in dlr.inbound. The DLR Webhook Service continues returning 200 OK to Celcom regardless — no DLRs are dropped.

Internal use only — Sema Link Engineering