AdTech & CRM Platform
Multi-module Laravel platform for Vietnamese ad agencies that reconciles Facebook ad spend against bank debits, automates VN e-invoicing, and pushes real-time balance changes over WebSocket.
- Client
- Confidential · AdTech / Agency SaaS
- Role
- {{TODO: e.g. Solution Architect / Lead Backend Engineer}}
- Period
- {{TODO: e.g. 2023–2024}}
Summary
Overview
The platform lets Vietnamese digital agencies run Facebook ad campaigns and bill their customers in one platform. It syncs campaign, ad-set and insight data from the Facebook Graph API, tracks per-customer balances and debt, issues legally compliant VN e-invoices, and drives fund transfers through Vietinbank iPayAPI — surfacing balance and campaign changes to operators in real time over Pusher.
The architectural core is multi-source financial reconciliation: money spent on Facebook must be matched against money debited from the bank, across systems that update asynchronously on different schedules. A dual-database design isolates denormalized, read-heavy Facebook data from the ACID transactional data that billing, banking and partner contracts depend on.
Context
The Problem
An agency's real cost of advertising is split across two systems that never agree in real time: Facebook reports spend on its own cadence, while the bank debits funds on another. Customers are billed on those numbers, so any drift becomes a financial discrepancy — and Vietnamese tax law requires a signed, auditable e-invoice for every charge.
Constraints
- Facebook spend and bank debits must be reconciled despite async updates and clock skew between systems.
- Financial state must stay correct while many queue jobs mutate balances concurrently.
- Every invoice must be signed and filed to satisfy VN e-invoicing regulation (MeInvoice + HSM).
- Read-heavy, denormalized Facebook data must not contend with transactional billing and banking writes.
- The system must degrade gracefully when Redis, the Facebook API, or the banking API is unavailable.
Scope
Requirements
Functional
- ▹Facebook Ads sync (campaigns, ad sets, ads, creatives, insights, activities) via Graph API v22
- ▹Campaign status detection, daily budget management and future balance projection
- ▹Hierarchical customers (parent/child) with tiered service-fee auto-calculation
- ▹Vietnamese e-invoicing via MeInvoice with HSM signing and invoice versioning
- ▹Multi-snapshot balance tracking (T0 / T1 / T2) with debt and payment reconciliation
- ▹Vietinbank iPayAPI login, OTP, fund transfers and card-account reconciliation
- ▹Partner hierarchy with contract generation (LibreOffice) and role-based access
- ▹Real-time campaign/balance broadcasts (Pusher) plus Zalo notifications and Excel exports
Non-functional
- ▹Reconciliation accuracy between Facebook spend and bank debit under async updates
- ▹Financial consistency while concurrent queue jobs mutate balances
- ▹Regulatory compliance for VN e-invoicing (MeInvoice + HSM signing)
- ▹Isolation of read-heavy Facebook data from transactional billing/banking writes
- ▹Graceful degradation: DB-backed queue fallback when Redis is absent, retry logic for API calls
- ▹Auditability via activity logs, transaction tracking and invoice versioning
System Design
Architecture
Dual-Database Persistence
A transactional main MySQL connection for billing, banking, partners and contracts, plus a read-only Facebook MySQL connection holding denormalized, non-incremental-ID insight data so sync churn never contends with financial writes.
Reconciliation Engine
Queue jobs and scheduled commands match Facebook spend against Vietinbank debits with timestamp tolerance, tracking cross-system transactions and surfacing discrepancies.
Balance Snapshots (T0/T1/T2)
Multi-snapshot balance tracking separates settled state from projected state so async updates never corrupt the number a customer is billed on.
E-Invoicing Pipeline
Invoice generation embeds MeInvoice submission with HSM signing inside queue jobs, applying VAT and tiered service fees and keeping versioned records for tax audit.
Async Workers + Scheduler
25 queue jobs and 14 scheduled commands run campaign sync, balance calculation, bank checks and notifications off the request path, with DB-backed queues as a Redis fallback.
Real-time Broadcast Layer
Pusher (with a Soketi compatibility layer) pushes campaign and balance changes to operators, using device tokens for targeted delivery; Zalo carries balance and billing-threshold alerts.
Stack
Technology Stack
Backend
Data
Frontend
Realtime / Files
Integrations
Ops / Monitoring
Persistence
Data Model
- Customer (main): hierarchical parent/child, balance snapshots T0/T1/T2, service tier
- Invoice (main): MeInvoice integration, VAT + service-fee calc, status draft/published/paid, versioned
- Transaction (main): cross-system reconciliation record matching Facebook spend to bank debit
- Partner (main): contract document generation, smartCA integration, role hierarchy
- User (main): roles manager/accountant/partner, device_token for WebSocket subscriptions
- AdAccount (Facebook DB): balance, threshold alerts, transaction history, activity log
- Campaign (Facebook DB): FK to ad account, budget tracking, daily/lifetime insights, status flags
Reasoning
Key Decisions & Trade-offs
Separate read-only Facebook database
trade-off
No normalized joins to transactional data and extra sync complexity, but read-heavy denormalized insight data with non-incremental IDs never contends with billing and banking writes.
Multi-module monorepo instead of microservices
trade-off
Module coupling limits independent scaling, but a shared Laravel stack keeps deployment simple and one codebase.
Redis optional with DB-backed queue fallback
trade-off
Lower throughput when Redis is absent, but the platform keeps processing jobs under spotty connectivity.
Dual e-invoicing via MeInvoice plus HSM signing
trade-off
More code and integration complexity, but it satisfies the Vietnamese legal and tax-audit mandate.
Balance snapshots T0/T1/T2 separating settled from projected state
trade-off
More balance rows to maintain, but async spend and debit updates can't corrupt the billed figure.
Operations
Scaling & Reliability
Horizontal: multiple queue-worker processes handle campaign sync, invoicing and bank checks in parallel, decoupled from user traffic.
Scheduling: a 14-command scheduler runs on 2-minute to daily intervals for sync, balance calculation, Vietinbank checks and invoice automation; Redis caches balance snapshots when available.
Data integrity: DB work-queue tables track incomplete syncs, API calls retry up to three times, and stale records are cleaned every 30 minutes.
Resilience: graceful degradation to sync/DB queues when Redis is down, batch Facebook endpoints to stay within rate limits, and OTP fallback on the banking API.
What made this hard
Architect's Challenges
- →Reconciling Facebook ad spend against Vietinbank debits across asynchronously updating systems with clock skew
- →Keeping financial state correct while many queue jobs mutate balances concurrently
- →Meeting Vietnamese e-invoicing regulation with MeInvoice and HSM signing inside async jobs
- →Isolating read-heavy denormalized Facebook data from transactional billing and banking without normalized joins
Results
Outcome
- ▹Async pipeline separates real-time from projected balances via T0/T1/T2 snapshots
- ▹Reconciliation matches cross-system transactions with timestamp tolerance
- ▹{{TODO: add real metrics — customers, campaigns synced, reconciliation accuracy, uptime}}