Translation Sync Engine — Reliable Async Processing for Your Localization Pipeline
A reliable, async translation pipeline that keeps your source code, translations, and CDN perfectly in sync — with conflict detection, activity logging, and zero data loss.
Why a Dedicated Sync Engine?
Translation workflows touch multiple systems — your Git repository, a translation database, a CDN for runtime delivery, and AI services for context analysis. Coordinating these systems in real time is fragile. A single API timeout can leave your translations in an inconsistent state.
Better i18n solves this with a dedicated sync engine built on Cloudflare Queues. Every operation — from importing keys to publishing translations — is processed as an async job with retries, ordering guarantees, and full observability.
The result: your translations stay consistent across every system, even when things go wrong.
Architecture Overview
The sync engine is a Cloudflare Queue consumer that processes structured messages. Each message represents a discrete unit of work — a sync to trigger, a file to upload, a batch to publish.
Messages flow through a well-defined lifecycle:
- Producer — Your action (push event, manual sync, publish) enqueues a message
- Queue — Cloudflare Queues provides durable, ordered delivery
- Consumer — The sync worker picks up the message and executes the job
- Activity Log — Every step is recorded as a sync activity for full traceability
This separation means your API responses are fast (the work happens in the background), and failures are retried automatically without user intervention.
10 Message Types
The sync engine handles 10 distinct message types, each responsible for a specific class of work:
| Message Type | Purpose |
|---|---|
| SYNC_START | Triggers a full or incremental GitHub sync — fetches files, compares keys, updates the database |
| REPO_PUSH_SYNC | Handles GitHub push webhook events — processes only changed files for fast incremental sync |
| CDN_SETUP | Creates the initial CDN manifest and empty language files when a project is first configured |
| CDN_UPLOAD | Uploads a single JSON translation file to R2 storage |
| CDN_MERGE | Merges new translation content into an existing CDN file without overwriting unchanged keys |
| CDN_CLEANUP | Removes all R2 files for a project (used during project deletion or reset) |
| AI_CONTEXT_ANALYSIS | Analyzes your website with Firecrawl + Gemini to build translation context |
| REPO_ANALYSIS | Scans your GitHub repo to detect frameworks, extract terminology, and build project context |
| PUBLISH_BATCH | Pushes approved translations to both CDN and GitHub in a single atomic operation |
| GLOSSARY_SYNC | Synchronizes terminology glossaries with DeepL for consistent machine translation |
Each message type has its own handler, validation, and error recovery logic. A failure in CDN upload never blocks a sync operation, and vice versa.
12 Job Types for Full Lifecycle Coverage
Jobs represent higher-level workflows that may produce multiple messages:
- initial_import — First-time key extraction from your repository
- incremental_sync — Sync only changed files since last run
- full_sync — Complete re-sync of all translation files
- source_sync — Sync source language changes only
- bulk_translate — Trigger machine translation for multiple languages
- publish / batch_publish — Push approved translations to production
- cdn_upload / cdn_merge / cdn_setup / cdn_cleanup — CDN lifecycle operations
- glossary_sync — Keep DeepL glossaries up to date
Job types give you granular control. You can trigger exactly the workflow you need — no unnecessary processing, no wasted compute.
45+ Activity Actions for Full Observability
Every sync job logs structured activity actions as it progresses. With 45+ distinct action types, you get a complete picture of what happened, when, and why.
A typical sync flow produces this activity trail:
SYNC_STARTED → FETCH_FILES → FILES_FETCHED → COMPARE_KEYS →
UPDATE_DATABASE → PR_GENERATION_STARTED → PR_CREATED → SYNC_COMPLETED
Activity actions are not just for debugging — they power the real-time sync status UI, letting your team see exactly where a sync is in its lifecycle. If something fails, the last recorded action tells you precisely what went wrong.
Conflict Detection and Resolution
When multiple sources modify the same translation key, conflicts are inevitable. The sync engine handles this with a dedicated conflict detection and resolution system.
How Conflicts Are Detected
During every sync, the engine compares incoming changes against the current database state. When a key has been modified in both the repository and the database since the last sync, it is flagged as a conflict.
How Conflicts Are Resolved
Conflicts are surfaced to your team with full context — the source value, the database value, and the timestamps of each change. You can resolve conflicts individually or in bulk:
- Keep source — Accept the repository version
- Keep database — Preserve the existing translation
- Manual merge — Edit the final value yourself
The engine never silently overwrites a translation. Every conflict is logged, tracked, and requires explicit resolution.
Reliability by Design
The sync engine is built for reliability at every layer:
- Durable message delivery — Cloudflare Queues guarantees at-least-once delivery. Messages survive worker restarts and infrastructure failures.
- Automatic retries — Failed jobs are retried with exponential backoff. Transient errors (API timeouts, rate limits) resolve themselves.
- Idempotent operations — Every message handler is designed to be safely re-executed. Retries never create duplicate data.
- Ordered processing — Messages within a project are processed in order, preventing race conditions between related operations.
- Activity logging — Every step is recorded, giving you a full audit trail for debugging and compliance.
Get Started
The sync engine works out of the box when you connect your GitHub repository to Better i18n. There is nothing to configure — syncs, CDN updates, and conflict detection are all handled automatically.
For teams that need more control, the sync engine exposes job-level APIs that let you trigger specific workflows, monitor progress, and resolve conflicts programmatically.