finalBackendDevConnect

DevConnect Background Import Service

An authenticated service that accepts a batch of records, acknowledges it immediately, and processes it in a background worker.

Run locally

Requirements: Node.js 20 or newer.

npm install
AUTH_TOKENS="alice=local-alice-token,bob=local-bob-token" npm start

On Windows PowerShell:

$env:AUTH_TOKENS = "alice=local-alice-token,bob=local-bob-token"
npm start

The default port is 3000. GET /health is public. All import routes require Authorization: Bearer <token>.

API

Create an import. The 202 response is returned before the worker processes the records:

curl -i -X POST http://localhost:3000/imports \
  -H "Authorization: Bearer local-alice-token" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: import-001" \
  -d '{"records":[{"email":"ada@example.com"},{"email":"grace@example.com"}]}'

Poll the returned statusUrl:

curl http://localhost:3000/imports/<id> \
  -H "Authorization: Bearer local-alice-token"

Completed work returns importedCount and the normalized records. Repeating the same request with the same idempotency key for the same account returns the original job rather than creating another import. Idempotency keys are scoped to an account. A different account cannot read the job and receives 404.

Errors are JSON with an actionable error.code and error.message. Missing or invalid credentials return 401; malformed input returns 400; an account-scoped missing import returns 404.

Worker and failure behavior

The worker polls the durable JSON job store every 250 ms. Claiming a job changes it to processing, increments attempts, and writes a 30-second lease. If the process dies after claiming a job, the lease eventually expires and a later worker claims it again. Processing is deterministic and the only outcome write is guarded by the job’s processing state, so running the same job twice produces the same result rather than duplicate records. A processing exception is recorded as failed with an error code and message visible through the status endpoint.

For a multi-instance production deployment, use a shared transactional database and a queue with the same lease/idempotency rules. The included Render configuration uses a persistent disk for this single-instance demonstration.

Deploy

  1. Push this repository to GitHub.
  2. In Render, choose New > Blueprint and select the repository. Render reads render.yaml.
  3. Set the AUTH_TOKENS secret in the Render dashboard, for example alice=<long-random-token>,bob=<another-long-random-token>. Do not commit this value.
  4. After deploy, use the generated https://...onrender.com URL as the public service URL and link it with this repository when submitting.

The repository contains no production credentials. .env and the local data directory are ignored by Git.