Seeding Tens of Millions of Coupons for Load Tests in Under Five Minutes

Before big campaigns, we load- and chaos-tested the coupon domain at Trendyol for peaks of 5–10x normal traffic. Those tests are only as good as their data. A nearly empty database behaves nothing like one holding tens of millions of coupons in an active-active data layer that spans two datacenters.

Preparing that data used to take around 30 manual requests and about an hour. I replaced them with a load-test data generator, built with Java and Spring Boot: a single parallel call generates tens of millions of coupons and cuts load- and chaos-test setup to under five minutes.

This post covers the approach, not our code: what a generator like this needs before you can trust it the night before a peak. The snippets are illustrative.

One request describes the dataset

A pile of manual requests encodes a dataset as a sequence someone has to remember. A declarative spec says what should exist and leaves the how to the generator:

{
  "runId": "peak-rehearsal-01",
  "environment": "load-test",
  "totalCoupons": 20000000,
  "batchSize": 5000,
  "maxInFlight": 32,
  "mix": [
    { "type": "PERCENTAGE", "share": 0.7 },
    { "type": "FIXED_AMOUNT", "share": 0.3 }
  ]
}

Shape matters as much as volume. If the mix of coupon types, validity windows and usage limits doesn't resemble production, the test exercises the wrong code paths and indexes. Store the spec with the run, so the same dataset can be produced again.

The call returns a run id right away instead of holding a connection open for minutes. That's what 202 Accepted is for: RFC 9110 defines it as accepted for processing but not completed, ideally pointing to a status monitor.

Fan out, with a ceiling

The spec becomes batches, and the batches run in parallel. But "in parallel" needs a number:

void generate(DatasetSpec spec) throws InterruptedException {
    long batches = (spec.totalCoupons() + spec.batchSize() - 1) / spec.batchSize();
    Semaphore inFlight = new Semaphore(spec.maxInFlight());

    try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
        for (long n = 0; n < batches; n++) {
            BatchId id = new BatchId(spec.runId(), n);
            if (ledger.isDone(id)) {
                continue; // finished by an earlier attempt
            }
            inFlight.acquire(); // waits while maxInFlight batches are running
            executor.submit(() -> {
                try {
                    writeBatch(spec, id);
                    ledger.markDone(id);
                } catch (Exception e) {
                    ledger.markFailed(id, e);
                } finally {
                    inFlight.release();
                }
            });
        }
    } // close() waits for every submitted batch
}

Taking the permit before submitting is the point. The loop can't run ahead of the work, so at most maxInFlight batches are in flight at any moment, whether the spec asks for one million coupons or fifty million. Virtual threads are too cheap to pool, and JEP 444 says so: when what you need is a concurrency limit, use a semaphore.

If the batches travel through Kafka instead, one message per batch, the partition count becomes the ceiling, because each partition is consumed by exactly one consumer in a group at a time. Kafka also delivers at least once by default, which brings up duplicates.

Idempotent batches

Batches fail and get retried, instances restart, messages arrive twice. None of that may create a duplicate coupon or skip one.

• Deterministic identity. Batch n of a run always has the same id, and the i-th coupon in it always gets the same code, derived from the run id, the batch number and the index. A retried batch writes the same keys, and an insert-if-absent write turns the second attempt into a no-op.
• A ledger. Every batch's status is recorded. Running the same spec again skips finished batches and redoes the rest, so a crash means resume, not start over.

Backpressure

The ceiling is fixed, but the target's capacity isn't. The environment being seeded is the one about to be load-tested, and the generator shouldn't be the first thing to knock it over.

Treat 429, 503 and timeouts as a request to slow down: retry with exponential backoff and jitter, and honor Retry-After. Retries need a limit too. After a few attempts the batch is marked failed and waits for the next pass.

Written isn't the same as ready. If the data layer replicates between datacenters asynchronously, report the run as done only when the data is visible on both sides. Otherwise the first minutes of the test hit a half-seeded datacenter.

Progress you can watch

The ledger doubles as the progress report: batches done, failed and in flight, coupons per second, time remaining. Anyone can check a run's progress without reading logs, and every failed batch shows its number and error. It helps chaos experiments too: after a failover drill, compare what each datacenter holds for the run with what the ledger says should exist.

Teardown is part of the feature

Everything the generator creates carries the run id, in the coupon codes and as a field. Cleanup then means deleting a run, batched, throttled and idempotent like generation. If the store supports expiry, set one longer than the test as a safety net, so forgotten data ages out instead of piling up.

Make production impossible, not unlikely

A coupon generator pointed at production is a discount generator. Layer the guards so that no single mistake is enough:

• The entry point only exists when @ConditionalOnProperty(name = "seeding.enabled", havingValue = "true") matches, and a missing property doesn't match. Production never sets it.
• The spec names its target environment. The service refuses unless that matches the environment it runs in and the environment is on an allowlist.
• Production credentials and network routes aren't available where it runs.
• Generated data is recognizable by its run id prefix, so anything that leaks is easy to find and remove.

Takeaways

• One declarative spec per dataset, stored with the run.
• Fan out under a hard ceiling: a semaphore, or the partition count.
• Deterministic ids and a batch ledger make retries, redeliveries and reruns safe.
• Back off when the target asks. Done means visible everywhere.
• Tear down by run id, and guard production in layers.