Fan-Out Without Falling Over: Timeouts and Partial Responses in NestJS

A gateway (or BFF) that builds a screen usually calls several services at once: one for the main entity, a few more for everything around it. The first version is often await Promise.all([...]) with default HTTP client settings. That makes the screen as slow as the slowest dependency, and it fails completely if any dependency fails.

Working on a gateway, I've found it comes down to three decisions per downstream call: how long to wait, whether the screen can live without the result, and how you notice when it's missing.

Give every call a real deadline

The obvious timeouts fall short in two ways. First, axios implements its timeout option in Node with req.setTimeout, which is a socket inactivity timer: a downstream that keeps trickling bytes never trips it, the same gap a read timeout has in Java. Second, in @nestjs/axios 2.x, unsubscribing from the HttpService observable (which is what RxJS timeout() does) doesn't cancel the request. The cancel token it creates never reaches axios, so the call keeps running in the background.

An AbortSignal in the request config fixes both. It's a deadline for the whole call, and it really aborts the request:

import { HttpService } from '@nestjs/axios'
import { Injectable, Logger } from '@nestjs/common'
import { firstValueFrom } from 'rxjs'

@Injectable()
export class DownstreamClient {
  private readonly logger = new Logger(DownstreamClient.name)

  constructor(private readonly http: HttpService) {}

  async get<T>(name: string, url: string, deadlineMs: number): Promise<T> {
    const controller = new AbortController()
    const timer = setTimeout(() => controller.abort(), deadlineMs)
    try {
      const { data } = await firstValueFrom(this.http.get<T>(url, { signal: controller.signal }))
      return data
    } catch (err) {
      const reason = controller.signal.aborted ? `no answer within ${deadlineMs}ms` : (err as Error).message
      this.logger.warn(`${name}: ${reason}`)
      throw err
    } finally {
      clearTimeout(timer)
    }
  }
}

Settle everything, then decide

The page service starts all three calls at once and only looks at the results once every call has either answered or hit its deadline:

import { BadGatewayException, Injectable } from '@nestjs/common'
import { DownstreamClient } from './downstream.client'
import { Product, ProductPage, ReviewSummary } from './product-page.types'

@Injectable()
export class ProductPageService {
  constructor(private readonly downstream: DownstreamClient) {}

  async build(id: string): Promise<ProductPage> {
    const [product, reviews, similar] = await Promise.allSettled([
      this.downstream.get<Product>('catalog', `http://catalog/products/${id}`, 800),
      this.downstream.get<ReviewSummary>('reviews', `http://reviews/products/${id}/summary`, 300),
      this.downstream.get<Product[]>('recommendations', `http://recommendations/products/${id}/similar`, 300),
    ])

    if (product.status === 'rejected') {
      throw new BadGatewayException('catalog unavailable')
    }

    const degraded: string[] = []
    const optional = <T>(section: string, result: PromiseSettledResult<T>): T | null => {
      if (result.status === 'fulfilled') return result.value
      degraded.push(section)
      return null
    }

    return {
      product: product.value,
      reviews: optional('reviews', reviews),
      similar: optional('recommendations', similar) ?? [],
      degraded,
    }
  }
}

Promise.allSettled never rejects, and every call has its own deadline, so this fan-out takes about 800 ms at most, whatever the downstreams do. Promise.all would reject on the first failure: the other calls keep running anyway, but their results are thrown away.

The product is required. Without it there is no page, so that failure becomes a 502 (a catalog 404 should become a 404, of course). Reviews and recommendations are optional: they fall back to null or an empty list, and their names go into degraded so the client can render a placeholder.

The deadlines come from a budget: parallel calls must each fit inside the screen's latency target, and calls that depend on each other have to share it. All of it has to finish well before the app's own request timeout. Otherwise the app gives up and retries, and the gateway does the whole fan-out twice.

Make degradation visible

A degraded response is still a 200, so it never shows up on an error-rate graph. A global interceptor, registered with app.useGlobalInterceptors(new TimingInterceptor()), logs the duration, status and degraded sections of every request:

import { CallHandler, ExecutionContext, HttpException, Injectable, Logger, NestInterceptor } from '@nestjs/common'
import { Observable, tap } from 'rxjs'

@Injectable()
export class TimingInterceptor implements NestInterceptor {
  private readonly logger = new Logger('HTTP')

  intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
    const http = context.switchToHttp()
    const { method, url } = http.getRequest()
    const started = Date.now()
    const log = (status: number, degraded: string[] = []) =>
      this.logger.log(`${method} ${url} ${status} ${Date.now() - started}ms degraded=[${degraded.join(',')}]`)

    return next.handle().pipe(
      tap({
        next: (body) => log(http.getResponse().statusCode, body?.degraded),
        // exception filters run after interceptors, so take the status from the error
        error: (err) => log(err instanceof HttpException ? err.getStatus() : 500),
      }),
    )
  }
}

A counter per degraded section is even better than a log line, because you can alert when recommendations quietly disappear for everyone. And if the response is cacheable, skip the cache for degraded ones, or a thirty-second hiccup can turn into a partial page served for an hour.

Checklist

• Every downstream call has its own deadline, and the deadlines fit the screen's latency budget.
• A deadline aborts the request instead of just no longer waiting for it.
• Each section is explicitly required or optional, and only required ones can fail the response.
• Fan out with Promise.allSettled and decide afterwards.
• Degraded responses are marked, counted and never cached.
• Retries, if any: idempotent reads only, at most once, and only when the remaining budget allows.