Hello, I'm Maneshwar, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.
Checkout calls the payment service on every order. One call, one arrow on the architecture diagram, the least interesting line in the codebase.
Take the payment service completely down. Every call to it is refused in about three milliseconds. Checkout returns a clean failure for that order, and every other page on the site keeps serving.
Now leave it running and answering every call correctly, only slower. 40 milliseconds becomes 30 seconds.
Checkout has nothing to react to, because from its point of view nothing failed.
On paper the second case is the healthier one. Every call is answered, correctly, with the right data.
The reason is not subtle once you see it. Your service holds a worker for the entire length of every call it makes.
When the dependency is down, that worker comes back in milliseconds and moves straight on to the next request.
A slow dependency keeps the same worker for 30 seconds, and you only have so many of them.
So this whole article is about a pattern that protects you, not the service you are calling.
Every call your service makes takes a small pile of things with it while it runs.
It holds all of that until the call finishes, one way or the other. A slow call just holds it for longer.
You have a fixed number of each, and nothing new can start once they are all in use.
That is why the product page died in the opening scene, even though the product page never calls payment. The blocked requests were sitting on threads and connections that the rest of your system shares.
One slow dependency can saturate every resource on every one of your services in seconds.
And this is not a rare situation, because dependencies multiply. Say your service calls 30 of them, and each one is up 99.99% of the time. You need all 30.
That is 3 million failed requests in every billion, and over two hours of downtime a month, with all 30 dependencies behaving exactly as advertised.
The fix is to fail fast. Turn down a call you already expect to fail, instead of waiting for it to time out.