Buying a tool is buying its failure modes

Checked 22 Sep 2026 · By Luke Czak

ArticleOpinionFree to read

Trying a tool on its happy path tells you almost nothing useful. What you are actually adopting is how it behaves when the thing it depends on is down.

When I evaluate a new tool before wiring it into my pipeline, the natural instinct is to try the thing it is supposed to do and see whether it does it. Does the credential store return the credential. Does the queue deliver the message. Does the deploy command actually deploy the thing it was pointed at. Almost every tool I have ever looked at passes this test without difficulty, because the happy path is the path its own documentation walks you through step by step, and a clean working example is the easiest thing in the world for whoever built the tool to produce and show off.

The question that actually matters is a different one, and it is much harder to answer by trying the tool for an afternoon: what does it do when the thing underneath it is unavailable. Does it fail loudly, with an error that names the actual problem, or does it fail quietly and return something that looks like a valid answer but is not one at all. I adopted a credential tool once that behaved perfectly in every manual test I ran, sitting at my own machine with my own session already logged in and everything working exactly as advertised, and then failed silently the moment I tried to use it from an automated context with no session behind it, returning a response that read exactly like "this credential does not exist" when the actual situation was "I cannot currently check."

I spent real time chasing the wrong problem because of that one failure mode. I believed the credential was genuinely gone, because that is what the tool told me, clearly and without qualification, in language that gave me no particular reason to doubt it. The tool was not lying exactly. It was reporting the only thing it knew how to report given the context it found itself running in, which happened to be indistinguishable, from the outside, from the thing it actually meant to report when a credential was truly missing rather than merely unreachable. That distinction was a property of the tool, not of the world, and I had adopted the tool without ever finding out the distinction was sitting there waiting to be tripped over.

I do not think this is fully solvable before you commit to something, and I do not want to pretend it is. Reading a tool’s source line by line before every adoption is not realistic on any normal schedule, and most failure modes only reveal themselves under conditions you cannot fully simulate in an afternoon of testing: a specific kind of outage, a specific kind of missing context, a specific race that only shows up once something has been running for a while under real load rather than under a demo. What changed for me is not eliminating that risk, because I cannot, it is admitting the risk is there rather than quietly assuming a tool that works cleanly in a demo will also fail cleanly later, which was the actual habit doing the damage all along.

So the question I actually ask now, before adopting anything that sits between me and a decision I care about, is not "does this do the thing" but "what does this do when it cannot." If the honest answer is I do not know, that is not automatically a reason to avoid the tool, but it is a debt I have knowingly taken on rather than one that surprises me later wearing the shape of a mystery I have to solve under pressure. The tools that have actually hurt me were never the ones with a bad happy path, because a bad happy path gets noticed and fixed or replaced almost immediately. They were the ones whose failure mode I never bothered to ask about until it had already happened to me, at the worst possible time, exactly like these things tend to do.

Comments (0)

Sign in to comment.