The most expensive moment to learn the truth
Integrations rarely fail because engineers can’t write code. They fail because the truth about a vendor API arrives at the worst possible time: mid-build, after the contract is signed, after the roadmap has been promised, when the sandbox finally returns something the documentation never mentioned. At that point every surprise is rework, and every workaround becomes permanent architecture.
It’s not that vendors lie. It’s that API documentation usually gets written by people who have never had to integrate against their own product on a deadline. The docs describe how it’s meant to work. Your team is building against how it does work, and the space between those two things is where the budget goes.
The usual advice is to test before you sign. In practice that’s often not on the table: plenty of vendors don’t hand over sandbox access until there’s a contract, and by the time access arrives the commercial decision has been made. What hasn’t been made yet is the technical one: what your developers build on, and how.
So at the payments platform I ran solutions for, once access came through, no vendor API reached the development team until someone had actually used it. Not just read the docs — made real calls against the real sandbox, on purpose, looking for the parts that don’t hold up. That was my job.
What the gate actually tests
1. Auth, first and hardest
Auth is where the gap between the pitch and the product shows up first. Does the model in the docs match what the sandbox will actually accept? How long do tokens really last, and what happens on refresh or revocation? Can credentials be scoped and rotated? If I can’t get auth working cleanly and repeatably in the first sitting, I’ve already learned something about the rest of it.
2. The unhappy paths
Any API can demo the happy path. What I want to know is what it does when things go wrong, because production is mostly small things going wrong. Can you tell one failure from another, or is everything a 500 with an essay in the body? Is anything idempotent, or will a timeout charge someone twice? Are the rate limits documented, real, and findable before you hit them at 2am?
3. Documentation against behavior
I read the docs the way an implementing engineer will, then test whether they’re telling the truth. Every field the docs call required, every enum value, every pagination promise gets checked against a live response. In payments this matters doubly: minor units, currency handling, and settlement timing are exactly the places where a doc that’s “mostly right” produces a ledger that’s exactly wrong.
4. Whether the sandbox is telling the truth
A sandbox that doesn’t behave like production is worse than not having one, because it hands you confidence you haven’t earned. Does it enforce the same validation? Can you make it fail on purpose, or does everything come back 200? If it can’t produce the edge cases you’ll hit in the real world, your first declined payment happens with a customer attached to it.
5. The vendor, not just the API
Do webhooks actually arrive, in order, and get retried? Is there a status page with history behind it, or just a green dot? And the testing will raise real questions for the vendor on its own. How long a competent answer takes, often across time zones, is the clearest preview you’ll get of the support you’ll have once you’re live.
Quick testing, slow answers
The testing itself was the quick part: focused, precise, and done fast. What took time was everything around it. Each issue meant a question to the vendor, often across time zones, and a wait for an answer, while I worked through whatever else the API had turned up. That wait is a finding in its own right. A vendor that takes days to answer a clear, reproducible question during an evaluation rarely gets faster once your customers are live on it.
What comes out of it
The testing is the easy half. With the contract already signed, walking away is rarely the outcome. What changes is what the developers receive: what actually works, where the documentation is wrong, which assumptions are unsafe, and what to build defensively around, written for the people who have to build the thing. Not a summary of the vendor’s docs. A record of what we checked ourselves versus took on trust, and the open questions still with the vendor.
That last bit does most of the work. Every claim in the handoff says how we know it. Engineers stop rediscovering the same surprise one incident at a time, and nobody promises a customer something built on a capability we never watched work.
Why this is a foundation fix
In Foundation Layer terms, integration pain looks like an L4 problem — delivery late, quality wobbling — but it comes from L2, where someone took on a dependency nobody had checked. The gate doesn’t make the surprises go away. It moves them to the point where they’re still cheap: before your developers have built on them, and before any of it becomes permanent architecture.
Signing commits the budget. Building on it unchecked commits everything else.
I run this evaluation as a standalone engagement — one vendor API in, a tested recommendation and build-ready documentation out. Here’s how to start. The same discipline, systematized, is what became UAT-Ops.