Designing a Payment API That Can Safely Be Called Twice

Idempotency means an operation can be retried without causing duplicate or inconsistent data.

Having a proper idempotency key - a unique HTTP header value that lets the API know it has already seen this request - hangs between success and meaningless failure when a customer loses their connection, an overloaded server times out, or a flare of solar wind knocks out a data center.

But those idempotency keys, while a must, aren't actually a magic bullet. The vendor's idempotency rules act a lot more like a contract with hidden terms and renewal windows than like a one-size-fits-all best practice.

Why retries need a memory sends API requests, waits for the server to respond, then displays the request and logged results alongside each other in the same tab. was using to send a payment request to his company's checkout page. had blown up the round-trip delivery quotas.

Alex's phone app showed the payment had failed, but there was no duplicate charge. Fortunate for him – and for the payment network he was using – idempotency keys ensure that happened.

When Alex's phone attempted that retried payment, the payment gateway recognized the unique key the phone had sent with the request. The gateway recognized the key and recalled the original result.

Sure, the payment network appreciated it – it definitely didn't make any extra money off a duplicate charge. But there's more to the story: for the payment gateway, failing to recognize a duplicate charge could mean lost accuracy, suspicious activity flagged in the account, or even operational downtime.

When the API's idempotency logic works correctly, the developer avoids those major pain points.

How idempotency is implemented

Payment processors confirm idempotency keys ensure no duplicate charges by checking the developer documentation.

When a developer configures idempotency in the API client, the payment network suggests generating a random key, like a UUID. Checkout.com lets clients generate their own keys, but requires them to start with a "Cko-" prefix.

Either way, the key is stored alongside the original request and result. When another request to the same endpoint comes in, the API client sends the same key as a custom HTTP header. It looks like this in C#:

``` var request = CreateHttpRequestRequest((idempotentKey)); request.Headers["Request-Id"] = idempotentKey;

```

When the API server receives the request, it checks the headers for an idempotent key. If it finds one, it looks up the key. If it finds a match, the API server logs that a duplicate request was made. The server then returns a 200 response with the cached original result.

Amazon Pay's idempotency allows developers to accept the idempotency key, but still return the original response.

Adyen logs each API request and prevents duplicate charges when the same idempotency key is used.

In the real world, servers don't always save the results of expensive operations like authorization checks, calculated fees, or bulk data fetches. In those cases, an idempotent header tells the server to perform the expensive operation again. Say a lost network response led to a foregone payment in the client. The client sends the request again, the server successfully completes the payment, and that becomes the idempotent result stored for future reference.

The policy choices hidden in vendor docs

That's idempotency in a nutshell, but the details do differ. Platforms provide different default settings, and impose somewhat different restrictions.

Checkout.com recommends a 30-second wait between retried requests, probably to prevent the API server from getting hammered by repeated attempts in a short time.

Checkout.com lets the key expire after 24 hours, which is a small window compared to other providers. Amazon Pay stores keys indefinitely, while Adyen's limit is 7-14 days.

Amazon Pay and Checkout.com each store a separate key for each merchant account. Adyen, on the other hand, checks that incoming transaction keys are unique for the company, even without separate sandbox and live transaction keys.

What happens when the request is retried anyway

The policy differences exist for good reason. Those vendors see different upstream events, different back-office practices, and different regulatory requirements.

Amazon Pay returns a DuplicateIdempotencyKey error if the key provided doesn't match what the API was expecting, or if the parameters of the request don't match the original request.

Checkout.com says that if the original request returned a 4xx or 5xx response code, the payment will be reattempted. In other words, Checkout.com won't cache a known error as an idempotent result.

Adyen says that if the idempotency-key is missing in an idempotent request, the request will be processed as a new one.

The article’s missing second act:;

That's why idempotency isn't a bulletproof safeguard. It mostly prevents duplicate duplications, not all duplications. Let's say a network outage causes a payment to get stuck in the API server's processing queue. The client retries and the API responds with an idempotent match. Later, the network reconnects and the originally-in-flight payment finally processes as well. There's still a double charge, even though no single request duplicated.

The API developers communicate their idempotency contract in a public API docs page, and in internal payments code comments. The API team sets default timeouts, and troubleshoots underlying network issues and upstream errors. But they don't handle the post-payment practices: checking that the constitutive parts add up, paring out over-fetching, and subtracting out-of-band results. That's handled by the core payment operations team, the business intelligence integrators, the data analysts, and sometimes the fraud prevention pros.

The design implication for payment APIs

Idempotent keys store more than just a static value. The API's idempotency behavior needs to make sense for the app's job: different logic for store purchases, online subscriptions, peer-to-peer transfers, or bank-to-bank ACH payments. Different sellers or customers may even need different idempotent login, since they expect different wait times and fault behaviors.

And there's always room for the unforeseen hiccup: the retry that leads to failure, the duplicate that slots right into the gaps. That's here the tried and true best practices come into play: catching suspicious data in a single pass, ensuring the changed operations are explainable, and reconciling quirks back to a logical consistency over the long haul.