How We Built the Software that Processes Billions in Payments

Testing: testing is at the forefront of our development philosophy. We never need to check our code coverage to know that it's at 100%: with disciplined TDD, no line of code will be written without a test.

Bravo. In my experience that can be overkill, but with finance, I agree: why risk it. TDD everything.

We don't have a QA team.

WTF?

That might be terrifying when you consider the type of software that we're building, but we're confident that our automated testing is thorough and will catch any regression bugs.

Are regression bugs the only kind of bugs?

We use continuous integration to test every version of every client library against our gateway.

What happens when someone uses your client library in a way you didn't anticipate?

What happens when johnny-botnet hits your API directly without using the client library?

I spent several years developing games, with QA teams that outnumbered the developers. The QA team did not just play the levels through and say "it works!". Sure, they did that for the first hour. Then they'd start doing all those things that they thought someone might try (e.g. in a fit of boredom, or for a laugh). After that they'd just try breaking stuff. What a lot of bugs they would find!

As I write I find it hard to believe that I, a game developer, is having to explain the importance of QA to a financial company.

How do you guys maintain an ever growing code base?

It seems to me that the practices you have--TDD with full code coverage and regression tests, pair programming everything, many client libraries--would lead to a massive amount of code that needs to be maintained.

In my experience I've found that TDD often leads to a huge amount of code that needs to be rewritten if you do a major change to your codebase. If you have a new pattern you want to implement it may require rewriting dozens or more tests. Also, when I'm working alone I constantly go back and clean up functions and what not to make them shorter and clearer. I've found I do that less while pairing because you want to keep moving forward.

My opinion is probably just biased by the fact that I've done relatively little pair programming and TDD. I'm sure the more you do the better you get.

But I'm curious if you could take some about the size of your code base and how these practices effect that?

...we are able to perform all our maintenance without downtime. We can deploy new versions of our software, make database schema changes, or even rotate our primary database server, all without failing to respond to a single request. We can accomplish this because we gave ourselves the ability suspend our traffic. To make this happen, we built a custom HTTP server and application dispatching infrastructure around Python's Tornado and Redis.

Why is it necessary to suspend traffic to make these kinds of changes? Just curious.

And that is how you write a great job advert.

More specifically, it gets across:

- the company culture

- the development practices/methodologies in place

- what's exciting about the work/company

- what technologies they like (and the fact they use a variety of technologies)

- the fact that they get that typical job adverts suck

... all while looking like a regular article.

Great work.

Different strokes for different folks i guess. I know a lot of people will be fans of this, but

we pair program to write all of our software. We work on Mac Pros with two keyboards and two monitors. We work in an open team room; no cubicles or private offices. 

No thanks, if I'm the developer. And if i wouldn't do it, why would i make my employees.