The data arrived. That wasn’t the test.

In April 2018, TSB moved its customers onto a new banking platform over a single weekend. When it switched back on, many customers couldn’t log in, some saw other people’s accounts, and the problems lasted for weeks.

In December 2022 the Financial Conduct Authority and the Prudential Regulation Authority fined TSB a combined £48.65 million. By then the bank had already paid £32.7 million to customers who had lost out.

What happened

The striking detail in the regulators’ account is this: the data itself was migrated successfully. The records arrived. It was the platform carrying them that failed.

An independent review commissioned by TSB’s board, carried out by the law firm Slaughter and May and published in 2019, described how that happened. The migration was done as a single “big bang”, without the board being shown a full range of alternatives. Testing covered too small a part of the platform. Some performance testing was done in the live environment. When internet banking missed its original load target, the target was lowered.

What was actually wrong

I’ve worked in testing for eight years, and the last of those findings is the one I’d underline. When a test fails and the answer is to change the test, the test has stopped measuring anything. It has become a box to tick on the way to a date that was fixed before the evidence came in.

Every migration has two separate questions:

1. Did the data move intact? Every record, every field, every link between them.
2. Does the thing it moved into work, with this data, at this scale?

It’s easy to answer the first, see a green tick, and feel the second has been answered too. TSB answered the first well. The second was answered with lowered targets and partial tests.

What a check would have shown

Most migrations are not banks, and most failures are not on this scale. But the same shape turns up in small projects all the time. The data is loaded, the counts match, and everyone signs off. Then the first week of real use shows that a status nobody mapped puts every record in the wrong queue, or that a report everyone relies on reads a field the migration left empty.

The checks that catch this are not exotic. Run the new system’s real processes on the migrated data before go-live: the reports, the automations, the screens people will actually use. Decide what “ready” means as numbers before the test runs, and write down who can change those numbers and why.

Before your own migration

• Separate the two questions. “The data arrived” and “the system works with this data” need separate evidence.
• Fix the bar before the test. If a target changes after a failure, record who changed it and why, where the people signing off will see it.
• Test with real data, at real volume, somewhere that isn’t live. A test environment holding 1% of the records proves the screens load. It doesn’t prove the business runs.

Sources: Financial Conduct Authority, “TSB fined £48.65m for operational resilience failings”, 20 December 2022; Bank of England (PRA) news release of the same date; Slaughter and May, independent review of the TSB migration, published November 2019.