A disaster recovery test can pass and still leave the business's ability to recover unproven.
The contradiction disappears when we look at what the test actually asked. A server restarted. An application opened. A team followed the runbook. Those results may all be accurate. The trouble begins when they are reported, or understood, as proof that the business itself has recovered.
One case makes the gap concrete. A critical application passed its recovery test: the server came back, the application started, and the evidence was collected. A subsequent incident exposed unusable backups. The earlier exercise had shown that the system could start. It had not shown that usable data could be restored.
The test passed. The capability the business depended on remained unproven.
The test may be telling the truth
The easy conclusion is that the passing result was false. It may not have been.
If the exercise was designed to validate server restart and the server restarted, it passed. If it was designed to validate application startup and the application started, it passed. The technical result can be sound while the conclusion drawn from it is not.
The gap opens when the meaning of the result expands beyond the design of the test. An exercise validates one component. Its report records success. Later, that success becomes evidence in a decision about the whole service, even though the necessary restoration, dependencies, or business validation were never examined.
The scope of the evidence has stayed narrow. The scope of the assurance has grown.
That is a governance failure even when the technical team did exactly what it was asked. The organization has done something real and collected evidence of it. It has then asked that evidence to support a conclusion it cannot carry.
Restart is one condition. Recovery requires others.
An application opening does not prove that it is safe to use. A database existing does not prove that its contents are complete. A user reaching a login screen does not prove that the service behind it works.
Recovery means resuming the business service in a known, usable, controlled condition. What that requires depends on the service. It may include restored data, transaction integrity, user access, downstream connectivity, functioning controls, reconciliation, business validation, and explicit acceptance of residual risks.
For a financial service, the distinction becomes immediate. Account data may be incomplete even though the application is available. Transactions may be entered while downstream settlement remains uncertain. Users may have access while a control is not operating normally.
Each condition changes what the business can responsibly do. Availability is useful evidence, but it does not answer every question about safe resumption.
NIST's Contingency Planning Guide for Federal Information Systems draws the same line. Its recovery sequence moves through data validation testing and functional validation testing before the system is declared recovered. It is federal guidance, not a banking mandate, but the logic carries.
The condition that counts as recovered should be defined before the incident. Otherwise, the organization may discover what it meant by recovery only after the system comes back.
Ask the restoration question
Every test is a set of questions. Its result can answer only the questions its design and execution actually examined.
Can the application start? Can the data be restored? Can the business validate the recovered state? Can users operate, controls function, and downstream processes continue? Can transactions be reconciled across the outage?
These are related questions. They are not interchangeable.
An exercise that brings up the application and completes the runbook may produce real evidence. But if it never restores usable data, it has said nothing about restoration. If it never involves the business, it has said nothing about whether the business can use what came back.
“Did anyone restore the data?” should therefore be part of the design wherever recovery depends on that capability. It should not arrive as a surprised question after the evidence package is complete. In the case above, the answer arrived with the incident.
“As scoped” should make the limits visible
No exercise can test everything. Time, cost, environment constraints, data sensitivity, business participation, and the risk of disruption all affect scope.
The phrase “as scoped” acknowledges those boundaries. On its own, it protects the report more than it informs the reader. It earns its place when the report says what the scope left out.
Compare these two statements:
The recovery test passed.
Infrastructure restoration and application startup passed. Data restoration and business validation were not tested.
The second statement lets a leader understand both the achievement and the remaining uncertainty. It prevents success in one part of recovery from being read as assurance about the whole.
The same discipline applies to exclusions elsewhere. If downstream interfaces were not tested, end-to-end recovery is still an open question. If reconciliation was excluded, so is post-recovery transaction integrity.
The result may still be valuable. Its value needs to be stated precisely.
Incomplete testing becomes dangerous when it is presented as complete assurance.
Evidence should follow the capability
Evidence collection is necessary. Screenshots, timestamps, activity records, and signoffs help show what happened and allow the result to be reviewed.
But those artifacts can become the practical center of the exercise. A team may complete the required steps, capture the expected records, and produce a well-organized package without examining whether the recovered service is usable.
The package then proves that the exercise happened. It may say much less about the capability leaders are relying on.
Scripted exercises and rehearsals have legitimate purposes. They can familiarize staff with procedures, test defined components, and expose problems. It goes wrong when a predictable exercise repeatedly confirms a narrow set of conditions while leaving the decisive recovery assumptions untouched.
FFIEC's exercise and test objectives include the recovery of business processes and critical system components. The relevant question is whether the exercise addresses the recovery capability being claimed—not whether it produces the expected artifacts.
Challenge the assumptions behind the plan
A good recovery test examines the assumptions that make the plan credible.
If the plan depends on restoring data, restore and validate it. If it depends on the business operating from a recovered environment, involve the business. If it depends on downstream services, examine those dependencies. If it depends on manual workarounds, test their capacity, timing, reconciliation, authority, and risk acceptance.
The time target needs the same treatment. Suppose the plan promises recovery in four hours.
Four hours to what?
Server startup, application availability, data restoration, and safe business-service resumption describe different conditions. A reported elapsed time means little unless the report says what condition was reached when the clock stopped.
Fast recovery into an unusable state is not resilience.
The service is what the business needs back
Recovery testing often starts with components because they are tangible: servers, databases, applications, networks, authentication, and interfaces.
Those components matter. But the business depends on the service they support—a payment, claim, settlement, customer-account inquiry, payroll run, or report.
A recovered application can still leave that service broken if a necessary interface is unavailable. A technically functioning service can still be unsafe if its data cannot be trusted. Manual controls may be required before normal processing can resume.
The exercise therefore needs a clear relationship between the components tested and the business capability claimed. Component tests can contribute evidence to that assessment. Passing components, one by one, do not prove that they work together.
Technology recovery is necessary. Business-service recovery is the point.
Report the result leaders can act on
A recovery report should answer two questions: what capability did the exercise demonstrate, and what material capability did it leave unproven?
An actionable report might read:
Demonstrated: infrastructure restoration and application startup within the target window. Not tested: data restoration, transaction reconciliation, downstream interfaces, and business-service resumption. Decision required: schedule a restoration test, or formally accept the exposure until one is run.
That is an honest result. It gives leaders a basis for deciding whether further testing is needed and whether the remaining exposure is acceptable.
FFIEC's post-test guidance calls for results to be assessed against the objectives and success criteria set out in the exercise and test plans, and for identified issues to be documented. Where an item has not been remediated, management should record its decision to accept the risk identified during the exercise. Its guidance supports disciplined reporting and follow-through; the broader decision-integrity argument here is the author's analysis.
Recovery testing helps an organization decide what it believes about its own recoverability. If its evidence is narrow but its confidence is broad, leaders can make funding, recovery-strategy, vendor, or customer-commitment decisions on a capability no test has shown.
The test's value lies in improving that understanding. A green result is worth something only when the people acting on it know what it means.
The purpose is to find out
The purpose of recovery testing is not to pass. It is to find out whether the plan, data, controls, dependencies, and business operations support the recovered condition the organization needs.
A test designed mainly to pass may avoid the questions that could expose a weakness. A test designed to find out brings those questions forward while there is still time to act on the answers.
Every exercise should be honest about what it did and did not prove.
When an incident comes, the passing result will not recover the business. Only the capability behind it can. If no one tested that capability, the story ends where this one began:
The test passed. The capability the business depended on remained unproven.
