Verification Effort

Verification has a place in the software scheme of things.

That is undeniable.

But the effort one needs to put into verification is not emphasised enough.

The problem with verification is that if an error goes undetected in one of the phases of the software development process, the chances are that the error will go undetected in each of the subsequent phases till it is very late. The earlier an error slips past, the later it can be detected.

Here's an example:

Supposed there is a mismatch between the SRD Requirements and the SDD Requirement. Let's assume one bit of some transmission protocol has been interchanged. And the SDD developer and the SDD reviewer have missed it. You think it can be caught in the subsequent phases? Very unlikely. Here's why?

Verification in a given phase, unlike Validation, focuses on correctness and completeness with respect to inputs of that phase. And it goes like this:

SRD ----> SDD (error exists in SRD; should have been detected but missed)

SDD ----> Code (no error since SDD and Code match perfectly)

SDD ----> Low-level testing (no error since SDD and Code match perfectly)

SRD ----> High-Level testing (should find the error here - hopefully; since SRD and Code will not match)

And if the SRD Requirement is not well written (perhaps that's why the SDD review against SRD missed out the error in the first place!), the High-Level testing will miss the error too. It would eventually be found at the System Validation phase.

I have often heard managers say that such and such phase is low risk because there are many more tests down the line that will catch the error. This is exactly this kind of mentality that causes maximum damage.

The age old mantra of perfection applies here too: anything worth doing is worth doing well. Each verification phase exists for a reason. Better allocate sufficient time and effort to catch errors as early as possible.

Comments

[bikramdas]

We show how the effort needed to verify a transformed base model can be reduced by analyzing the definition of the modification. The Common Variability Language (CVL) is a generic language for modeling variability, where a CVL model describes the increment from one base model to another (transformed) base model. Assuming that a property of the base model has been verified, we use the CVL model to reduce the effort needed to verify the property of the transformed model. Based on the CVL model, we narrow down the set of traces required to be verified, including the increment and the cascading effects. We apply CVL to several models of the Train Control Language (TCL) to illustrate how the effort of verifying safety properties of transformed train station models can be reduced.We show how the effort needed to verify a transformed base model can be reduced by analyzing the definition of the modification. The Common Variability Language (CVL) is a generic language for modeling variability, where a CVL model describes the increment from one base model to another (transformed) base model. Assuming that a property of the base model has been verified, we use the CVL model to reduce the effort needed to verify the property of the transformed model. Based on the CVL model, we narrow down the set of traces required to be verified, including the increment and the cascading effects. We apply CVL to several models of the Train Control Language (TCL) to illustrate how the effort of verifying safety properties of transformed train station models can be reduced.We show how the effort needed to verify a transformed base model can be reduced by analyzing the definition of the modification. The Common Variability Language (CVL) is a generic language for modeling variability, where a CVL model describes the increment from one base model to another (transformed) base model. Assuming that a property of the base model has been verified, we use the CVL model to reduce the effort needed to verify the property of the transformed model. Based on the CVL model, we narrow down the set of traces required to be verified, including the increment and the cascading effects. We apply CVL to several models of the Train Control Language (TCL) to illustrate how the effort of verifying safety properties of transformed train station models can be reduced.We show how the effort needed to verify a transformed base model can be reduced by analyzing the definition of the modification. The Common Variability Language (CVL) is a generic language for modeling variability, where a CVL model describes the increment from one base model to another (transformed) base model. Assuming that a property of the base model has been verified, we use the CVL model to reduce the effort needed to verify the property of the transformed model. Based on the CVL model, we narrow down the set of traces required to be verified, including the increment and the cascading effects. We apply CVL to several models of the Train Control Language (TCL) to illustrate how the effort of verifying safety properties of transformed train station models can be reduced.
more information