CSC 509 Lecture Notes Week 10

CSC 509 Lecture Notes Week 10
Introduction to Formal Program Verification

Two introductory definitions
1. Testing: show that a program is correct for some (finite) set of inputs.
2. Verification: prove that a program is correct for all possible inputs.
Review of the problem with testing
1. For a large system, testing cannot cover all possible cases.
2. Hence we can never be 100% sure that a system is correct.
3. For some systems, such as safety critical applications, this is simply not good enough
4. Enter program verification, the goal of which is to demonstrate in some convincing way that a system works in general, for all possible inputs.
5. To work our way up to formal verification, we'll start with a look at an abstract form of testing, called symbolic evaluation.
An example for comparing testing to symbolic evaluation and verification
1. Here's a very simple example function that we'll use to study symbolic evaluation and formal verification
```
/*
 * Compute factorial of x, for positive x, using an iterative technique.
 *
 * Precond: x >= 0
 *
 * Postcond: return == x!
 *
 */
int Factorial(int x) {
    int y;
    y = 1;
    while (x > 0) {
        x = x - 1;
        y = y * x;
    }
    return y;
}
```
2. Question: Is this correct? (Answer: No, since the two statements in the loop are in the wrong order.)

Symbolic evaluation

So far in the testing schemes that we've considered, all test inputs and outputs have been concrete values, as opposed to symbolic values.
To see what we mean by this, let's consider how we would test the factorial function with concrete values.

Table 1 shows a typical unit test plan for function Factorial.

Test No.	Input	Expected
		Results
1	x = -1	ERROR (precondition violation)
2	x = 0	return = 1
3	x = 1	return = 1
4	x = 4	return = 24
5	x = 6	return = 120
6	x = 70	return > 10**100 (possible numeric overflow)

Table 1: Unit testing Factorial.

To test, we feed these concrete values in and check out the results.
Two important questions even for this very simple function:
1. How do we know what the expected results are supposed to be? (Ans: an oracle.)
2. How do we know that it works for all inputs? (Ans: we'll have to prove it.)
One way to help answer these questions is to consider supplying a single symbolic value for the input, and analyzing a resulting symbolic output.
Here are the details (assuming that we've corrected bug by changing the order of the two loop statements):
1. For every expression that involves some input variable, we compute a symbolic result rather than a concrete result.
2. E.g., the symbolic computation of the statements:
```
y := 1;
y := y * x;
```
  results in y having the symbolic value of 1 * x which simplifies to just x.
3. Suppose we do some more symbolic computation:
```
x := x - 1;
y := y * x;
```
4. This results in a symbolic value of x * (x - 1) for y.
5. A bit more symbolic computation:
```
x := x - 1;
y := y * x;
```
  results in y's symbolic value of x * (x - 1) * ((x - 1) - 1) which simplifies to x * (x - 1) * (x - 2)
6. The idea is that we treat input values as symbols inside expressions rather than as concrete values
What a symbolic evaluator would do for us in the factorial example is produce symbolic output results such as the following (on the correct version of the function): y = 1 y = 1 * x y = x * (x-1) y = x * (x-1) * (x-1-1) y = x * (x-1) * (x-2) * ((x-2)-1) . . after N times through the Factorial loop symbolically . y = x * (x-1) * ... * (x-N)
What's nice about this is that we don't need to worry about any concrete values at all, but we see an informative symbolic pattern developing that tells us by inspection if the function appears to be working.
It's also interesting to look at the erroneous case where the loop statements have been transposed (i.e., the way the function is defined originally above): y = 1 y = 1 * (x-1) y = (x-1) * ((x-1)-1) y = (x-1) * (x-2) * (x-3) . . . y = (x-1) * (x-2) * ... * (x-1-N)
Here the error manifests itself in a symbolic expression that's clearly wrong for computing factorial (i.e., it's off by an increment of 1).
We could do this kind of evaluation by hand, but it's much nicer if we have an augmented form of interpreter to do it for us. (Such symbolic interpreters in fact exist, and they aren't even that tough to build.) operations.)

Moving on to formal verification
1. While symbolic evaluation is more general than concrete testing, it still involves some informal human analysis to verify that the output of a program is correct.
2. What we want is a statement of mathematical certitude that a program is correct for all inputs -- i.e., we want to mathematically prove program correctness.
3. That is, we seek to prove that a program meets its specification fully.
4. The general steps to perform formal program verification are the following:
  1. We will need to make some formal connection between a program written in a programming language and the language of formal mathematics.
    1. We do this by defining mathematical meaning for a particular programming language.
    2. Specifically, the meaning is given by a set of rules for how each statement in the language behaves mathematically.
    3. For example, we can define an assignment statement in terms of mathematical variable substitution; a programming language if-then-else is defined pretty directly as mathematical if-then-else.
    4. In these notes, we will give a few such rules, and see in general what it takes to define programming language formally.
    5. We call such a set of rules a mathematical (or sometimes axiomatic) semantics for a programming language.
  2. After we define the mathematical language rules, we need a general procedure that tells us how to assign meaning to a particular program.
    1. This procedure is rather like symbolic evaluation, in that we walk through a program producing symbolic rather than concrete results.
    2. Coming up, we will look at specific verification procedure known as ``backwards substitution''
5. Given the language rules and verification procedure, we need to state formal pre and post conditions for any function that we want to verify formally.
  1. These are precisely the kind of pre/post conds that we did for the specs.
  2. As it turns out, we'll need just a bit more than function pre/post conds -- we also need preconds for each loop within a function.
  3. These loop preconds are called loop (or inductive) assertions, and they can be generated with the help of a symbolic evaluator.
6. After the pre, post and inductive assertions have established, we apply our rule-based verification procedure to the desired function, the result being the mathematical verification of the proposition:
```
    precond  postcond
```
  i.e., the precond mathematically implies the post cond when the mathematical rules of the function are obeyed.
7. To make it clear that the rules of the function body must be obeyed in order for the implication to be true, program verifiers have invented a new notation
```
    precond {function body} postcond
```
  which means that if the precond is true before the function body is (mathematically) executed, then the postcond must be true after the execution is complete.
8. This notational form is called a "Hoare triple", named after C.A.R. Hoare, one of the acknowledged founders of formal verification, among other computer science principles.
9. A final step in many formal verifications is to prove the termination condition, that says under which conditions the function will actual finish its computation; more on this later.
10. We will look at a set of verification rules for a Pascal-class language, and an actual verification of the Factorial example.
Simple Flowchart Programs (SFPs)
1. When we looked at path testing, it was convenient to think of programs in graphical flowchart form.
2. Flowcharts are also a helpful representation for understanding formal verification.
  1. There's nothing particularly special about the flowchart representation of programs.
  2. It's just a graphical view of programs that's isomorphic to a textual version written in some textual programming language.
  3. For our intro to formal verification, we'll use a notation that covers just three of the most basic program constructs.
  4. With some additional work (that we don't do here), these simple basic constructs can be generalized to cover all of the constructs of a high-level programming language.
3. The basic constructs are:
  1. an assignment statement,
  2. an if-then-else statement
  3. a top-of-loop node that is used in conjunction with an if-then-else to form while loops in a flow chart.
  4. a function call
4. Graphical versions of these rules follow.
The semantic rules for SFPs
1. The rule of assignment
  1. The picture describes the meaning of assignment in terms of variable substitution.
  2. Specifically, the precondition for var = expr is derived from the postcondition by systematically substituting all occurrences of var in the postcondition with expr in the precondition.
2. The rule of if-then-else
3. The rule for loops
Application of semantic rules
1. Recall from above that the overall goal of a program verification is to prove
```
precond {function body} postcond
```
2. That is, the precond implies the postcond through the program.
3. In order to push a pre- or postcond through a program, we'll use the semantic rules for the program statements.
4. What each semantic rule provides is a way to mechanically push predicates through a program in order to prove the desired implication goal.
The backwards substitution technique
1. To actually carry out the verification process, we do a kind of symbolic evaluation on an SFP.
2. But, we are evaluating with predicates rather than program variable values.
3. In theory, we can evaluate in either a forward or backward direction.
  1. In practice, however, backwards evaluation is easier.
  2. This is due to the way the rules are constructed and the fact that we are trying to prove an implication from precond to postcond.
4. Here are the steps to carry out a verification, given a SFP program and the semantic rules for the SFP language:
  1. Annotate the program with pre and post conditions.
  2. At each loop node, provide an additional predicate (like a postcondition), called the inductive assertion (more on this below).
  3. Take the overall program postcondition and push it through the program using the semantic rules.
  4. At every point that a "pushed-through" predicate "runs into" a supplied predicate, we have a verification condition (VC) that must be proved.
  5. After all VCs are proved, the program proof is complete, except for a termination condition may need to be proved.
  6. We do not deal with proof of termination in these notes.

Before we tackle verification of the Factorial example, let's see how the preceding verification rules can be used to prove that 2+2=4 (a clearly stunning result).

Here's the program:

int Duh() {
    /*
     * Add 2 to 2 and return the result.
     *
     * precondition: ;
     * postcondition: return == 4;
     *
     */

    int x,y;
    x = 2;
    y = x + 2;
    return y;
}

Here's the SFP:

The example above concluded with the startling result that a program correctly adds 2+2 to get 4.
1. Let's try to prove the following implementation:
```
int ReallyDuh() {
/*
 * Add 2 to 3 and return the result.
 * precondition: ;
 * postcondition: return == 4;
 */

    int x,y;
    x = 2;
    y = x + 3;
    return = y;
}
```
2. Here's the proof attempt
3. What happens here is that we are left with the VC
```
true  4 == 2 + 3    ==>
true  false
```
  which is false.
4. In general, proofs will go wrong at the VC nearest the statement in which the error occurs.
The basic ground rules of implication proofs
1. You may recall from your discrete math class the following truth table for logical implication:
  
  p q p q
  0 0 1
  0 1 1
  1 0 0
  1 1 1
2. That is, the logical implication p q is only false if p is true and q is false.
3. Now, in a formal program verification, we assume that the p in the implication formula is true, since it represents the precondition.
4. Hence, the basic way that a VC will fail to be proved is if q in the implementation is false (as was the case in the attempt to prove 2 + 3 == 4).
Now let's try a proof of the Factorial example.
1. Here's the (correct) function definition:
```
int Factorial(int N) {
/*
 * Compute factorial of x, for positive x, using an iterative technique.
 *
 * Precond: N >= 0
 *
 * Postcond: return == N!
 *
 */
    int x,y;     /* Temporary computation vars */

    x = N;
    y = 1;
    while (x > 0) {
        y = y * x;
        x = x - 1;
    }
    return y;
}
```
2. Note that this is slightly different than the version presented at the beginning of the notes.
  1. Here the formal parameter has been changed from x to N, and x is now a local variable.
  2. We'll discuss why this change was done a little later.
3. Figure 1 outlines the proof; VC proof details follow shortly.
  Figure 1: Factorial proof outline.
Logical derivation of inductive assertion ``y * x! = N!''
1. At the top of loops, we ask ourselves what relationship should exist between program variables throughout the loop. I.e., what relationship should x, y, and N have to one another each time through at the top of the loop?
2. Looking at it another way, we want to characterize the meaning of the loop in terms of program variables.
3. Since the meaning of whole program is y = N!, the meaning of the loop is something like ``y approximates N!'' But how?
4. Putting things a bit more precisely,
  y R f(x) = N!
  for some relation R. And it looks like R is multiplication, i.e.,
  y * f(X) = N!
5. So what is f(x)? I.e., how much shy of N! is y at some arbitrary point k through the loop? It looks like y is growing by a multiplicative factor of x each through, so at point k we have
  y = x * (x-1) * (x-2) * ... * (x-k) * (x-k-1) * ... * 1 = N!
6. I.e.,
  y * x! = N!
7. This kind of reasoning is typical of that used to derive loop assertions.
8. An alternative to puzzling it out with abstract reasoning is to use symbolic evaluation as an aid in deriving loop assertions, which topic will look at shortly.
Further tips on doing the proofs
1. Generally, the proofs of verification conditions are not that difficult.
2. If the program is correct, then the proofs generally involve simple algebraic formula reduction.
3. A discrete Math book (e.g., from CSC 245) contains rules for logical formula manipulation.
4. In addition, here are some rules for reducing "if-then-else" style formulas:
  1. if t then P1 else P2 <=> t P1 and not t P2
  2. if t then t => true
  3. if t then if t then P1 else P2 => if t then P1 else P2
  4. if t then t and P => if t then P
  5. if t1 then if t2 then P => if t1 and t2 then P
  6. t and (if t then P) => P (modus ponens)
  7. t and (if t then P1 else P2) => if not t then P2
  8. xt;=n and xt;=n => x==n
  9. x>n and x<n => false

A closer look at the factorial verification conditions (VC's)

According to the proof strategy outlined earlier, we are obligated to prove each verification condition.
For factorial, VC1 is trivial.

Proof of factorial VC2:

if (y*x! == N! and x>=0) then if (x>0) then y*x*(x-1)! == N! and (x-1)>=0  =>
if (y*x! == N! and x>=0) then if (x>0) y*x! == N! and x>=1  =>
if (y*x! == N! and x>=0) then if (x>0) y*x! == N!  =>
if (y*x! == N! and x>=0) then y*x! == N! and x>0  =>
true

Proof of factorial VC3:

if (y*x! == N and x>=0) then if (x<=0) then y==N!  =>
if (y*x! == N! and x==0) then y==N!  =>
if (y*0! == N!) then y==N!  =>
if (y*1 == N!) then y==N!  =>
true

Looking at some possible errors in factorial and how they would manifest in the verification.
1. Suppose we transpose the two loop body statements (``x = x-1'' and ``y = y*x''), as was the case in the original Factorial function presented above?
2. The ultimate result is we'll get the following erroneous VC3:
```
y * x! = N!  and  xt;=0  and  x>0    y * (x-1) * (x-1)! = N!  and x-1 t;= 0   ==>


y * x! = N!  and  x>0    y * (x-1) * (x-1)! = N!   (oops)
```
3. Suppose we have ``x t;= 0'' in the test (instead of x strictly greater 0); we'll get the following:
```
y * x! = N!  and  xt;=0  and   (xt;=0)    y = N!   ==>


y * x! = N!  and  xt;=0  and  x<0    y = N!
```
Automatic inductive assertion generation via symbolic evaluation
1. A mechanical technique for generating loop assertions is to apply the idea of symbolic evaluation discussed above.
2. Starting with the output predicate, symbolically cranking through the factorial loop looks like this:
  y = N! y * x = N! y * x * (x-1) = N! y * x * (x-1) * (x-2) = N! y * x * (x-1) * (x-2) * (x-3) = N! . . . y * x * (x-1) * ... * (x-N) = N!
3. By inspecting the result of this symbolic evaluation, we notice that the general relationship that remains true during loop execution is y * x! = N!.
4. It's also interesting to look at the erroneous case where the loop statements have been transposed:
  y = N! y * (x-1) = N! y * (x-1) * (x-2) = N! y * (x-1) * (x-2) * (x-3) = N! . . . y * (x-1) * (x-2) * ... * (x-N) = N!
5. In the erroneous case, the symbolic evaluation will lead us to derive the wrong loop assertion.
6. This will ultimately cause the verification to fail (if we don't notice that the assertion is clearly wrong before we attempt the verification).
The verification rule for function calls:
1. Intuitively, what we're doing is substituting the function precondition for the postcondition.
2. Recall in earlier discussions of formal specification we indicated that there are two methods to ensure that function preconditions are maintained:
  1. Specify explicit exceptions that are thrown by a function.
  2. Verify that a function will never be called if is precondition is false.
3. We're now in a position to see how to do the latter of these two methods.

Example verification that function Factorial is never called with a false precond.

Consider the SFP in Figure 2 that calls fact in a verifiably correct way.
Figure 2: Factorial call proof outline.

Table 2 shows the details of the proof, top-down.

Label      Predicate                                       Proof Step

VC:     true => forall (x: integer)                     Rule of verification
            if (x>=0) then x!==x! else x==x               condition generation
        =>
        true                                            Induction on x
Pre:    true                                            Given
P5:     forall (x: integer)                             Rule of readint
            if (x>=0) then x!==x! else x==x
P4:     if (x>=0) then                                  Rule of if-then-else
            if (x>=0) then x!==x! else x!==x                P2
        else
            if (x>=0) then y==x! else x==x                  P3
        =>
        if (x>=0) then x!==x! else x==x                 Simplification
P3:     if (x>=0) then y==x! else x==x                  Rule of assignment
P2:     if (x>=0) then x!==x! else x!==x                Rule of function call
P1:     if (x>=0) then y==x! else y==x                  Rule of assignment
Post:   if (x>=0) then return==x! else return==x        Given

Table 2: Proof that Factorial call does not violate precondition.

Partial versus total correctness The preceding verification methodology demonstrates partial program correctness. Partial correctness means that the program is correct, if and only it terminates To achieve total correctness, we must supply an additional proof of program termination. Such proofs generally involve an induction on one or more program variables. Further discussion of total correctness is beyond the scope of these notes.
Verification and programming style In order to make a program verifiable using the simple rules we've discussed thus far, certain stylistic rules must be obeyed. Here is a summary of rules we've assumed thus far Functions cannot have side effects. Input parameters cannot be modified in the body of a function. (This is why we added the input N to the implementation of the Factorial function earlier.) A restricted set of control flow constructs must be used, i.e., only those constructs for which proof rules exist.
Some critical questions about formal program verification. Can it scale up? Fisher says yes, with appropriate tools. Why hasn't it caught on (yet)? Fisher says for the same reasons that formal specification hasn't caught on yet. When will it catch on? Fisher says when software engineers receive adequate training in formal methods, and production quality tools become available, and software users and customers get sufficiently sick of crappy products. Verification tools include formal specification languages automatic assertion generators automatic theorem Such tools have been developed and studied widely in the research community and at a handful of commercial locations.
An optimistic conclusion -- it will happen, when some or all of the above conditions are met.