CSC 309 Lecture Notes Week 5

CSC 309 Lecture Notes Week 5
Specification Refinement for Testing Purposes
Testing Implementation



  1. A specification refinement example.

    1. In 309 Milestone 3, you are continuing the formal specification of methods by copying 308 Spest specs into your 309 implementation.

    2. These formal specs need to be refined in the following ways:

      1. If the specs are incomplete, they need to be updated using the methods discussed in 308; see the 308 lecture notes weeks 6-7 for explanation of these methods.

      2. The specs need to be updated to be consistent with any changes that have been made to method signatures in 309 implementation compared to the 308 abstract model.

      3. If a method throws one or more exceptions, the Spest specification needs to separated into "normal_behavior" and "exceptional_behavior" sections.

      4. If the specification using any unbounded quantification, it needs to be refined to use a bounded form.

    3. Examples of these refinements follow.


  2. Given below are the original abstract specification of the Schedule.scheduleEvent operation, followed by the refined specification of the Schedule.scheduleEvent method.

    1. The 308 abstract operation spec:
      /**
       * ScheduleEvent adds the given Event to the current calendar, if an event
       * of the same start date and title is not already scheduled.
       *
        pre:
          //
          // The title field not empty.
          //
          (event.title != null && event.title.size() >= 1)
      
              &&
      
          //
          // The startOrDueDate field is a valid date value.
          //
          (event.startOrDueDate != null) && event.startOrDueDate.isValid()
      
              &&
      
          //
          // If non-empty, the EndDate field is a valid date value.
          //
          (event.endDate != null) ==> event.endDate.isValid()
      
              &&
      
          //
          // The current workspace is not null.
          //
          (calDB.getCurrentCalendar() != null)
      
              &&
      
          //
          // No event of same startDate and title is in the current workspace
          // calendar.
          //
          !exists (ScheduledItem item ;
              calDB.getCurrentCalendar().items.contains(item) ;
                  (item.startOrDueDate.equals(event.startOrDueDate)) &&
                  (item.duration.equals(event.duration)) &&
                  (item.title.equals(event.title)));
      
        post:
          //
          // If preconds met, a scheduled item is in the output calendar if
          // and only if it is the new event to be added or it is in the
          // input calendar.
          //
          forall (ScheduledItem item;
              calDB'.getCurrentCalendar().items.contains(item) iff
                  (item.equals(event) ||
                   calDB.getCurrentCalendar.items.contains(event)))
      
              &&
      
          //
          // Also, requiresSaving is true in the output calendar.
          //
          calDB.getCurrentCalendar().requiresSaving;
       @*/
      public void scheduleEvent(Event event);
      


    2. The refined spec and method implementation:
      /**
       * ScheduleEvent adds the given Event to the given CalendarDB, if an event
       * of the same start date and title is not already scheduled.
       *
        pre:
          //
          // The Title field is not empty.
          //
         (event.title != null && event.title.length() >= 1)
      
              &&
      
          //
          // The startOrDueDate field is a valid date value.
          //
          (event.startOrDueDate != null) && event.startOrDueDate.isValid()
      
              &&
      
          //
          // If non-empty, the EndDate field is a valid date value.
          //
          if (event.endDate != null) event.endDate.isValid()
      
              &&
      
          //
          // The current workspace is not null.
          //
          (calDB.getCurrentCalendar() != null)
      
              &&
      
          //
          // No event of same title and start date is in the current workspace
          // calendar.  The UserCalendar.getItem method does the work.
          //
          calDB.getCurrentCalendar().getItem(event.getKey()) == null;
      
        post:
         if (!scheduleEventPrecondViolation.anyErrors())
          //
          // If preconds met, a scheduled item is in the output calendar if and only
          // if it is the new appt to be added or it is in the input calendar.  Note
          // that this is a refined version of the original more abstract spec, to
          // make this spec more implementable.
          //
          // The "more implementable" part is constraint clause in the forall that
          // constrains the quantification space to just the items in the output
          // version of the calDB.  In contrast, the original abstract spec did not
          // have this constraint, which meant that the quantification space was all
          // possible ScheduledItems.
          //
          // The reason this constraint makes the postcond more implementable is that
          // the postcondition validation method only needs to range over items in the
          // output DB, not all items of type ScheduledItem.  The latter space is
          // potentially very large.
          //
          forall (ScheduledItem item;
              calDB'.getCurrentCalendar().getItems().contains(item);
                  item.equals(event) ||
                     calDB.getCurrentCalendar().getItems().contains(item))
      
              &&
      
          //
          // Check that the size of the number of items in the output calendar is one
          // greater than the input calendar.  This conjoin is new here compared to
          // the original abstract spec.  Its addition, together with the uniqueness
          // condition being met ensure no junk and no confusion.  The reader should
          // convince her or himself that this is the case.
          //
          calDB'.getCurrentCalendar().getItems().size() ==
              calDB.getCurrentCalendar().getItems().size() + 1
      
              &&
      
          //
          // Also, requiresSaving and hasChanged are both true in the output
          // calendar.  The hasChanged boolean-valued method is part of the
          // observer/observable pattern we'll look at in next week's notes.  The
          // mean of hasChanged from specification perspective is that the method's
          // postcondition is indicating that the method is a mutator the changes the
          // value of the class itself.  In this case, the mutation is additive, by
          // putting a new event in the calendar.
          //
          calDB.getCurrentCalendar().requiresSaving()
              &&
          calDB.getCurrentCalendar().hasChanged();
      
         else
          //
          // Throw exception if preconds violated.
          //
          throw == scheduleEventPrecondViolation;
      
       */
      public void scheduleEvent(Event event)
              throws ScheduleEventPrecondViolation {
      
          /*
           * Clear out the error fields in precond violation exception object.
           */
          scheduleEventPrecondViolation.clear();
      
          /*
           * Throw a precond violation if the validity check fails on the title,
           * start date, or end date.
           */
          if (validateInputs(event).anyErrors()) {
              throw scheduleEventPrecondViolation;
          }
      
          /*
           * Throw a precond violation if an event of the same start date and
           * title is already scheduled.
           */
          if (alreadyScheduled(event)) {
              scheduleEventPrecondViolation.setAlreadyScheduledError();
              throw scheduleEventPrecondViolation;
          }
      
          /*
           * Throw a precond violation if there is no currently active calendar.
           * Note that this condition will not be violated when interacting
           * through the view, since the 'Schedule Event' menu item is disabled
           * whenever the there is no active calendar.
           */
          if (calDB.getCurrentCalendar() == null) {
              scheduleEventPrecondViolation.setNoActiveCalendarError();
              throw scheduleEventPrecondViolation;
          }
      
          /*
           * If preconditions are met, add the given event to the currently
           * active calendar.
           */
          calDB.getCurrentCalendar().add(event);
      
      }
      


    -- Now onto a Testing Implementation --

    Note: For Milestone 3 you need to refine the Spest and define class test plans, but not do any testing implementation.

    For Milestone 4., you need to do some testing implementation, details of which are covered in the remainder of these notes.



  3. The common core for a unit testing is the same for just about any testing framework:

    1. Setup -- set up the inputs necessary to run a test

    2. Invoke -- invoke the method under test and acquire its actual output

    3. Validate -- validate that the actual output equals the expected output


  4. Terminology for testing methods and classes.

    1. In any style of testing, log-based, JUnit3 or 4, or TestNG, the terminology is the same:

      1. The method under test needs to get called.

      2. The testing method does the calling.

      3. In object-oriented languages like Java, these methods reside a in their own classes

    2. General form of a class and method under test:
      class X {
      
        // Method under test
        public Y m(A a, B b, C c) { ... }
      
        // Data field inputs
        I i;
        J j;
      
        // Data field output
        Z z;
      
      }
      

    3. General form of the testing class and method:

      class XTest {
        public void testM() {
      
          // Set up
          X x = new X(...);
          ...
      
          // Invoke
          Y y = m(aVAlue, bValue, cValue);
      
          // Validate
          assertEqual(y, expectedY);
        }
      }
      


  5. Summary of where test specification and planning fits in:

    1. The javadoc comment for the method under test contains the Spest spec that specifies what must be true for inputs and outputs used in the tests.

    2. The javadoc comment for the testing class specifies the major phases of the testing, including the order in which the unit tests are executed.

    3. The javadoc comment for the testing method defines the unit test plan in tabular form; the plan has inputs, expected outputs, and remarks for each test case.


  6. Different forms of functional test implementation.

    1. Pure end user testing

      1. The tests are written as scripts to be followed by human end-users.

      2. The tests use the same form of testing plans and heuristics discussed in Lecture Notes Week 4, namely:

        1. Test plans are written a sequence of test cases.

        2. Each case has an input, expected output, and remarks about what the test case is intended to do.

      3. What distinguishes pure end-user testing is that humans communicate with the program via the HCI rather than the code-level API.

        1. A user-level test case says something along the lines of "enter this data ..., then press this button, then expect to see a display that looks like this ...".

        2. This is the analog of an API-level test that says "call this method with these data and expect the following method output ..."

      4. That is, humans analyze the results of test by looking at output on the HCI of the program and comparing it to a description of the expected output in the test plan.


    2. Log-based testing.

      1. This form of testing was illustrated towards the end of Lecture Notes Week 5

      2. The tests are implemented by calling methods with test case data and writing the results to an output log.

      3. Additional information about the progression of the tests can also be written to the log, such as what methods are being tested and what cases are being executed for each method.

      4. After the first run of a test, humans inspect the log to confirm that the output is correct.

        1. This inspection is be done manually by looking at the logs.

        2. It can also be done by generating a separate log inspector that validates that the expected and actual outputs match.

      5. After the initial validating inspection, all subsequent runs of the tests, i.e., the "regression executions" are done by mechanically differencing expected and actual output, e.g., using a difference utility such as UNIX diff.


    3. X-Unit testing, e.g, JUnit, Cpp-Unit, and TestNG

      1. The overall structure of the tests is the same as the log-based implementation.

      2. What primarily distinguishes X-Unit test is that expected and actual outputs are compared using auxiliary testing methods, which report with Java asserts

      3. The other distinguishing characteristic of X-Unit tests is that overall results are typically reported in a web-based report, rather than a raw output log.


  7. Pure End User Testing Pros and Cons

    1. Pros:
      • Ensures HCI evaluated by humans.
      • Makes programmer involvement indirect.

    2. Cons:
      • Difficult to verify coverage.
      • Difficult to do stress testing.

    3. Other considerations:
      • Works well with suitable work force.
      • Should always be done in some form.


  8. Log-Based Testing Pros and Cons

    1. Pros:
      • Expected test results defined concretely.
      • Reports only differences.

    2. Cons:
      • Generating expected results may be tedious
      • Does not implement oracle explicitly.

    3. Other considerations:
      • Useful when spec is data-oriented, e.g., a compiler.
      • Can be used in combo with X-Unit.


  9. X-Unit Pros and Cons

    1. Pros:
      • Formal spec oracle directly implemented.
      • No need to generate expected output data.

    2. Cons:
      • Oracle implementation may itself be buggy.
      • Oracle execution may take longer than diff.

    3. Other considerations:
      • Becoming an industry standard.
      • Can support log-style if desired.


  10. What We'll Do in 309

    1. X-Unit or log-based testing of model and process packages, with TestNG being the recommended framework.

    2. Partial X-Unit testing of program views.

    3. Pure end-user testing, after unit testing; we'll call this "acceptance testing".


  11. Recap of highlights for Milestone 4

    1. Approximately 75% of model/view done

    2. Finish Spest for all methods

    3. Write unit tests for 8 methods per person

    4. Unit test development steps:

      1. Finish Spest specs.

      2. Use specs and testing heuristics to development unit test plan

      3. Implement unit test plan.


  12. Concrete details of using TestNG

    1. TestNG is the recommended functional testing framework for 309.

    2. There is a very good how-to document for TestNG at http://testng.org/doc/documentation-main.html

    3. Examples of using TestNG to implement class and unit tests are in the 309 Milestone 4 example for the Schedule Test

    4. We'll go over these examples in class, in particular the example for Calendar Tool scheduling:


  13. Here is a pure logging style version of the preceding TestNG example.

    1. The excerpts show how test output are reported to the log.

    2. Examples of jUnit-style testing follow after the logging example, including a comparison of xUnit-style testing with logging style.

      package caltool.schedule;
      
      import caltool.caldb.*;
      import mvp.*;
      import java.util.*;
      
      /****
       *
       * Class ScheduleTest is the companion testing class for class <a href=
       * Schedule.html> Schedule </a>.  It implements the following module test plan:
       *                                                                        <pre>
       *     Phase 1: Unit test the constructor.
       *
       *     Phase 2: Unit test the simple access method getCategories.
       *
       *     Phase 3: Unit test the constructive methods scheduleAppointment,
       *              scheduleTask, and scheduleEvent.
       *
       *     Phase 4: Unit test the constructive methods scheduleMeeting and
       *              confirmMeeting.
       *
       *     Phase 5: Unit test the changeItem and deleteItem methods.
       *
       *     Phase 6: Repeat phases 1 through 5.
       *
       *     Phase 7: Stress test by scheduling and deleting 100000 items.
       *                                                                       </pre>
       */
      public class ScheduleTest extends Schedule {
      
          /*-*
           * Public methods
           */
      
          /**
           * Construct by calling the parent constructor with a null View and the
           * given stubbed CalendarDB.
           */
          public ScheduleTest(CalendarDB calDB) {
              super(null, calDB);
          }
      
          /**
           * Run all the phases of this.
           */
          public void run() {
              phase1();
              phase2();
              phase3();
              phase4();
              phase5();
              phase6();
              phase7();
          }
      
      
          /*-*
           * PhaseX methods are for each module testing phase.
           */
      
          /**
           * Execute test phase 1 by calling testSchedule.
           */
          protected void phase1() {
              dumpPhaseHeader(1);
              testSchedule();
              dumpPhaseEndSpacing();
          }
      
      ...
      
          /*-*
           * Individual unit testing methods for member methods
           */
      
          /**
           * Unit test the constructor by building one Schedule object.  No further
           * constructor testing is necessary since only one Schedule object is ever
           * constructed in the CalendarTool system.
           */
          protected void testSchedule() {
              dumpUnitTestHeader("Constructor");
              schedule = new Schedule(null, calDB);
              dumpUnitTestEndSpacing();
          }
      
          /**
           * Unit test getCategories by calling getCategories on a schedule with a
           * null and non-null categories field.  The Categories model is tested
           * fully in its own class test.
           *                                                                    <pre>
           *  Test
           *  Case    Input                   Output          Remarks
           * ====================================================================
           *   1      schedule.categories     null            Null case
           *            = null
           *
           *   2      schedule.categories     same non-null   Non-null case
           *            = non-null value      value
           *                                                                   </pre>
           */
          protected void testGetCategories() {
              Categories result;                              // method return value
      
              /*
               * Dump a unit test header.
               */
              dumpUnitTestHeader("getCategories");
      
              /*
               * Do case 1 and validate the result.
               */
              dumpUnitTestCaseHeader("getCategories ", 1);
      
              schedule.categories = null;                     // setup
      
              result = schedule.getCategories();              // invoke
      
              if (!validateGetCategoriesPostcond(result))     // validate
                  dumpMessage("FAILED");
      
              dump("Case 1 Result:");                         // dump result
      
              /*
               * Do case 2 and validate the result.
               */
              dumpUnitTestCaseHeader("getCategories ", 2);
      
              schedule.categories = new Categories();         // setup
      
              result = schedule.getCategories();              // invoke
      
              if (!validateGetCategoriesPostcond(result))     // validate
                  dumpMessage("FAILED");
      
              dump("Case 2 Result:");                         // dump result
      
              dumpUnitTestEndSpacing();
          }
      
          /**
           * Unit test scheduleAppointment by ...
           */
          protected void testScheduleAppointment() {
              /* ... */
          }
      
          /**
           * Unit test scheduleTask by ...
           */
          protected void testScheduleTask() {
              /* ... */
          }
      
          /**
           * Unit test scheduleEvent by supplying inputs that test the value range of
           * each Event data field and exercise each precondition clause.  The cases
           * are as follows:
           *                                                                    <pre>
           *  Test
           *  Case    Input                   Output          Remarks
           * ====================================================================
           *   1      {"Event 0",             Input event     Lower end of possible
           *           {1"January",1},        added to items  event data range
           *            null,                 no junk,
           *            null,                 no confusion
           *            "public"}
           *
           *   2      {10000-char string,     Input event     Upper end of possible
           *           {1,"January",1},       added to items  event data range
           *           {31,"December", 9999}, no junk,
           *           10000-char string      no confusion
           *             reversed,
           *           "private"}
           *
           *   3      events with start and   Input event     Events with all combos
           *  thru    end date ranging thru   added to items  of days, dates, and
           *  2564    each legal day, each    no junk, no     months
           *          legal date, and each    confusion
           *          legal month, with
           *          arbitrary year,
           *          category, and security
           *
           *  2565    32 cases to exercise    Schedule        Events that violate
           *  thru    possible boolean        Event           precond logic.
           *  2597    values for 5 precond    Precond
           *          clauses                 Exception
           *                                   thrown
           *                                                                   </pre>
           */
          protected void testScheduleEvent() {
      
              /*
               * Dump a unit test header.
               */
              dumpUnitTestHeader("scheduleEvent");
      
              /*
               * Do the cases.
               */
              try {
                  testScheduleEventCase1();
                  testScheduleEventCase2();
                  testScheduleEventCases3Thru2564();
                  testScheduleEventCases2565Thru2597();
              }
              catch (ScheduleEventPrecondViolation e) {
                 /* This should not happen at this level, but in case it does ... */
                  System.out.println("scheduleEvent EXCEPTION:");
                  for (int i = 0; i < e.numberOfErrors(); i++) {
                      System.out.println(e.getErrors()[i]);
                  }
              }
      
              /*
               * Dump the test case results.
               */
              dump("");
              dumpUnitTestEndSpacing();
      
          }
      
          /**
           * Run testScheduleEvent case 1.
           */
          protected void testScheduleEventCase1()
                  throws ScheduleEventPrecondViolation {
      
              dumpUnitTestCaseHeader("scheduleEvent", 1);
              schedule.scheduleEvent(new Event(
                  "Event 0",                          // title
                  new Date(1,                         // start date
                      new MonthName("January"),
                      1),
                  null,                               // end date
                  new Category("Category 1"),         // category
                  new SimpleSecurity("public")        // security
              ));
          }
      
          /**
           * Run testScheduleEvent case 2.
           */
          protected void testScheduleEventCase2()
                  throws ScheduleEventPrecondViolation {
      
              int i;
              String longString = "";
              String longStringReversed = "";
              for (i = 0; i < 10000; i++) {
                  longString += (char) i % 256;
                  longStringReversed = (char) i + longStringReversed;
              }
      
              dumpUnitTestCaseHeader("scheduleEvent", 2);
      
              schedule.scheduleEvent(new Event(
                  longString,                         // title
                  new Date(1,                         // start date
                      new MonthName("January"),
                      1),
                  new Date(31,                        // end date
                      new MonthName("December"),
                      9999),
                  new Category(longStringReversed),   // category
                  new SimpleSecurity("private")       // security
              ));
          }
      
          /**
           * Run testScheduleEvent cases 3 through 2564 by looping through various
           * combinations of days, weeks, and months.
           */
          protected void testScheduleEventCases3Thru2564()
                  throws ScheduleEventPrecondViolation {
      
              int day, number, month, year, i, n;
              Date date;
      
              for (i = 3, number = 1; number <= 31; number++) {
                  for (month = 0; month < 12; month++) {
      
                      date = new Date(
                          number,
                          new MonthName(month),
                          year = number * 100);
      
                      if (date.isValid()) {
                          dumpUnitTestCaseHeader("scheduleEvent", i++);
      
                          schedule.scheduleEvent(new Event(
                              "Event " +                              // title
                                  String.valueOf(n = year + month + day),
                              date,                                   // start
                              date,                                   // end
                              new Category(String.valueOf(n*10)),     // category
                              ((number % 2) == 0) ?                   // security
                                  new SimpleSecurity("public") :
                                  new SimpleSecurity("private")
                          ));
                      }
                  }
              }
      
          }
      
          /**
           * Run testScheduleEvent cases 2565 through 2597 to fully exercise the
           * precond logic.
           */
          protected void testScheduleEventCases2565Thru2597() {
              /* ... */
          }
      
      
          /*-*
           * Postcondition validation methods.
           */
      
          /**
           * Evaluate the Schedule constructor postcond on the given calDB and
           * cheduleEventPrecondViolation inputs, with this.schedule as the actual
           * output.
           */
          boolean validateSchedulePostcond(CalendarDB calDB,
                  ScheduleEventPrecondViolation scheduleEventPrecondViolation) {
              return this.calDB == calDB && scheduleEventPrecondViolation != null;
          }
      
          /**
           * Evaluate the getCategories postcond with the given returnVal as the
           * actual output.
           */
          boolean validateGetCategoriesPostcond(Categories returnVal) {
              return schedule.categories == returnVal;
          }
      
      
          /*-*
           * Testing utility methods.
           */
      
      ...
      }
      


    3. And here is the testing driver, in ScheduleTestDriver.java.

      package  caltool.schedule;
      
      import caltool.caldb.*;
      
      /****
       *
       * Test driver for ScheduleTest.
       *
       */
      
      public class ScheduleTestDriver {
      
          /**
           * Construct a stub CalendarDB and call ScheduleTest.run().
           */
          public static void main(String[] args) {
              CalendarDB calDB = new CalendarDB();
              ScheduleTest scheduleTest = new ScheduleTest(calDB);
      
              scheduleTest.run();
          }
      
      }
      


  14. The following excerpts are from a JUnit3 version of the precediing ScheduleTest.java.
    package caltool.schedule.junit3;
    
    import junit.framework.*;
    
    import caltool.caldb.*;
    import mvp.*;
    import java.util.*;
    
    /****
     * This is a JUnit 3 version of the SceduleTest class.  This class has the same
     * logical testing structure as the preceding tests, but uses JUnit conventions.
     * The code comments indicate the differences between the logging-style testing
     * compared to the JUnit assert-style testing in this file.
     *
    ...
     *
     */
    public class ScheduleTest extends TestCase {   // Note extension of TestCase
    
    ...
    
        // In JUnit, the run and  methods are supercede by JUnit's test runner.
    
    
    ...
    
        /**
         * Unit test getCategories by calling getCategories on a schedule with a
         * null and non-null categories field.  The Categories model is tested
         * fully in its own class test.
         *                                                                    <pre>
         *  Test
         *  Case    Input                   Output          Remarks
         * ====================================================================
         *   1      schedule.categories     null            Null case
         *            = null
         *
         *   2      schedule.categories     same non-null   Non-null case
         *            = non-null value      value
         *                                                                   </pre>
         */
        protected void testGetCategories() {
            Categories result;                              // method return value
    
            /*
             * Dump a unit test header.
             */
            // dumpUnitTestHeader("getCategories");         // automatic in JUnit
    
            /*
             * Do case 1 and validate the result.
             */
            // dumpUnitTestCaseHeader("getCategories ", 1); // automatic in JUnit
    
            schedule.categories = null;                     // setup
    
            result = schedule.getCategories();              // invoke
    
            assertTrue(validateGetCategoriesPostcond(result))     // validate
    
            // dump("Case 1 Result:");                      // automatic in JUnit
    
            /*
             * Do case 2 and validate the result.
             */
            // dumpUnitTestCaseHeader("getCategories ", 2);
    
            schedule.categories = new Categories();         // setup
    
            result = schedule.getCategories();              // invoke
    
            assertTrue(validateGetCategoriesPostcond(result))     // validate
    
            // dump("Case 2 Result:");                      // automatic in JUnit
    
            // dumpUnitTestEndSpacing();                    // automatic in JUnit
        }
    
         ...
    
    }
    


  15. Here is the JUnit3 version of the test driver.
    package  caltool.schedule.junit3;
    
    import junit.framework.*;
    import junit.runner.BaseTestRunner;
    
    import caltool.caldb.*;
    
    /****
     *
     * Test driver for ScheduleTest.  This driver class contains only a simple main
     * method that constructs a JUnit test suite containing ScheduleTest.  A JUnit
     * runner, either command-line or in an IDE, takes it from there.
     *
     */
    
    public class ScheduleTestDriver {
    
        /**
         * Construct Junit test suite containing ScheduleTest and call the runner.
         */
        public static void main(String[] args) {
            junit.textui.TestRunner.run(suite());
        }
    
        /*
         * Construct the test suite containing XTest.
         */
        public static Test suite() {
            TestSuite suite= new TestSuite("XTest");
            suite.addTestSuite(XTest.class);
            return suite;
        }
    }
    


  16. JUnit Styles and Commenting Conventions

    1. In the preceding example, each JUhit assert corresponds to a test case in the unit test plan.

    2. The example has multiple asserts in one testing method.

    3. A different JUnit convention is to have only one assert per testing method.

      1. The primary rationale for this convention is that when a JUnit assert fails, the method in which the assert appears exits immediately with an exception.

      2. This means that subsequent asserts in the method are not executed until the cause of the first assert failure is fixed.

    4. If you use the one-assert-per-method style, then the following is a commenting convention that is consistent with the 309 unit test plan style:

      1. Define a no-op testX method, with the unit test plan as its javadoc comment.

      2. For each test case, define a method named testXCasen, with a two-word javadoc comment "Case n".


  17. JUnit Implementation Options

    1. In JUnit Version 3, test method names prepended with "test".

    2. In Version 4, test method definitions are preceded by the Java annotation "@Test", with the "test" name prefix being optional.

    3. Also in Version 3, there is an optional setUp method for common pre-test code.

    4. In Version 4, the setUp method is preceded by "@Before", and the method name need not be "setUp", as is required in version 3.

    5. In Version 3, the optional tearDown method complements setUp.

    6. In Version 4, the tear-down method is preceded by "@After", and the method name need not be "tearDown", as is required in version 3.


  18. Summary comparison of JUnit/TestNG with Log-Style testing.

    1. Class-level phases are purely commentary in xUnit, but part of the testing structure in the logging style.

    2. All message dumping is done automatically TestNG/JUnit, whereas it's explicit in the logging style.

    3. Output and differencing is done by TestNG/JUnit; with logging it's done with an external differencing tool, such as UNIX diff.




index | lectures | handouts | examples | doc | lib | grades
//