CSC 405 Lecture Notes Week 3
Overview of Software Testing Concepts

1. Goals for 405 lecture and lab in weeks 3 and 4.
   1. In lecture, cover conceptual and theoretical details of software testing.
   2. In lab, come up with testing framework(s) and tools to use for testing our projects.

2. A bit of motivation.
   1. In the last decade, there has been Highly significant shift in the industry mindset regarding the importance and utility of testing.
   2. A number of good studies provide evidence that testing can be very cost effective.
   3. The amount of emphasis placed on testing in industrial settings is likely to increase in coming years.
   4. Some costly and spectacular software failures have been due at least partially to inadequate software testing.

3. Some information on the expense of poorly tested software.
   1. See the NIST report, "The Economic Impacts of Inadequate Infrastructure for Software Testing" (2002), wherein it says:
      1. Inadequate software testing costs the US alone between $22 and $59 billion annually
      2. Better approaches could cut this amount in half.
   2. There have been huge losses due to web application failures:
      1. Financial services: $6.5 million per hour (just in USA)
      2. Credit card sales applications: $2.4 million per hour (in USA)
      3. In Dec 2006, amazon.com's BOGO offer turned into a double discount
      4. In 2007, Symantec says that most security vulnerabilities are due to faulty software.
   3. Full citations and further discussion of these points is in the book "Introduction to Software Testing", by Ammann and Offutt, Cambridge University Press, 2008.

4. Testing terminology
   1. You can consider the following topic list to be a quiz of sorts
   2. It tests your knowledge of testing concepts form 308, 309, and wherever else you may have learned about software testing.
   3. The quiz rules are these:
      1. based on common use in the testing vernacular, stick the word "test" or "testing" in front of or after each of the following terms
      2. having so constructed each term, ask yourself if you know what it means
   4. The terms are these:
      1. automation
      2. unit
      3. module
      4. integration
      5. system
      6. acceptance
      7. design
      8. plan
      9. case
      10. oracle
      11. stub
      12. driver
      13. harness
      14. framework
      15. top-down
      16. bottom-up
      17. regression
      18. coverage
      19. subsumption
      20. automation
      21. mutation

5. Automated testing frameworks (further concrete details next week).
   1. X-Unit frameworks -- JUnit, CppUnit, CxxUnit, etc.
   2. File-based frameworks -- using make, ant, or comparable build tools for testing activities.
   3. Testing kernel-based and real-time systems (such as Batman and 80211s) -- the Linux Testing Project provides support for this.