Many computerized testing suites exist, but each seems to fall short in one or more important areas. Below we list six commercially available test creation and/or electronic test-taking applications.
Unfortunately, each of the above lacks one key feature: adequate support for programming-based questions. Using the free response question type requires the test taker to write and test his/her code elsewhere then manually copy and paste his/her answers. This reliance on outside tools prevents the instructor from locking the test computer into the testing suite. Moreover, questions of the type "fill in this missing code" can be become tedious and confusing when viewed as a multi-part free response question.
The related systems also lacked the full set of test generation features the client needs. For example, one suite supported randomized tests but didn't allow users to control the type, length, or arrangement of questions. Instructors with specific test needs are typically forced to create tests manually.
The combination of these two missing features and/or poor user interface implementation forced us to reject each of the tested software suits.
See section the Feature Comparison Matrix above for full detail.