The following systems perform similar functions to the Test Tool.
Here are reviews of those related systems.
Charles Kelly's Online Quiz
Generator
Easy Test Maker
Test Generator (review 1)
Class Builder
abc.test
Digital Teacher
SkillCheck.com
Easy Test Maker
Test Generator (review 2)
Unitest System
To compare the features of the related systems, the following link
shows how well the system performs a specific task. The scores range
from 0 to 10, where 0 means that the tool does not support the feature
at all, 10 means the tool supports the feature as well as possible, and
a rating between 1 and 9 indicates some degree of partial support for
the feature.
1.5.1. Feature Comparison Matrix
Charles Kelly's Online Quiz Generator
Description about related system
Good qualities about related system:
Flaws/What’s Missing about related system:
The Easy Test Maker tool is designed to help users make short test online. The tool cannot be downloaded to the user’s computer and it does not save the questions the user inputs into a test.
When the user first opens this tool, he is faced with two options: Edit and Print an old test or create a new test. After selecting create new test, the next window asks the user to name the test, select a desired font and font size. Then the main window pops up in which the user can either add a question or edit an existing question. This tool has five question options: true/false, multiple choice, short answer, matching and fill-in-the-blank. The tool allows the user to format the appearance of the questions depending on the question type. For example, in a multiple choice question you can change the font and size, bold, highlight or italicize the question. Also, there is an option to organize the answer choices in one, two or three columns. While creating answer choices, the user can click the radio button next to the correct choice, allowing him to print a master answer sheet later. Once a question has been added, the browser allows the user to view the entire test and gives the options to print the test and/or the master answer key. This tool is offered for free and is very simple to use if the user wishes to create simple tests. However, this tool is very different from the one we will be making.
In order to create a fill in the blank question, the user is forced to create a word bank. There is no feature to exclude the word bank, if the user does not wish to add one to his test, making this option no different then the multiple choice question. This tool does not allow the user to customize the test very much. This tool also does not create a data base of questions, so every time the user wants to make the test, they must manually enter the questions in. Also, the students will not be able to take the test online using this tool. The focus of this tool appears to be more on aestheticism then on generating quality test. Because this tool is so simple and does not offer many features, it is easy to find many things that it does not support.
There is no way of saving the questions in a database. Every
time the user wants to make a new test, he has to retype the questions
and the answers. Therefore, the only benefit of this tool is the
way the questions are formatted. This tool also does not
allow students to take the test online; it only offers a printable
version. Therefore, it also does not grade tests. Also, the
amount of question options is very limited and the help option is not
very helpful. This tool is a very simple application that
lacks many features because it is offered for free and it differs
greatly from what we are trying to create.
Test Generator is a test generation tool for educators designed to make the process of creating, administering, and analyzing tests quick and easy. It provides a number of features for both test generation and test taking. In this document I will discuss its strongest feature, question type support, it weakest feature, the question database, and a major missing piece of functionality, cross platform support.
One of Test Generators strongest features is its test generation support. More specifically, it has excellent support for generating tests from eight different question types. Test Generator supports multiple choice, check box, true/false, ordering, fill-in-the-blanks, short answer, point-and click, and essay type questions. Each of these, except essay questions, can be automatically graded. Educators can generate tests targeted at a wide range of grade levels thanks to the wide variety of question types.
The question database is one of the weaker features of Test Generator. Currently, it has no support for ranking the difficulty or expected time required to complete a question. The test generator can randomly generate different tests from a pool of similar questions to prevent cheating; however, there is no assurance that each test will be of similar difficulty. Although a teacher can categorize questions by subject, there is no support for selecting questions from a database pool of questions contributed by multiple teachers.
Although Test Generator is a fairly complete suite of tools, it does not support multiple platforms for test generation. Test Generators tool for creating tests only runs on Microsoft Windows. This is a major problem for educators using other platforms such as Macintosh or Linux. A java client for test generations would allow non-Microsoft users to use this product. Although Microsoft has a very large share of the market, many professors at the University level choose to use other platforms.
Overall, Test Generator is a very solid set of tools for automating parts of the test creating and administering process.
Class Builder is a complete teacher office suite with exam creation,
grading, gradebook, reports,
lesson plans, assignments, calendar, randomized exams, team grades,
individual grades, attendance, etc.
The following review contains details of the exam creation tool of the
Class Builder software.
Exam Creation Tool
Supported Test Types
- Multiple Choice
- True/False
- Matching
- One Line Response
- Essays
- Crossword Puzzles
- Word Find
Question Database
The Class Builder test tool generates tests from a database of text
questions which the user (teacher)
enters manually. I did not find detailed information reguarding
how the questions are stored or sorted.
Creating a Test
The Exam Creator will generate a test from the number and types of
questions the teacher specifies.
The user has 3 options to the order of
the test
1. Scramble the answers to questions (multiple
choice, matching)
2. Vary the order of the questions.
3. Vary the order of the sections.
The user has 3 options to the variety
of the tests.
1. Can create 4 different exams A, B, C, D.
2. Can create a different exam per student.
3. Can create a different exam per class.
The point value is displayed with each question.
Administering and Grading a Test
The teacher can administer the test online or print it out. For
printed tests the teacher can easily print
the appropriate answer keys for the generated tests. Multiple
versions of the test will require multiple
answer keys.
A printed test will require manual grading. The online test has a
supported grader for muitiple choice,
true/false, and matching questions.
Drawbacks
- No option for computer lab supported tests.
- Unnecessary features such as Crossword Puzzle and
Word Find
abc.test is a test-taking and test-making tool with basic functionality that could be considered compact and elegant if it did not separate itself into five separately-running programs, most of which run only in a dialog box. This is far too cumbersome, especially for such a simple implementation of the electronic test tool idea.
A good idea, but not properly implemented, was the program's "simulation" feature. I could not understand how to get it to work (the documentation was not helpful) and the sample test displayed an error message when I requested to run the simulation. This could be useful, because it would allow any sort of media (images, sounds, movie clips) to be associated with a question.
abc.test has a logging feature, which writes to an Excel spreadsheet data concerning who has been taking tests and at what times. The logging feature, if implemented well enough, could possibly collect information that would expose cheaters (for example, keeping track of any websites that were visited or what applications have been running).
This system is more concerned with simplicity than flexibility, and I have already mentioned all features of interest. What we should also learn from this program is that writing a test is simple on paper, so it should also be simple on a computer. Let's not overwhelm our test creators with too many options that are only applicable to a narrow range of test, course, and teacher types.
Digital teacher is an electronic testing tool that generates, administers, and grades tests. One of its main features is generating a simulated test. This option allows instructors to conduct trial runs of their tests and make any modifications they see fit. The tool allows instructors to set test properties including a time limit, giving feedback, or shuffling test questions into a random order as a student begins a test. He can also set the test to be administered on a PC desktop, LAN, or the WWW. For more information on setting test properties, look for test properties under the index of the Digital Teacher authoring application. One feature that should not be used is the white outlined F1 help command that appears on certain menus. This command should be omitted since a Help menu is available and a white outline makes the command hard to view on a screen. The one critical feature Digital Teacher lacks is a Generate Test button that makes a test. The instructor must still manually construct one test page at a time. The Test Tool should also include a scroll-down bar that can transport a student immediately to a specific test question.
The SkillCheck.com test authoring system has several good points about it. First, it has network capability, meaning it can be administrated over a network. This feature seems to be well implemented, although it was unable to be tested. This test system also has the ability to show statistical information about scores as well as other informative data useful to the test administrator. One feature that was particularly well implemented was the ability for test takers to go back examine questions which they had not answered at first or wish to change from the original answer. Another good point about this test taking/authoring system is that it allows the scores to be sent to a central server or database, which is helpful for test administrators. The Test Center feature allows test creators to add new questions, move questions around, set up multiple tests to be taken in sequence, and randomly generate tests on the fly. The Test Center and the tests as seen by test takers feature a standard windowing interface which allows the tests to be easily created and navigated. The test taking center also has the option of using a countdown timer to aide the test taker in planning how long to spend on each question.
While the SkillCheck.com test authoring system has several good points, there exist a few bad points which bring it down overall. The web based interface is usable but the test system uses a Java plug-in which emulates a standard window interface. While this makes it easier to understand for a person who knows little about computers, the system is sluggish and quite slow when it comes to taking the test. However, there is also a standalone version of the test taking system which is quicker and easier to use. The test question attributes are very limited, having only two attributes associated with each question.
The main missing point was that basically this program has little to no security from cheating. Any other program on the computer could be run and used as a tool for cheating. A printed version of the test as well as manual grading by the administrator is not available in this testing tool.
The purpose of the test creation software was to enable math and science professors the ability to generate, implement, and grade test using on-line resources. The software focuses on the three major apects of the test taking process - determining a test syllabus, administering the generated test, and grading the answered results. In this document I will explain the strong aspects, the weakest, and the major missing features that are neccessary for the testing domain.
The software did not stray away from the art of professional teaching by creating control over the quality of questions inserted in the question repository. The questions consisted of small author notes about the credibility, reason, and emphasis of the question. The software designers called these attributes "Meta-Tags" for each question contained. The question repository was well documented with many important traits that held long term purpose. The idea of a question repository was to develop a living entity that would be constantly improving with quality of the tests being generated for students and teachers. The teacher could generate a test suitable for the a specific class elminating the time of procurring each individual question.
The software's negative aspect arrived in the grading tool that did not allow the user to escape the small time consuming aspects of physically hand grading . The software did not assist in changing the point total of a question on the test after it was taking. If the user did not feel the question contained the proper points for the difficulty of the question, there was no feature to edit the point received for every student that took the test. Grading the test manually did not contain the same quality control as generating the test. The grading tool supported an automatic grader, but leaving out possible notes or comments the user might want to insert regrading a student's answer.
The actually testing with students lacked one thing, testing over a variety of different test takers such as disabled people. The generated test used slides in window applet for each question, moving forward by pressing the 'next button' and backwards by pressing 'prev button'. With this design the only intention of test administering tool was purely over the internet through a computer. However in this media the test could only be distributed electronically, not considering other possible students who need to be accommodated for physical or mental handicaps and disabilities.
Overall the test tool maintained quailty control over the questions inside the repository by tagging specific attributes that clarify to validity of the author and the quesiton, developing suitable tests.
The program I reviewed in the last week was Test Generator. The
software contains all the specs. listed in the feature comparison
matrix. However, I was unable to explore all the options for this
product. This was mainly due to the fact that the software comes with
two different versions. One version is for the professor to make the
test, and the other version was made for the sole purpose of having
students take the tests. Test Generator won't allow me to install both
version of the software onto the same computer. Therefore, I was only
able to explore the teacher package.
Test Generator starts up with a user name and password. This feature was great since it identified the user and gave them access privelege according to whom they are. Once log on, to create a new test, I had to declare the class the test was for and its subject. After that, the user has access to the test generatiron screen. The user can select from a list of many types of questions including multiple choice, T/F, essays, etcs. After creating the number of desired questions, all the user had to do was save the test and log out. There are also options for adding and removing undesirable questions within the test. This process of creating the test has a few setbacks.
This product had many great features that I thought could be reused in our own Test Tool project. When creating the test, it has a screen section that lets the professor write notes about the level of difficulty, class, time, date, and hints for individual questions. This feature help organize the professor's files and save time. Test Generator also has a secure password protected feature. The program would only allow students to take the test if the instructor specified within the make test portion of the product that the student has access priveledge.
This software package had some options that were not included that should have been. For example, the option for automatic grading was terrible. The program could only grade multiple choice and T/F questions. For questions such as essays, the professors would have to manually grade it themselves. This is bad for classes such as programming courses. What if the professor wanted the test to automatically check to see if the student program compile. If it compile, the professor could maybe then grade it by hand. If not, then the student would get a 0 just like the regular classes.
Overall, this software is not worth trying for classroom use. The learning curve for this program took over an hour and the results are not all that great. On a scale of 1-10 for overall usability, I give it an all time low 5.
Sight2k's Unitest system is a multifunction tool kit used for purposes of generating, taking, and grading electronic exams. It comes in a package of four different tools: Editor, Monitor, Test, and Report. Each tool provides different functions in the testing process.
Editor is the test generator. It has a wide variety of text editing tools found in most word processors for writing the test, which is a nice addition, but really has little to do with the functionality. It has a straightforward format for setting up the questions and the answers, including the ability to award partial credit for some answers. It has a variety of multiple choice formats, but does not include any other question types, such as fill in the blank, true-false, or any type of long answer question. Other options in test creation include a random shuffling feature, customizable position and size of the questions and answers, and the ability to "lock" the next question until the current one has been answered. Unitest only allows the test taker to see one question at a time, however, before advancing.
Monitor is a security program for the test administrator. It allows the user to see the other users connected to the system, and gives him the power to block or disconnect certain test takers. It also gives the user data concerning each test, such as how well an individual is scoring and how close they are to being finished.
Test is the test taking tool provided in Unitest. When a user starts the program, they are first required to fill in a personal data sheet, which can be customized by the test creator. Once all necessary fields are filled in, the test begins. The test screen consists of a status bar, indicating the current question, total number of questions, and the amount of time left per question/test, depending on the style chosen by the creator. The question appears in a large white text box on the top, and the multiple answers are shown underneath, in similar test boxes. The testee must select an answer, then click the Answer button at the bottom of the page to continue. If specified by the creator, the test will tell you if you answered each question correctly or incorrectly. At then end of the test, a status screen shows how many questions you answered correctly, as well as your calculated percent grade.
Report is the final tool in the Unitest package. It provides a list of results for the test administrator to review. It lists the test takers by name, and also displays other specified data fields that were filled in before the tests were taken. When a name is selected, the individual's test results are displayed. This information includes the question number, the selected answer, credit earned, question type(s), and the test that it belonged to. There is a filter option at the top, apparently for displaying results, but it is confusing and difficult to use.
| Prev: 1.4. Impact Analysis |
Up: 1. Introduction |