The question properties are defined in the form as shown in Figure 2.3.1 and is the first step to adding a new question to the local database. The instructor adding the question is automatically set as the author to the question. The type of question selected determines the structure for the questions in the following step.

Figure 2.3.1: New Question Properties Specifier

To create any test question, the user must input the question type, difficulty setting, question text, and estimated time for the question. These criteria are the same for every question. In the following scenario, the instructor selects a coding type of question from the drop down menu next to Type. The question Difficulty is entered as medium, and the instructor fills in the Question Text with instructions for the coding question. The Time in minutes is entered as an estimated five minutes to complete the question.

Figure 2.3.2: New Coding Question Properties Scenario

The question text is manually edited by selecting the source view. This feature is accessed by clicking on the Source tab above the text area. To toggle back to the plain text view, select the Plain Text tab. The instructor edits the html source code to change the look and feel of the question text. Below, the source veiw for the previous coding question scenario is shown.

Figure 2.3.3: New Coding Question Properties Source View

Though optional, the commands to Import Blank and Import Image can be used in any question. Import Blank imports a blank into the question structure, and in the the scenario of a Fill-In-The-Blank question, this feature will detect how many answers the question requires. Import Image references an image file on the local drive that is uploaded and saved in the local database upon completing the new question.

Depending on the type of question selected, the next step after filling in the question properties will vary to accomodate answers for different types of questions, as shown in the links below. The six types of questions are:

• Multiple choice
• True/False
• Coding
• Short answer/Essay
• Fill-in-the-blank
• Matching

Difficulty setting: The difficulty setting is used to add weight to the question. The difficulty setting has no effect on the point value of the question.

Question text: The area where the instructor inputs the actual text for the question. The instructor may enter a question, statement, scenario, instruction, etc.

Estimated time for the question: The instructor must set the time they expect, on average, the question will take to complete. This is used in automatic test generation when the instructor chooses the length of the test to generate.

Insert Blank: Blanks can be inserted into any question type and is shown in the question text in a pseudo-html format. When used with the Fill-in-the-blank type question, the number of blanks are automatically integrated into the question answers in the following step.

Insert Image: The instructor may chose to enter images into the question from a local disk. This is inserted into the question text in a pseudo-html format. The image will be uploaded into the database and added to the test.

After entering the question general properties shown in Section 2.3.1, the next step is to enter the question's answers, as shown in Figure 2.3.4 below.

Figure 2.3.4: Multiple Choice Answer Input

In this dialog box, the instructor may input as many answer choices as desired. By default there are four answer spaces available to start with. Additional answer choices can be added by clicking on the Add Choice button. Each row has a remove button that removes the corresponding answer choice from the list when clicked.

The option for partial credit is provided in the Multiple Choice answers. Percentage values between 0 and 100 can be entered by the instructor in the Answer Credit column which is used to calculate scores when providing automatic grading for proctored tests. Incorrect answers that contain no partial credit may be filled in with a zero or left blank in the Credit column by the instructor. To proceed to the next step, at least one answer credit percentage must be filled in with a "100" to imply the correct answer to the problem. Also, entering a percentage over 100 will produce are warning that will prompt to enter a new percentage between 0 and 100. In Figure 2.3.5 below, the instructor fills in the answer fields and then decides to add a new question choice by clicking on Add Choice.

Figure 2.3.5: Multiple Choice Answer Input Example

Figure 2.3.6 shows the newly added answer option. The instructor fills in the fields for the new answer choice below.

Figure 2.3.6: Multiple Choice Answer Input Example - Add Choice

In Figure 2.3.7 the instructor decides to remove an answer choice and clicks on Remove next to the first row to remove it. The first answer choice rhwn is removed and the dialog box is updated as shown below.

Figure 2.3.7: Multiple Choice Answer Input Example - Remove

By clicking Next the instrutor will be able to view the question properties and answers for final confirmation, as shown in Section 2.3.8.

Answer Credit: This is the portion of credit is awarded for each answer to the question.

After entering the question general properties shown in Section 2.3.1, the next step is to enter the question's answers, as shown in Figure 2.3.8 below.

Figure 2.3.8: True/False Answer Input

The instructor selects either true or false in the dialog box as shown in the Figure 2.3.8 example above. By clicking on Next the instructor will be able to view the question properties and answer for the final confirmation, as shown in Section 2.3.8.

After entering the question general properties shown in Section 2.3.1, the next step is to enter the question's answers, as shown in Figure 2.3.9 below.

Figure 2.3.9: Coding Answer Input

There are three ways to input answers for a coding question, which include Program Output, Program Text, and Program Script. Each one will produce a different grading scheme for the automatic grading feature. The instructor may utilize all or none of the answer options, given in more detail below. Each answer option will grade the answer independent of the other grading schemes and produce separate results for the instructor to analyze upon reviewing a proctored test.

Program Output: The instructor enters the completed program output into the text area as shown in Figure 2.3.10. The automated grading will give zero or full credit depending on if the instructor output matches the student output exactly. If this box is left blank, then this grading scheme will not occur.

Figure 2.3.10: Coding Answer Input - Program Output

Program Text: The instructor enters the completed program text content into the text area as shown in Figure 2.3.11. The automated grading will give zero or full credit depending on if the instructor output matches the student output exactly. If this box is left blank, then this grading scheme will not occur.

Figure 2.3.11: Coding Answer Input - Program Text

Program Script: The instructor enters a grading script into the text area as shown in Figure 2.3.12. The automated grading will run the script and grade the student code through tests and comparisons, giving specified credit according to the script. If this box is left blank, then this grading scheme will not occur.

Figure 2.3.12: Coding Answer Input - Program Script

If all answer options are left blank, then no grading scheme will occer and answers submitted by the student must be manually graded by the instructor.

A check box beneath the program output box gives the option of launching an IDE (default) for students to code their answer in. Leaving this box unchecked will give the student a text box in the testing environment to type their code into. By clicking on Next the instructor will be able to view the question properties and answers for the final confirmation, as shown in Section 2.3.8.

After entering the question general properties shown in Section 2.3.1, the next step is to enter the question's answers, as shown in Figure 2.3.13 below.

Figure 2.3.13: Short Answer/Essay Answer Input

The short answer/essay question is commonly graded by the instructor, though the TestTool offers a keyword feature that can be used for assisting the instructor. The keyword feature is optional. To use this feature, the instructor inputs keywords, separated by commas, into the text area under the Keywords tab as shown in Figure 2.3.10 below.

Figure 2.3.14: Short Answer/Essay Answer Input - Keywords

When sending graded feedback to the instructor after a proctored test, the TestTool program will highlight all the keywords, and provide the instructor with a list of keywords used and keywords unused for a short answer/essay question.

The instructor may also enter an answer text by clicking on the Explanation tab. In this text box the instructor enters the solution to the essay problem as shown in Figure 2.3.15 below.

Figure 2.3.15: Short Answer/Essay Answer Input - Explanation

By clicking Next the instructor will be able to view the question properties and answers for final confirmation, as shown in Section 2.3.8.

After entering the question general properties shown in Section 2.3.1, the next step is to enter the question's answers, as shown in Figure 2.3.16 below.

Figure 2.3.16: Fill-In-The-Blank Answer Input

Depending on how many blanks were imported into question text in the question properties in Section 2.3.1, this dialog box will show the same amount of answer options. Each row corresponds to one answer blank. The instructor inputs the answer into the first column. If the answer has multiple terms that satisfy a single blank, they can all be entered in answer colomn separated by comas, as shown below in Figure 2.3.17.

Figure 2.3.17: Fill-In-The-Blank Answer Input Example

The second column is optional and may be left blank. The instructor may enter percentage values that provide more point weight to a certain blank than another. If this option is used, each section in the second column must contain input from the instructor and the total of the input must add up to 100 for the instructor to continue to the next step. If the second column is left blank by the instructor, automatic grading will weigh each blank answer equally for the question. By clicking Next the instructor will be able to view the question properties and answers for final confirmation, as shown in Section 2.3.8.

Answer Credit: This is the portion of credit is awarded for each answer to the question.

After entering the question general properties shown in Section 2.3.1, the next step is to enter the question's answers, as shown in Figure 2.3.18 below.

The matching question consists of a list of terms that are matched to an answer bank of terms or definitions. The first column of text areas contain the list of terms. Each of these terms must have a matching answer in the answer bank. The second column of text areas correspond to the answer bank of terms or definitions. Unlike the first column terms, there can be unmatched terms or definitions in the answer bank. By default there are four term options and answer bank options to be filled in by the instructor.

Figure 2.3.18: Matching Answer Input

The instructor enters a list of terms in the first column, and the list of matching answers in the second column. To remove an item in the list of terms or answer bank, the instructor clicks on the Remove button next to the corresponding item. To add an item to the list of terms or answer bank, the instructor clicks on Add Option button below the corresponding column. To match a term with an item in the answer bank, click on the icon next to a term in the first colunm and drag it to the icon next to the corresponding answer in the answer bank. Multiple terms in the first column can be matched to the same answer in the answer bank column.

In the example shown below in Figure 2.3.19, the instructor enters a list of terms in the first column, and this list of matching answer in the second column. An answer option is then added to the second column answer bank.

Figure 2.3.19: Matching Answer Input Example - Add Option

The instructor then enters an answer into the added answer option and drags the icon next the terms to corresponding matching item in the answer bank for each term, as shown below in Figure 2.3.20.

Figure 2.3.20: Matching Answer Input Example - Matching

After matching the terms to items in the answer bank, the instructor deletes two items from the answer bank column that are not matched to a term in the first column, as shown in Figure 2.3.21 below.

Figure 2.3.21: Matching Answer Input Example - Remove

The TestTool will randomize the list when proctoring the test. By clicking Next the instructor will be able to view the question properties and answers for final confirmation as shown in Section 2.3.8.

After all properties have been submitted for the question, final properties and answers are presented for the instructor final confirmation before being added to the local database of questions. A preview for a coding question is shown in Figure 2.3.22 below.

Figure 2.3.22: Previewing a Coding Question - General

To edit a part of the question, the instructor can either click the Back button to edit different question properties and answers in the wizard, or edit the question in the preview dialog box and navigate by clicking on the different tabs. Note that data is not lost by clicking on Back.

An advanced setting is used in the preview dialog box and is optional for the instructor. The advanced setting is used to add the question to a previously created test in addition to the local database upon completion. The instructor selects the check box to use the advanced setting (the settings are greyed out when unchecked), selects previously created tests from the left box and clicks on Add. The selected tests will then show up in the right box as shown in Figure 2.3.22 above and the completed question will be added to these tests upon confirmation. To remove tests that the question will be added to, the instructor selects on tests from the right box and clicks on Remove.

Other question properties are shown and editable in the Question and Answer tab. The Question tab is shown in Figure 2.3.23 and the Answer tab is shown in Figure 2.3.24 below.

Figure 2.3.23: Previewing a Coding Question - Question

Figure 2.3.24: Previewing a Coding Question - Answer

When satisfied with the new question, the instructor clicks on Done to finish creating the question which is then added to the local database.