CSC 590 Assignment 2 -- Thesis Evaluations

Requirements

In this assignment you are critically evaluating two MS theses, at least one of which should be in your general area of thesis research. Some examples of noteworthy recent theses from Cal Poly CSC are available on the 590 we 590/reference/theses . These works are from recent Cal Poly MS graduates. They represent areas that are typical for thesis research at Cal Poly. If one of these happens to be in or near your area of research, you can choose it to review. Most likely, however, you will review a thesis of a recent student of your current thesis advisor.

For the second thesis to review, choose a thesis from a university other than Cal Poly, if possible. It can be from a MS-granting institution comparable to Cal Poly, such as another CSU, but it need not be limited to such institutions. If there are two theses from Cal Poly that are particularly relevant to your work, then both of your reviewed theses can be from Poly.

To perform the evaluation, you will use the criteria in the one-page evaluation sheet entitled "Quality Assessment of the MS Thesis". This is the sheet used by thesis committee members to evaluate a thesis at the conclusion of the thesis defense.

For this assignment, in addition to scoring the assessment criteria numerically, you will also provide specific justification for your scores. The justifications include critical evaluation, and are based on citing specific portions of the work to support your critique.

The format of an evaluation is as follows:

Title of Thesis:
Author:
Date of Publication:
Institution:
Type of thesis (project, experimental, theoretical, survey, other (specify)):
Area of work(e.g., AI, Distributed, Networks, SE):

1. Problem definition

Score: (1 - 5, fractional scores are OK)
Justification and critique ...

Sections 2 through 9 cover the additional specific assessment criteria --
"Writing Quality" through "Overall quality of the thesis"

They have the same format as Section 1, i.e., score and justification.

You are to produce two separate evaluations in this format, one for each thesis.

Discussion

As outlined above, you will provide both a numeric score for each of the ten assessment criteria, as well as justification of your scores. The following guidelines describe the nature of the justification and critiqe for each criterion.

Problem definition:

Here you rate whether the problem(s) addressed by the thesis are well defined. To justify your rating, summarize what the problem(s) is (are). If you cannot readily do this, say so. Not being able to summarize the problems is justification for a low score on this criterion.

To critique this aspect of the thesis, indicate whether or not the thesis delivers a good solution to the problem(s) it purports to solve. Justify this critique by citing specific parts of the thesis that make it clear how the problems are in fact solved.

Writing quality

This is a general evaluation of the writing quality. Presumably there should be no problems in spelling or grammar, but definitely say so if there are.

There are a vast number of ways to evaluate writing quality. One of the most important writing qualities for a thesis is clarity of presentation. So for this critique, focus on this aspect of the writing, and justify your critique as follows:

• cite one or two well-written passages, and explain how the passage adds to the clarity of the presentation (if there are zero well-written passages, say so)
• cite one or two poorly-written passages, and explain how the passage detracts from the clarity of the presentation (if there are zero poorly-written passages, say so)

Does this thesis advance the state of the art in its specific area? Justify your evaluation by summarizing precisely how it does so, or if it does not, how it falls short.

An important measure to justify a thesis' contribution to the state of the art is how thoroughly it cites related work. The thesis should include a thorough comparative analysis, where it compares and contrasts key related work to that of the thesis. In your critique of this "contribution" criterion, include a discussion of the following points regarding related work:

• Are there an ample number of related works cited?
• Does the thesis do a good job in comparing and contrasting its contributions to the related work?

Is the work presented in the thesis original and/or innovative? Justify your evaluation by summarizing in what ways it is or is not. You should be able to identify one or a few new ideas that the thesis contains. If you cannot do so, then a low score is appropriate for this criterion. Technical depth

The technical depth of a thesis varies based on its type. If it is experimental, then the experiment should be asking a hard and important question, and use rigorous techniques to analyze the experimental results. If it is a project, then it should involve non-trivial specification, design, and implementation techniques to complete. If it is theoretical, then it should use rigorous techniques to prove its results. If it is a survey, it should provide in-depth coverage and critical analysis of the material it surveys.

Justify your evaluation here by citing specific parts of the thesis where the technical depth is achieved. Implementation of technical content

As footnoted in the attached sheet, implementation is also type-specific. I.e., it's the conduct of the experiment for an experimental thesis; the program design and implementation for a project; the presentation of proof and/or argumentation for a theoretical thesis; some suitable interpretation for another type of thesis.

As with technical depth, justify by citing specific parts of the thesis where the implementation is presented.
Validation of the work, as appropriate to the subject matter

Validation is a third type-specific evaluation criterion. For an experiment, the thesis should compare its results to the work others in the field, and verify that its results are significant. For a project, the thesis must validate that the implementation works, including with usage studies if appropriate. For theoretical work, the proof must be complete and correct. For a survey, the critical analysis must be cogent, and backed by a substantial number of references to the literature.

Justify your evaluation here by citing specific parts of the thesis where the validation appears. Potential for publication

Based on your reading and knowledge in the area, do you think the work of the thesis has potential for publication? If so, for what specific conference(s) (most likely) or journal(s) (perhaps) is it suited? Justify your evaluation by referring as appropriate to the critique points you have already made.

Potential for future research

Does the work of thesis provide a platform for future research or development? If so, summarize what this would be. If not, say why. Overall quality of the thesis

Summarize the results of the preceding critiques in one or a few sentences that sum up what you think of the thesis overall.

Example

See http://www.csc.calpoly.edu/~gfisher/classes/590/examples/assignment2/

Turn-in Procedure

Please submit your two reviews to the Assignment 2 590 PolyLearn site, per the instructions there.