CMPT 480 / 840 Accessible Computing

Project Evaluation


Due Date: 12:00 noon, Mon, Nov. 28, 2016


The evaluation of each project will depend on the project being evaluated and the level of the course (480 or 840 in which the student is registered). Students are advised to discuss their plans for evaluation with the instructor to ensure that they are suitable.

NOTE: there are many similarities between what is needed for a good evaluation report and the various published research papers that you should have referred to in your analysis

Project Evaluation for the 2016 Class

This year we have 3 CMPT 480 Projects that all focus on topics that could be difficult to evaluate in the time available. While it is possible to set up user testing of products (such as ADEs), without having products to test, the most appropriate type of testing is often expert evaluation. Ideally it would be nice to have your analysis and design reports evaluated by a real expert who is independent from the class. However, this might be too time consuming to obtain in time for you to complete an evaluation report by the due date.

The role of evaluation in this project, is to recognize that the intial analysis and design can benefit from iterative improvements. Thus, in addiiton to evaluating challenges and opportinutues, it is important to beab le to take action on these challenges and opportunities to improve your project.

Therefore I am suggesting a three part procedure for you to do to perform a suitable evaluation:
  1. Instructor Evaluation - the instructor will evaluate your analysis and design and suggest challenges and opportunities that need your attention. This instructor-based evaluation will focus on identifying the challenges and opportunities and not necessarily on suggesting what needs to be done about them.
  2. Self Evaluation - you should critically evaluate your analysis and design to identify challenges and opportunities that need to be addressed. This could include identifying further useful information that could be used to improve your project.
  3. Proof by Example - in addition to parts 1 and 2, you should create examples to illustrate that your design works as it is intended to work. In creating examples you will either prove that it can work or that it needs improvement. If it needs improvement, then you should also make the improvements and create examples to show that the improvements make it work.
Your evaluation report should contain:
  1. A report on what you have done, including:
    1. A discussion of the instructor based evaluation and what you have done about it to improve your project. This can be done using the same format as for class critiques.
    2. A discussion of your own critical analysis of the project. This also can be done using the same format as for class critiques.
    3. A discussion of how you went about creating and using examples and of what you found from using them, including any challenges or opportunities thta they identified.
  2. A revised version of your analysis and design report, based on what you ahve done.
    1. changes should be easily identified by the instructor (preferably using Word's track changes feature) and should include a brief comment (either in italics and parenthesis or using Word's comment feature)
    2. the examples should be included in the design portion of the report in an easy to use manner.


Background (used other years)

ReDesign

In order to have something to evaluate you require a suitable design (and implementation - if applicable).

Many students might need to revise their design from the analysis and design to take into account any important concerns provided in the marking of your analysis and design report. Those marking comments provide a first level of evaluation and need to be taken into account before you have a suitable design/implementation to use in this phase of your project.

The first section of your evaluation report needs to identify the comments received in the marking of your design and explain what was done to address them. If there were serious problems with your design, you should also provide a revised version of your design as part of this section of your evaluation report.

Implementation (required for CMPT 840)

CMPT 840 students are required to report on their implementation as part of their report for this stage of the class project.
See below for details.

The particular way of developing your implementation will depend on your project.
In addition to actually implementing the design, you need to provide a report on the implementation, that should include:

Evaluation

It is important that you find a way of evaluating your design/implementation.
Your evaluation report should include:
Additional information on  different forms of evaluation including user testing and expert evaluation is contained in Chapter 16 of the course text.

Students who are unsure of what will be a suitable evaluation method for their project should discuss their ideas with the instructor as soon as possible.
Evaluation of an ADE should involve actual user testing.  

CMPT 840 Evaluation Report

Your evaluation report should include:
Additional information on  different forms of evaluation including user testing and expert evaluation is contained in Chapter 16 of the course text.

Copyright 2010, 2016 - Jim A Carter Jr