CMPT 480 / 840 Accessible Computing

Critiques

Purposes of a Critique

Critiques get you thinking about the important parts of what is contained in a chapter, by getting you to go beyond what it contains. There is no need to spend class time on going over material from the chapter that everyone can get from reading the chapter. Instead, classes will be customized to focus on those topics that are of particular interest to students that discuss particular issues related to this material.

NOTE: Critiques are not book reports or summaries of the chapter or the key points in the chapter. They need to go beyond the chapter to have value.

Critiques are worth 30% of your mark in the course. They take the place of a final exam and thus are an important source of demonstrating that you have mastered the content of the course. Critiquing also provides you with practice on a very important skill that you can utilize beyond this course.

Submitting your Critique

All students are required to e-mail the instructor their critiques

The subject of the e-mail should be "480/840 Critique for Chapter xx - your name", where xx is the number of the chapter the critique is for (not the number of the week in course) and where you insert your own name in the subject line.

Use the critique template for formatting their critiques.

It would be appreciated if you would follow the following convention for the files you send for your weekly critique

Please name the files: nn-your name

where nn is the number of the chapter that the critique is for

Please send the files in Word (doc or docx) format - that way I can respond by using track changes for my comments

Contents of a Critique

Do a critique of the readings assigned for next week's class by identifying and discussing at least five major challenges and/or opportunities.

Challenges identify portions of a reading where significant improvements should be made.
Opportunities identify omissions from a reading where significant additions should be made.

NOTE: The focus of improvements or additions should be on the knowledge contained in the reading (and not on how to improve the reading itself) and important issues for us to discuss beyond the contents of the reading. There will be no marks for comments that are directed at the author of the text.

Each discussion of a challenge or opportunity should contain (each of the following items of information that are identified in bold):

identify the challenge or opportunity in terms of:

a meaningful self-descriptive name for the challenge / opportunity
the type of critique item (either a challenge or opportunity)

Challenge: what needs improvement (further discussion and/or consideration of an alternate viewpoint)

challenges should be major issues and not just identification of typos or other minor problems in the text

Opportunity: what's missing (not discussed but should be included in a discussion of this area)

e.g. additional user accessibility needs belonging to a principle are excellent opportunities

information about the location in the chapter the challenge / opportunity occurs

the section number and title where this issue belongs being discussed
what the section/subsection of the chapter said: either a short quote or a summary of what is said;

a brief discussion of the issue

a short statement of what is missing / incorrect (which will be discussed further in the suggestion)

explain the significance of the proposed addition or improvement to accessibility

in terms of

why this challenge or opportunity is important for accessibility; and
why this is significant to the topic of the chapter being critiqued

be specific as to what the significance is and how it may effect the success of applying this information in achieving accessibility for some users within the context of the topic of the week
- Challenges can be significant
- Opportunities can be significant
it is expected that significance provides a good reason why we should discuss this challenge or opportunity in class

suggest what should be done about this challenge or opportunity

this can include suggestions such as:

there is an additional factor that is important to consider [identify and explain it]
there is a good way / technique to deal with this concern [identify and explain it]

it is expected that your suggestion includes

a summary of what you know about this the answer to this challenge or opportunity (beyond the reading)

some support for ideas that are not totally obvious, such as useful references or examples to support your suggestions

suggestions should be the starting point for our discussion in class

It is highly recommended that you use the critique template linked here to ensure that you do not miss some parts of the critique.

[Further detailed discussion of individual parts of a critique is provided below in Guidance on Creating Good Critiques.]

Each critique is marked out of 10.

Getting a better mark than a 7 generally requires going significantly beyond just meeting the basic requirements.
Individual items (challenges or opportunities) may get either a 0, 1, or 2.

Items that are not significant or that repeat other items get a 0.
Items of weak significance or where the suggestions are not strong enough will typically get a 1.
The best an incompletely documented item can get is a 1.

Getting full marks on a critique is based on having at least 5 strong critique items.

Critiques may go beyond the basic requirement of discussing 5 challenges and/or opportunities (the best 5 will be marked).

Bonus marks may also be awarded to items with exceptional insights and/or helpful suggestions and/or good references that support a suggestion.

Examples of strong critique items from another course include (each of these items would be worth 2 marks so 5 items of this quality would get the full 10 marks available):

1. Example of a Challenge:

Identification

Type: Challenge
Name: Accommodation vs. Accessibility
Location: Ch x.y.z title of section paragraph 2. It is said that, "Accessibility is a need and a right of all people."
Issue: It is generally impossible to develop a system that can meet all the accessibility needs of all people.

Significance:

If all people need and have a right to accessibility, there is a need to find alternatives to making systems perfectly accessible.

Suggestion:

[an unacceptable suggestion (that will result in 0 marks for the critique item) because it is directed to the author of the chapter and not towards what we will discuss in class]

The chapter should discuss that perfect accessibility is not possible

[a weak suggestion (for a 1 point critique) does not add anything beyond the basic discussion/significance and/or personal opinion]

We need to recognize that while we can strive for accessibility, we cannot ever expect to achieve it fully.

[strong suggestions (for a 2 point critique) suggest a means of satisfying the opportunity]

The concept of "accommodation" focuses on taking extra actions to meet the needs of individuals who find systems to be inaccessible. It places the emphasis on the organization providing the system to find an alternative way to meet their needs. For example, where a system is inaccessible to some users, the organization provides a human contact that the users can use to interact with the system on their behalf.
The Canadian Government has termed this the Duty to Accommodate and provides A General Process For Managers to help them understand their responsibilities in this matter.

2. Example of an Opportunity:

Identification

Type: Opportunity
Name: Diverse Systems
Location: a.b.c title of section this section focuses on computers, tablets and mobile phones
Issue: There are an increasing number of devices (such as smart appliances and Internet of Things (IoT) devices) which are also part of ICT and which have similar accessibility need.

Significance:

Diverse users have accessibility needs of increasingly diverse types of ICT devices. It is also important to provide these diverse users with consistent accessibility across devices.

Suggestion:

[an unacceptable suggestion (that will result in 0 marks for the critique item) because it just repeats what the discussion/significance said]

It is important to apply accessibility to all ICT devices

[a weak suggestion (for a 1 point critique) because it is a personal opinion that is not supported.]

There are likely to be different accessibility needs for different types of devices

[strong suggestions (for a 2 point critique) provides a justification for why the idea is being challenged]

Users interact with IoT devices primarily via the Internet with more traditional ICT devices which are covered by existing accessibility guidance. While smart appliances tend to have more direct interfaces for users to interact with, they could generally have the ability to also interact with the user via the Internet like IoT devices.
While there are not mobile specific guidelines yet, existing ICT accessibility guidelines are considered by W3C WAI to generally apply. The W3C WAI is creating a Mobile Accessibility Task Force to build on its current accessibility standards.

Please remember: Good critique items are intended to take us beyond the paper. It is essential that you all read all of this web page and understand what is required of a critique before you do your critique.

Subject of the Critiques

Chapters 3 -12 all have the same format (where X is the chapter number, and the following are sections within the chapter)

X.1 states the accessibility principle, which provides the scope for the chapter

X.2 provides a brief background as to where the accessibility principle came from

X.3 is the main content related to the principle that was created for the chapter

X.4 provides information that elaborates the accessibility principle

X.4.1.1 provides information on specific user accessibility needs related to the principle - these user accessibility needs attempt to be comprehensive

While X.3 provides discussions related to many of these user accessibility needs, it is not organized according to these needs and might not cover all needs.

X.4.1.2 provides references to various international standards that provide guidance related to the principle

This is general guidance, and is not intended to provide references to individual research papers, which are more appropriately discussed in X.3

X.4.2 discusses questions that could aid in applying the principle and some of the user accessibility needs

The discussion of these questions attempts to provide guidance on how to investigate various user accessibility needs.

X.5 provides suggestions for Accessibility Demonstration Experiences that can help you to internalize the principle and that are generally the basis for the assignments

Only sections X.3 and X.4 of each chapter are eligible as subjects for a critique.

NOTE: The section "X.3 Understanding the Principle" has been subjected to repeated critiques, making it somewhat difficult to come up with good new critique items. It is important that critique items focus on significant challenges or opportunities and not just on further details or examples that do not add any major new understanding to the section. Also material in this section discusses a variety of issues without necessarily relating each of these issues to accessibility. However, your critiques must relate to accessibility and it is not suitable to just explain how something in this section relates to accessibility where such a relationship is not explicitly discussed in the section.
The sections "X.4.1.1 User Needs Related to the Principle" and "X.4.2 Questions to Aid in Applying the Principle" have been subject to much less critiquing and might be easier to find good critique items to present. However, the location within a chapter for your critique items is up to you. While these critiques should not just restate ideas from section X.3, they can refer to them as part of justifying why they are important. Again, it is important that critique items focus on significant challenges to the section and not just on adding further details or examples without any major new understanding to the section.

Good critique items relating to "X.3 Understanding the Principle" could be focused on

Challenge: by explaining why there are limitations or problems with one of major topics (sub-sections) points (within sub-sections) that are discussed in this section
Opportunity: by identifying significant additional points related to one of the major topics that are discussed in this section
Opportunity: by identifying significantly new topics that belong in this section (especially ones related to a user need that are not discussed)
WARNING: be sure that what you are suggesting is important enough to be worth discussing in class and not just a minor variation on what is already discussed in the section.

Good critique items relating to "X.4.1.1 User Needs Related to the Principle" could be focused on

Challenge: (it would be difficult to create a challenge to these needs, which have been developed by a large number of international accessibility experts)
Opportunity: by identifying a significantly different additional (new) need that best belongs to this principle
Opportunity: by identifying a significantly different additional (new) instance of a need

Good critique items relating to "X.4.2 Questions to Aid in Applying the Principle" could be focused on

Challenge: by explaining why there are limitations or problems with one of the suggested ways of dealing with a possible answer to the question
Opportunity: by identifying a significant additional answers or ways of dealing with an answer
Opportunity: by identifying a significantly different additional (new) question (relating to a need not covered by an existing question)along with a start at discussing how to answer the question

Guidance on Creating Good Critiques

Remember the purpose of the critique

Critiques are intended to help us to discuss the topic of the week

This means that comments of an editorial nature are not applicable: critiques should be about important points that are worth discussing in class.

Critiques are not summaries of the chapter

It is expected that we all know what the chapter said. Please focus on challenges and opportunities.

Challenges occur when you disagree with major points that the chapter presents. It is not enough to just say you disagree.

You need to be able to explain why you disagree (in the significance),
and it is important that you can present a well reasoned alternative (in the suggestion) which can come from

your own experience
or from doing a little research into the area (this should include one or more references where applicable)

Opportunities occur when there is something important missing. It is not enough to just identify that there is something missing. You need to take the initiative to try and find/suggest it.

If a term is important in understanding the chapter, then at least google it or look in Wikipedia. Before making this the basis of a critique item, be sure that what you have found is really important.
If the discussion of some idea leads you to think about some idea that is related to or is missing from the discussion, you might have a good critique item by telling us about it (if it is important to the topic of the week). You might be able to tell us about this from

your own experience
or from doing a little research into the area (this should include one or more references where applicable)

Critiques (including challenges) are not just negative criticism

Negative criticism just aims to tear something down by finding problems / faults. It is easy to do since we live in an imperfect world.

Positive criticism starts with finding problems but then goes on to find solutions. It helps to make the world a better place.

Critiquing is important

Critiquing helps to develop critical thinking, which is especially important in the development of new / improved systems.

Remember: Critiques are a major component in this course. While it is often possible to come up with a number of good critique items from your own experience, it is also expected that you will do further research if necessary to learn about the material. In addition to using Google and Wikipedia for fast answers, Google Scholar is a good source of information on scholarly papers that could provide further information. However when you use Google Scholar, many of the papers it finds cannot be directly downloaded. You then need to login to the UofS library with your NSID and either search for an online version of the journal or search in one of the publisher databases. The databases most likely to have material of interest to us are:

A procedure for developing critiques

That means that you first have to understand the chapter. Here is a suggested procedure:
Throughout the process it is important to keep in mind accessibility and the topic of the chapter.
Critiques need to avoid repeating ideas from previous chapters or getting sidetracked into general usability issues and forgetting about accessibility.

Read through the chapter once, noting possible challenges or opportunities.

You could do this by using a highlighting sections or starting to fill the identification section in a number of critique templates.
Don't go any further until you have gone through the entire chapter (as the item you are concerned about might be discussed further later).
And don't limit yourself to the first five issues you encounter.

Evaluate the quality of your potential critique items, reviewing what the topic of the chapter is about.

Don't just pick the first five that you though of, especially if they are all in the background section (x.2) of the chapter.
This doesn't mean that you can't use background material in a chapter as a source of a critique, if you have an important critique item to discuss, it just means that you should also consider the rest of the chapter and focus your critique on sections after the first two (as discussed above in subjects of the Critique)

Now go through the chapter a second time, and fully develop your critique items as you go.

Use the critique template to create a critique in Word or rtf format.
Be sure that you put appropriate answers in each of the identification, significance, and suggestions sections of the critique.
Follow the guidance from this Web page to make sure that you have answered each of these with the type of answers that I will be looking for.
Remember that you are to go beyond the chapter, so if you don't have personal experience to help you do this, you might need to do a little on-line research to develop a suitable suggestion.

Finally, you should look over all your critique items, before submitting your critique.

Check that the items are good and well worded, and revise if necessary.
Check that you have put suitable information in each of the parts of each critique item.
Check that multiple items don't say substantially the same thing, and if they do combine them into a single item.
Check that you have enough good points ( 5 is the minimum for full marks).

Make any necessary revisions before handing your critique in

Remember, I won't be auditing that you followed this procedure, but I will be evaluating the quality of your critiques, whatever procedure you follow.

Remember the data structure of a critique

It can be more difficult to put data into a set of attributes than the data structure developer first expected, especially where you don't take into account how the attributes will be used. So here's some insight on the use of each component of the critique.

Identification

Name: - is something that we can use as a primary key. It can give me an idea of what your critique item is about. I can use it to provide you feedback on your critique items.

Type: Opportunity/Challenge - these are the only two allowable values. It is important that your suggestion is consistent with this type. You might want to reevaluate your type once you have completed the critique item.

Location: I use this to find the appropriate location(s) in the chapter.

Please provide at least the section/subsection number(s) where the challenge occurs/ opportunity belongs.
This should be as specific as possible
If you are focusing on a specific paragraph, sentence, or word, please make this clear, such as by providing a quote of the sentence or phrase where the critique applies the most or starts to apply the most.

Issue: This involves explaining what led you to identifying this opportunity or challenge and then briefly stating what is missing or incorrect.

This should briefly but also fully identify the issue that you are critiquing
This should start with a quote from the chapter, or a summary of what the section/subsection of the chapter said
Then briefly introduce the point you will be making in your suggestion (without helping relate it to what the paper said, leaving this linkage to the Location attribute).
Consider if you are in my place, what would help you to understand the basis of this opportunity or challenge. It also helps to make sure that you didn't misunderstand what the chapter was trying to say.

Significance: this is largely intended to get you to be sure that your opportunity or challenge is significant.

This must address why the challenge or opportunity is significant for accessibility

because only some users need them in order to be able to use the system or because some users might be discriminated against if it isn't accessible
and not just that it is generally useful for increasing the usability for all users.

This should also address how this challenge or opportunity is significant to the topic of the chapter

and doesn't just repeat material already there

The significance needs to be clearly explained. It isn't enough to just say that you think it is important.
Until you get into a good feel for what types of critique items are truly significant, you should check the significance section in each of your critique items against the requirements.
Please note: it is not enough just to say that something is wrong or missing. We all make little mistakes. You need to be sure that each of your critique items is important enough that it is worth discussing in class.

Suggestion: This is the main point of doing a critique. If you don't make a good suggestion you will only get one point out of a possible two.

Suggestions provide us the basis for going beyond the paper.
Thus they need to be more than just a question or request for further information.
While a question is important to start discussion, I also expect that you have tried to help us answer the question and that either your answer will enlighten us or lead us to further discussion.
As discussed earlier, this may come from your personal experience (if you can provide a concrete example to support the suggestion) or may come from doing a little research on your own (where you provide a reference and possibly a quote to support the suggestion). Either can lead to good suggestions.

If you didn't like/understand the chapter, you might want to find an alternate viewpoint

Sometimes, it is easier to understand some points in the chapter if you put them into context by also considering other author's viewpoints. (While this is not required or expected, it might be of interest or of use to some of you on occasions where you are having major difficulties with some content.) You can also search for other papers on the topic to help you get an understanding of the general area. If you find a good paper, please let me know. If you find ideas that directly relate to those in the assigned paper, they may suggest some good critique items.

Avoid the following problems in your critiques

The following are a collection of problems that have been accumulated from previous classes:

General problems:

Not providing all the components of a critique. The critique template can help to avoid this problem.
Not providing the types of answers that belong in the different components of the critique. By providing information that belongs in another component of the critique, it is likely that the same information will be provided multiple times and that some of the required information will not be provided. Reviewing your critiques before submitting them can help with this problem.

Problems with location:

The location must clearly identify where in the chapter that the critique applies to. Even if you are suggesting a new topic, you can suggest where you think that it best fits.
The location needs to be appropriate. An idea that you identified while reading one sub-section of the chapter might be covered or better belong in another subsection.

Problems with issues

The issue needs to provide sufficient discussion of what is wrong or missing. It needs to be clear that this is something other than what the text already says.

Problems with significance:

Critiques for this class must focus on accessibility. While much of the content in section X.3 does not specifically focus on accessibility issues, it is included because it is of some importance in dealing with accessibility and user accessibility needs.
Critiques must focus on the topic of the week. It is not enough that a critique deals with accessibility, it also needs to deal with the topic of the week (as discussed in the chapter)
Critiques must go beyond the chapter. The significance should explain why what you are suggesting is significant beyond what the text says, it should not paraphrase the text.
Critiques need to focus on a big issue or a practical suggestion. the focus should be in telling us something more that we should know. Please read the materials carefully to ensure that you are not repeating something already in them or misinterpreting what is being said. Suggestions should lead to increases (not decreases) in accessibility (including our understanding of accessibility).
Significance is not just a continuation of the discussion. The discussion provides the background and identifies a problem, while the significance focuses on why that problem is important.
Significance is different from a suggestion. Significance focuses on discussing the importance of the challenge or opportunity, while suggestion goes towards answering the challenge or opportunity.

Problems with suggestions:

Critiques must make suggestions that adds and discusses new material beyond what is in the chapter. It is not enough to suggest in the critique that new material be found.
Suggestions must present the new material to the class. They must not be suggestions to the author on how to rewrite the chapter.
Suggestions require sufficient backing to support that they are valid. “I think” is not good enough – you need to prove your point somehow. Please remember that there are some issues that are not worth arguing about.