CMPT 480 / 840 Accessible Computing

Guidance on Creating Good Critiques

After submitting your 1st critique, some of you caught on right away and some of you need some more guidance. It's now time to analyze what you did and what you should do to do better:

Remember the purpose of the critique

Critiques are intended to help us

This means

that you do not write critiques as comments to the authors: they have already published their paper and will not likely ever see your comments.
that comments of an editorial nature are not applicable: critiques should be about important points that are worth discussing in class.

Critiques are not summaries of the paper

It is expected that we all know what the paper said. Please focus on challenges and opportunities.

Challenges occur when you disagree with major points that the paper presents. It is not enough to just say you disagree.

You need to be able to explain why you disagree (in the significance),
and it is important that you can present a well reasoned alternative (in the suggestion) which can come from

your own experience
or from doing a little research into the area (this should include one or more references where applicable)

Opportunities occur when there is something important missing. It is not enough to just identify that there is something missing. You need to take the initiative to try and find/suggest it.

If a term is important in understanding the paper, then at least google it or look in Wikipedia. Before making this the basis of a critique item, be sure that what you have found is really important.
If the discussion of some idea leads you to think about some idea that is related to or is missing from the discussion, you might have a good critique item by telling us about it (if it is important to the topic of the week). You might be able to tell us about this from

your own experience
or from doing a little research into the area (this should include one or more references where applicable)

Remember: Critiques are major assignments in this course. While it is often possible to come up with a number of good critique items from your own experience, it is also expected that you will do further research if necessary to learn about the material. In addition to using Google and Wikipedia for fast answers, Google Scholar is a good source of information on scholarly papers that could provide further information. However when you use Google Scholar, many of the papers it finds cannot be directly downloaded. You then need to login to the UofS library with your NSID and either search for an online version of the journal or search in one of the publisher databases. The databases most likely to have material of interest to us are:

Critiques are intended to go beyond the paper

That means that you first have to understand the paper. Here is a suggested procedure:

Remember what this class is supposed to be about and remember the topic for the week. Good critique points need to address the class and the week's topic area.
Read through the assigned paper once, noting possible challenges or opportunities. You could do this by using a highlighter on a paper copy or starting to fill the identification section in a number of critique templates. Don't go any further until you have gone through the entire paper. And don't limit yourself to the first five issues you encounter.
After reading the paper, you should consider why the paper was selected for this week of this class.
Review what the class and weekly topic are supposed to be about. Evaluate the quality of your potential critique items. Don't just pick the first five that you though of, especially if they are all in the background section of the paper. While some papers are included because they have a good comprehensive analysis of the topic in their background, most papers also put forward new ideas that go beyond this background, and we want to go beyond that. (This doesn't mean that you can't use background material in a paper as a source of a critique, it just means that you should also consider the rest of the paper.
Now go through the paper a second time, and fully develop your critique items as you go. Remember that you are to go beyond the paper, so if you don't have personal experience to help you do this, you might need to do a little on-line research to develop a suitable suggestion.
Finally, you should look over all your critique items, before submitting your critique. Check that the items are good and well worded, and revise if necessary. Check that multiple items don't say substantially the same thing, and if they do combine them into a single item. Check that you have enough good points ( 5 is the minimum for full marks).

Remember, I won't be auditing that you followed this procedure, but I will be evaluating the quality of your critiques, whatever procedure you follow.

Remember what the class is about

I know I already told you that above, but each paper introduces a lot of different ideas that while potentially interesting are outside the scope of the course and/or the topic of the week. Some of these ideas may even be of more interest to you than the topic of the week or the course. However, only critique items related to the course and the topic of the week will be considered. (We don't have time to consider the others in class - if you want to discuss them with the prof, please do so outside class.)

Remember the data structure of a critique

It can be more difficult to put data into a set of attributes than the data structure developer first expected, especially where you don't take into account how the attributes will be used. So here's some insight on the use of each component of the critique.

Identification

Name: - is something that we can use as a primary key. It can give me an idea of what your critique item is about. I can use it to provide you feedback on your critique items.
Type: Opportunity/Challenge - these are the only two allowable values. It is important that your suggestion is consistent with this type. You might want to reevaluate your type once you have completed the critique item.
Discussion: This involves explaining what led you to identifying this opportunity or challenge. Some people provide a quote from the paper, some people summarize what the paper said, and some people just talk about the point they will be making in their suggestion (without helping relate it to what the paper said, leaving this linkage to the Location attribute). Consider if you are in my place, what would help you to understand the basis of this opportunity or challenge. It also helps to make sure that you didn't misunderstand what the author was trying to say.
Location: I use this as an external key to find the appropriate location in the paper (if I don't remember it based on your discussion). Different people give me varying amounts of data, e.g. page number, section heading title, paragraph number within the section, sentence number within the paragraph, all of the above. Consider if you are in my place, what would help you to quickly identify the area of concern in the paper.

Significance: this is largely intended to get you to be sure that your opportunity or challenge is significant. You need to refer to the discussion provided in the main critiques page. Until you get into a good feel for what types of critique items are truly significant, you should check the significance section in each of your critique items against the requirements.

Please note: it is not enough just to say the paper is wrong. We all make little mistakes. You need to be sure that each of your critique items is important enough that it is worth discussing in class.

You will also note that I did not require you to choose from a list of values (e.g. the individual bullets from the discussion on significance) like I do for type. This is both because there may be multiple values, or you may come up with some value (reason for why your critique item is significant) that I have not mentioned (identified in advance) but that is still valid.

Suggestion: This is the main point of doing a critique. If you don't make a good suggestion you will only get one point out of a possible two.

Suggestions provide us the basis for going beyond the paper. Thus they need to be more than just a question or request for further information. While a question is important to start discussion, I also expect that you have tried to help us answer the question and that either your answer will enlighten us or lead us to further discussion. As discussed earlier, this may come from your personal experience or may come from doing a little research on your own. Either can lead to good suggestions.

Remember that the paper was written a while ago

Don't complain about them being out of date

In fact some papers may seem really ancient to you. That doesn't mean that they do not have value in them. If anything, it provides you opportunities to challenge them if major ideas they present are no longer valid.

Focus on the main ideas, not the details

Technologies, companies, applications, and Web sites come an go. As such they are likely to fall into the category of merely editorial details unless their passing is of significance to the topic we are discussing. e.g. VisiCalc (the original spreadsheet application program) has been replaced by many different products. However its replacement was more due to business competition and buy-outs than in terms of major new technologies). So the fact that it is no longer and that Excel now has the market is only of interest to the topic of "keeping control of your intellectual property". In any other context, the fact that it is no more and that Excel has replaced it is just an editorial detail.

Consider how things might have changed since the paper was written

Changes over time can be very important. There are at least two types of change:

evolutionary change (where things keep changing in the same direction that was based on what was previously happening and thus might have been foreseen foreseen by most people as a logical addition to the current)

e.g. databases got bigger along with more use and the availability of bigger disks
e.g. e-r models evolved into object models by adding information about operations to information about data
e.g. object models led to greater consideration of the semantics of data

revolutionary change (where something happened that changed direction, or created new directions to add to the old and thus surprised many people when it first occurred)

after many years of data being kept as securely as possible within an organization, the Web focuses on sharing data and breaking down boundaries
after many years of the focus of computers on processing transactions between a user and the computer (data processing), a new social computing focus is on computers being used to help individuals communicate

Of these two, revolutionary changes are of the greatest interest, because they have the most significant impact and because they require us to develop new techniques to handle them. (With evolutionary change it is easier just to extend what we already know).

NOTE:The discussion of these two categories is not definitive. It is merely provide to try and get you thinking about areas where you can identify the most significant challenges and opportunities.

And if you didn't like/understand the paper, you might want to find an alternate viewpoint

Sometimes, it is easier to understand one paper if you put it into context by also considering other author's viewpoints. (While this is not required or expected, it might be of interest or of use to some of you on occasions where you are having major difficulties with an assigned paper.)

For each week you are only required to critique the first paper in the list of readings. It was selected to introduce the topic, The other papers also have something to add to the topic. Looking over them quickly might give you further insights that make it easier to understand and critique the assigned paper. However, don't just create a bunch of critique items that say that the author didn't deal with the contents of the other papers.
You can also search for other papers on the topic to help you get an understanding of the general area. If you find a good paper, please let me know. If you find ideas that directly relate to those in the assigned paper, they may suggest some good critique items.

Copyright © 2009 - Jim A. Carter Jr.