Your job is very simple! All you need to do is look out for the features that mark out sprial and elliptical galaxies. In fact, as you’re a human and not a computer, most galaxies should be easy to classify since they’re obviously spirals or obviously ellipticals. On this page, you will practice classifying galaxies. On the next page, you will take a short trial to test your skills. If you don’t pass the trial, you can try again. Once you pass the trial, you can start contributing to Galaxy Zoo science!
—Galaxy Zoo

I got a 13 out of 15 score on the qualifying test.

I gather that astronomers have developed software that chops telescope images up into chunks centered on bright objects, and that people are better at identifying the bright spots as clockwise spiral galaxies, counter-clockwise spirals, oval galaxies, or something else (like a star or a satellite). Or, to be more precise, the effort it would take to create a computer program that surpasses human skill is not worth investing, when it is possible instead to get an army of volunteer humans to do the work. (This is called “Flintstoning,” which refers to the usefulness of certain low-tech solutions, such as Fred using his feet underneath the car.)

For a long time, I have daydreamed about making a computer program that asks students to evaluate short writing samples, such as thesis statements or news leads. The idea is that I would first train students with samples that I have already graded. Then those students who have demonstrated that they are good at evaluating the writing samples (that is, those whose answers closely match mine) start being asked to evaluate samples written by their peers. I imagine that a particular student sample would get evaluated by at least two peers; if the peers disagree, then the system would flag that sample, and I would break the tie.

I wouldn’t use such a system to mark assignments for a grade — my thought that it would be useful for the kind of assignment where students get credit for simply doing the work, and the added value would be that students would get some kind of structured peer feedback. I would make sure to read and comment on at least a handful of exercises in each set, and this system would help me identify which students are not only doing a poor job writing their own exercises, but are also unable to recognize when a sample exercise meets the criteria.