Education: November 2005 Archive Page

Turning to a few student samples we can see this concept exemplified even more. Below is a quote from one of Dennis Jerz's students. This is one of the student's first postings, and you can see his/her enthusiasm about designing and creating a blog.

September 08, 2004: LOOK AT ME, I HAVE A BLOG!

I have a blog now. My very own. It's like a baby -- mine to mold, change, and create. Hundreds of people (or in my case, about 5) will be able to see my writings. People from other countries even. Hola! Bonjour! Guten Tag! foreign peoples! I think I may have become addicted though. I just found out how to personalize and change the colors. Now, every free moment I have will be spent changing colors and making the blog uniquely me. Homework? Who needs homework? I have to work on my blog. (Special K)

This student seems to be aware of, probably because Jerz emphasized to his students, the public nature of the weblog: "Hundreds of people... will be able to see my writings. People from other countries even." However, what is even more intriguing is that the student states in parentheses "(or in my case, about 5)," which would seem to indicate that he/she does not think many people will be reading her blog. --Ashley Joyce Holmes --Web Logs in the Post-Secondary Writing Classroom: A Study of Purposes (PDF) (North Carolina State University)

This was a pleasant find -- an NCSU master's thesis that analyzes the use of web logs in higher education. I think I'll throw a few quotes from this into my next annual review.

I'm much better at simply showing people what blogs are all about, so I appreciated Holmes' careful description of precisely how a blog works. I might use a section of this dissertation the next time I teach Writing for the Internet.

My student Vanessa Kolberg is the "Special K" blogger. The paper mentions her name correctly, but the URL given in the Works Cited list is wrong. It should be "http://blogs.setonhill.edu/VanessaKolberg".

Researchers unveiled a prototype of a $100 hand-cranked laptop computer on Wednesday and said they hoped to place them in the hands of millions of schoolchildren around the globe.

About the size of a textbook, the lime-green machines will be able to set up their own wireless networks and operate in areas without a reliable electricity supply, MIT researchers said at a United Nations technology summit.

--$100 laptops aim to bring children the world (Seattle Times)

This story broke during the week my blog was down, so I'm late to this party. One of minute of cranking is supposed to provide 40 minutes of computer power.

By studying journalism, you carry with you tools for assessing arguments, and a dogged determination to find the truth in yourself and in others.

I love this work, but it is work. Living up to the standards of this difficult, competitive field is taxing. I have a long, long way to go. --Amanda Cochran --A journalist's lessons (Girl Meets World)

Amanda is a junior journalism major here at SHU. I've asked her to speak to my "News Writing" class today.

One sunny day shortly after the start of the fall term, Robert M. Dawley was preparing to spend the afternoon tutoring two students on how to measure the DNA content in the cells of tadpoles.

But first he walked briskly out of his building and over to a small lounge with white cinder-block walls, stretched out on the carpeted floor, his head propped up on a bent arm, and asked a small class of bright-eyed freshmen sitting along the walls to reflect on Gilgamesh, the world's oldest-known epic poem. --Burton Bollag --Where a Geneticist Can Teach 'Gilgamesh' (Chronicle)

Serious Games Summit DC 2005, Day 2A Debate Between Jan Cannon-Bowers and Marc Prensky (Jerz's Literacy Weblog)

Do we need instructional design in serious games, or is making a good game enough? This debate is part of an ongoing turf battle within the serious games movement.

As is generally the case with conference liveblogging, these are lightly-edited notes, and shouldn't be taken as a verbatim, authoritative transcript.

Perry McDowell gave the introduction. Developer of "Delta 3D," an open-source games engine.

If games can train and educate without putting pedagogy into it, why waste time/money in it?

If games can't train without instructional design, the results can be scary ("negative training effects").

In the 80s, regarding AI, "Mouths made promises that brains couldn't keep." The result was "the AI winter" -- money for AI research dried up as a result of dissatisfaction with early attempts.

In the 90s, the same thing happened with VR. Serious Games is providing a reprieve for VR researchers. But if we can't show hard evidence that SG doesn't train people better, faster, cheaper, then before long Serious Games will be five guys drinking beer remembering the old days, while, the rest of us will be at the "Hula-hoops for Education Conference."

"We don't want Serious Games to be on the scrapheap of history" along with the pet rock. (Hmmm?. if serious games isn't any good, then why shouldn't it be discarded when the next thing comes along? The argument is predicated on the assumption that serious games do work. He's suggesting the issue is to preserve funding for serious games, not to solve the problems that most of us in this room assume serious games will solve.)

Jan Cannon-Bowers and Marc Prensky; two smart, very well-educated people who have almost diametrically opposing positions on the issue.

Contextualizing the start of the deabate, which started at another conference (sorry, I didn't catch that detail).

Marc -- teachers are basically babysitters, under certain circumstances we can get rid of teachers.

Jan -- "Marc, I could almost defend you until you went off the deep end."

The moderator helpfully contextualized the debate with Jerry Springer and Rock 'em Sock 'em Robots slides, and Dan Ackroyd's famous Saturday Night Live retort, "Jane, you ignorant slut."

So clearly, this debate was staged as entertainment, with the two participants ready to attack each other intellectally.

Jan Cannon-Bowers

Do we need pedagogy in educational games? Of course we do! What moron would think otherwise? Why is "pedagogy" a bad word? Gamers are afraid that pedagogy represents this noose we're going to put around their necks and tighten it to crush out all their creative juice. She suggests that it will be an interesting mix and culture clash. Learning doesn't have to be "fun" in order to be successful. Would you say everything you learned was fun? We can all recall cases in which we were so motivated to learn that we overcame bad instructional environments.

Should we try to make learning fun? Sure! If fun is one way to elevate motivation, then why not? Positive feelings about training -- "I liked it or didn't like it." There's a threshold below which no learning is happening, but does learning have to be "fun" in order to be effective? There's a point at which, once a certain threshold of comfort has been reached, there is little correlation between comfort and learning.

Besides fun, we should also consider that training is interesting, engaging, useful, challenging, fulfilling, and otherwise motivating. If we don't incorporate pedagogy, learning effectiveness is hit or miss (at best).

Jan would prefer that her doctor, mechanic, etc, be trained on a solid system that does its job, and she doesn't care whether they have fun in the process.


Marc Prensky

Had difficulty getting his laptop started. Heckled from the audience -- "Are you having fun?" Marc admitted that he should have tested it, and admitted "There's probably pedagogy involved in testing." The moderator jumped up and joked that the debate was over, Jan cried, "I won!"

To fill time, Jan fielded a question from the audience? what's the difference between enjoyable and fun? She replied that that's splitting hairs, but noted that people may go through an experience where they are sweating bullets, and then look back afterwards and say, "That was fun!"

When the laptop was ready, Marc sarted saying, "Sure sure sure sure." Began with a joke definition of pedagogy -- peda = "foot," gogy = "gouge" -- gagging on your foot in your mouth.

"Whenever you add an instructional designer, the first thing they do is suck the fun out."

Showed a list of 12 activities (trying, deciding, observing. etc.). You need motivation in order to get people to do those 12 things.

Will Wright --If a learner is motivated, there's no stopping him.

Effort for learning can feel like work, but can also feel like play. Learning feels like play when you have the engagement, motivation and passion -- and that should be our number one priority. The main reason you do all these things is to get people to finish the training you've invested your energy in.

A good chart

Curriculum Design v Game Design

Focus: Content vs. Engagement
Mode: Presentations vs. Gameplay
Decisions: Rare vs. Frequent and Important

Negative Training -- the instructional designer's way of instilling fear. "the herpes of training"

("Serious Games" is a subset of games for educational purposes. A term for a subset of education that focuses on enjoyment is "Fun Pedagogy.")

Jan of game designers -- "I don't trust you!" (guffaws). When asked to clarify, she said she didn't trust the instinct of game designers to be able to deliver the content that, for example, helps a pilot to fly a plane, without the help of instructional designers.

(Makes me think of Plato's Ion, the dialogue with a rhapsode who says that he is qualified to act the role of a general because he knows what a general would do or say.)

When asked how a pilot in a "fun" simulation would learn the details, Marc said "RTFM." Someone from the audience asked "Who wrote the FM?"

Marc -- we need good instructional strategies. "If they were good instructional strategies, they wouldn't suck the fun out." We've learned a lot about the world of Norrath, such that if there were a real world of Norrath, it could be validated.

Jan -- design is about making decisions. "I don't think the typical gamer who's not gotten some extensive exposure is going to make the appropriate design decisions."

Marc -- suggested that instead of starting from the beginning that instead you started in the middle, repairing an interesting thing that's broken, rather that creating something from scratch without a sense of how it all works.

Jan notes that moving from simulation to games, you need new, modern models of instruction that can accommodate immersive environments.

Ricardo Rademacher, who was sitting behind me in the audience, pointed to the slide on which Marc had displayed "Whenever you add an instructional designer, the first thing they do is suck the fun out," and noted that you could turn that around. "Whenever you add a game designer, they suck the learning out," and argued that's precisely what happened in the 90s with "edutainment software."

In the last few minutes of the panel, amidst much laughter and zipping-up of laptop cases, when Marc tried to get the crowd to join his rallying cry of "Let's not suck the fun out," Ricardo and I cried, "Let's not suck!"

In a parting shot, the moderator asked the crowd how many of them had changed their minds based on what they had seen. Predictably, nobody raised their hands.

Overall, I thought the whole premise -- do we need pedagogy at all -- gave Jan an uphill battle. Imagine, if you will, in the alternate universe, instead of "Serious Games Summit," a "Fun Pedagogy Summit."

What chance would a game designer have there?

Well, that's it -- time to publish this and head over to the reception. I'm driving back to Pennsylvania tonight.

Serious Games Summit DC 2005, Day 2Derek Wischusen: Expanding Teachable Moments in Serious Games using AI (Jerz's Literacy Weblog)

Began with a useful grid that recognizes that realism is not the only way to evaluate a game, and that realistic games are not the only kind of training games out there.

Wischusen presented two binary pairs:

Fun/Training
and
Realistic/Unrealistic

Goal: Fun -> Training

(I like the recognition that realism is one option, but there are many ways to be “unrealistic,” just as there are many ways to be nonwhite or un-American. Expressionism, impressionism, romanticism, abastraction etc. are all artistic alternatives to realism.)

Key messages:

Training isn’t “just in time” or “just in case”
Simulation isn’t training
Networks, virtual environments, games, process measures are not training


Training is “relevant practice with feedback”

Assessment and Feedback

Simulation/game provides the practice environment. Practice alone does not make perfect. Intelligent assessment and feedback is essential for effective training to occur.

Perfect practice makes perfect. To learn how to do it correctly, you need assessment and feedback to tell you when you’re not doing it correctly.

While the good instructor is always the best source of training, there are rarely enough instructors to meet training needs; and never enough to provide one-on-one training.

(Hmm… A embedded assumption is that one-on-one training is best.. but “best” is a value statement. If you factor in ROI, then one-on-one instruction may be inefficient. Are all instructors “good” at one-on-one training? The reference to a “good” instructor is sort of begging the question. )

The system can do pre-testing to identify the knowledge you bring into the system, personalizing the delivery according to the needs of the learner.

“Cognitive fidelity is as important as graphical fidelity for both training quality and end-user acceptance.” (Hmm… that phrasing privileges graphical fidelity, but the speaker noted that not all training simulations requires that kind of fidelity.)

Presentation of Virtual Interactive Pattern Environment and Radiocomms Simulator (VIPERS) – an Air Force Research Laboratory project designed to give student pilots practice performing a labor-intensive landing procedure. Typically simulated with a duct tape pattern on the floor, with the students calling out their status and responding to commands as the routine requires. A technique better practiced on the ground, but students have limited access to fancy simulators.

(Hooray – in an aside the presenter noted that 2D environments can provide just as good results as rich 3D simulations, and cautioned the assumption that realistic graphics are always the best option.)

Synthetic Teammates for Realtime Anywhere Training and Assessment (STRATA)

DARWARS – offers a massively multiplayer online war, that permits different training systems to plug into it. Emphasizes “headwork” (teamwork, not flying skills).

Giving the trainee a situation that contains an error; will the trainee note that the setup information contains an error that makes accomplishing the goal impossible.

AI agents should occasionally make errors, since people do make errors in the field. That’s part of the “realism” in the training environment.

Virtual Environment Cultural Training for Operational Readiness (VECTOR)

Building a 3D environment that teaches soldiers to interact with people in a foreign environment, so that the locals are more interested in helping the soldiers than hurting them. The AI agents respond to trainee actions based on the emotional state of the agent, according to rules determined by cultural subject-matter experts. Within the exercise is a synthetic instructor, who gives objectives, provides feedback on both military and cultural protocols. The training involves interacting with the locals, getting important information while following cultural rules.

VIGILANCE – HazMat training, skilled support personnel (SSP) who assist the first responders. They may operate heavy machinery. The concern is that our SSP haven’t been trained to assist an EMT during a HazMat response.

Serious Games Summit DC 2005, Day 2Sande Chen and David Michael: Roundtable -- Beyond Q & A: Assessment Methods for the Next Generation of Serious Games (Jerz's Literacy Weblog)

These are my very rough notes. Some of the speakers didn't give their names, sometimes I couldn't hear their names, and sometimes they gave their names while I was furiously typing something else.

The moderators are authors of Serious Games: Games that Educate, Train, and Inform

Presenters began this roundtable by noting an opposition between “Abigail McGillicuddy,” the traditional teacher, where the buck stops, and “HAL.”

Microsoft employee: emphasized the fuzziness of knowledge as the future, as opposed to a prescriptive system that records a single correct answer. In games, knowledge is treated as binary – right and wrong. Teachers, on the other hand, recognize few absolutes, emphasizing contextualization.

Human has to be there for the time being, but technology will

Richard Carey, Pearson Education. Pre-assessment to level a student into the gameworld; in an educational context you want to place the student right away into the proper level, rather than starting them out too easy. Getting the “picky teacher” to use the software.

Does “test” and “assessment” mean the same thing to everyone here?

John Fairfield, Rosetta Stone. Think of placement rather than assessment.

I noted that testing is just one method of assessing; that assessment can be done without any kind of test.

Eric Lauber, IUP – noted that the simulation “looks more real” but that we don’t have much evidence of transferability of what’s learned in one context to a different context.

Technical consultant – “What I care about is what is this information used to sell these games,” that is, convince people that they want to use it (even if it’s given away for free). A technical consultant needs better data, “more backend.” Won’t be able to convince medical clients to use those games unless he can provide longitudinal study data that shows improvement.

David Gibson – designing simulation that helps teachers deal with kids; wants to be able to represent the intellectual growth of simulated students. Noted that Vermont went to a portfolio method for assessment. Noted Ron Stevens’ work on transference in case-based learning, doing neural network analysis on problem solving, making early predictions of the strategies students are going to use. Also referred to research on the automated SAT, so that you don’t have to keep asking them the same kind of question once you’ve determined what the student knows.

Susan McLister, editor-in-chief of a technology magazine. Noted a major challenge in education is to convince teachers that games are not evil, that there is a direct tie-in between tames and what teachers are expected to do. Embedded in a game should be measurements for higher-order learning skills.

Military speaker – one of the longer histories of using games for training. Wants to know when to use a training game, now that they have games that they believe are training. Wants confirmation that they are doing what they think they are doing, what the ROI is versus traditional methods? Improved skill level, improved long-term retention?

Owen: Referred to the “happy path” and “adaptive thinking,” noted the need to create models around not just models and skills, but also confidence.

Completion assessment vs. process assessment.

Margaret Corbit, Cornell: Games are missing a piece in which students showcase and present what they learned in their own ways. Games don’t include a way for students to express what they learned.

Pat Youngblood, Stanford: [Was distracted… missed her point.]

Kids learning in cyberspace all the time… President of the U of Washington complained that media has not filled its mission of educating the masses. The spirit of games does not involve rigid assessment… (at least not about what educators value)

Designer: Can’t find off-the-shelf products that help him do what he needs to do, and assessment is typically one of the problems. Pressure from teachers who want games tied to state curricular expectations.

Mike Gibson: Beyond the learning within the games, how is it affecting the user’s ability in the real world. (A selling point.)

Another speaker – be sensitive to demographics of the audience. Want to be thinking about what you want to assess – a skill, knowledge, cognitive ability. Simulators typically do very badly in training people. Referred to data that says simulators have negative transfer training.

Microsoft: When it comes to assessment, be careful not to think that what we define as valuable data will be adopted by the education community. Embrace whatever the standars are that the education community accepts – pointed to “No Child Left Behind” (“love it or hate it”). While teachers prefer the qualitative model, there’s a need for quantitative methods. Instead of testing against content, assess the individual user’s strategic abilities – are the individual learners improving over time. Did they use the same strategy over and over in new situations that kept leading to failure, or did they adapt their strategies based on what they learned? Useful information for the teacher to determine where a particular student is falling behind.

Cynthia Phelps, School of Information Science in Houston. Are we missing the difference between assessing our learners, and assessing the learning environment (that is, the game). Instead of just assessing how it applies to the learner, how can the game design itself be iteratively improved.

Kip Carr, Lockheed Marketing. Asked how many in the audience play videogames (plenty of hands) more than three hours a week (about a quarter of the room). Noted that gamers are being assessed all the time without knowing it.

[I then asked how many don’t play – about a quarter raised their hands, but others looked uncomfortable. A quarter is probably a low number.]

Jake Troy, doing foreign language games. Games reach people who wouldn’t ordinarily do the training, when they have the choice of watching TV or doing something else.

Robert ? Health care. About games assessing the player – what kinds of methodologies are game developers using.

Chen referred to a Gamasutra article on how games assess players, and noted that a history of cheat codes and walkthroughs.

Karen from Ideas dismissed the idea that cheating is a problem. Teachers want their students to learn, they aren’t really interested in what they do to learn.

I spoke out in favor of the idea that game that can be “cheated” – such that the learner can get around the learning task and get the goal without having learned -- isn’t a well-designed game. A game that rewards unethical behavior is a simulation to train people how not to get caught, that’s bad, but if the game is designed with reference material, and the game gives the student a motivation to consult the reference material, that harnesses the student desire to “cheat” when bored with gameplay.

Game “cheat books” – the new textbook.

Microsoft noted that learners cheat when bored. [That info could be used to spruce up boring areas of a game, or to collect information on whether the student has really internalized the concept.]

Marine Corps: Game is a tool to be used by an instructor. In a training exercise where the instructor notices a leaner is using a cheat sheet, the instructor can “lovingly tell him” not to cheat during the exercise.

I noted that the whole concept of a game is a kind of cheating, in that you get to save and reload and try again. If the point system is closely aligned with the pedagogical goals, there’s little benefit to be gained from “cheating” (in the sense of bringing a cheat sheet that says what the “right” answer is). But if a student in a game has to keep turning to a particular page, whether on a cheat sheet or in an in-game reference tool, in order to get information that the other students have internalized, then that “cheating” won’t help the leaner complete the game objectives.

A speaker from Finland noted that she designs collaborative games, in which she doesn’t consider working together to be cheating.

I noted that the whole concept of a game is a kind of cheating, in that you get to save and reload and try again. If the point system is closely aligned with the pedagogical goals, there’s little benefit to be gained from “cheating” (in the sense of bringing a cheat sheet that says what the “right” answer is). But if a student in a game has to keep turning to a particular page, whether on a cheat sheet or in an in-game reference tool, in order to get information that the other students have internalized, then that “cheating” won’t help the leaner complete the game objectives.

A series of comments on collecting data from multi-user online games. These games can generate so much data that it can be prohibitive.

Microsoft Howard: Designers should talk to teachers about what precisely they want to measure, munging the data stream and delivering information that is meaningful to teachers.

Owen: Find the “happy path” of the assessor, rather than focusing on what the learner learns. Build the assessor’s needs into the system.

Multiplayer games are good at collecting and storing data. That computer can do pre-analysis.

(Continuing...)