Marking assignments electronically

Continuing with the theme of assessment and evaluation,  I thought I'd go off on a small tangent and explore two options for marking assignments that are submitted electronically. These could be useful in f2f as well as online courses. I will be trying electronic submission with all of my students this fall, instead of just in my online courses. I like the idea of paperless assignments!

You can use a Tablet to grade papers - this allows you to "hand-write" your comments on the screen. This is more like the experience of grading papers that anyone over a certain age is used to - it retains that "natural" feel that some instructors don't want to give up. However, the tablet is one more thing to carry around (although you could trade your PC for a tablet PC). You might also suffer from very bad handwriting - and your handwriting might even be slower than typing. So the tablet has pro's and con's.

Another option for marking assignments that are emailed as documents created in Word, is to use Word's Review feature to make comments, like this:

If you click on the Track Changes icon, commenting is even easier - just double-click where you want to change something and type:

One potential problem with Tracking Changes is that if you email the Word document back to your student with your changes, they can then "Accept" all of the changes without physically having to make the changes themselves - not a teachable moment. A better option is to save your graded file as a PDF, and email that back to the student, who can then make the changes in their Word file. PDF also means that the student should not have any compatibility issues if you use a different version of Word for example.

I recently learned about a new tool that looks like a great time-saver for grading in Word: Phrase Express. PhraseExpress allows you to create shortcuts for common phrases that you use frequently, thereby automating the grading somewhat. For example, if you often use the phrase "singular-plural confusion", you could create a shortcut, like, #sp, and whenever you type the shortcut, the entire phrase appears. You can also use PhraseExpress to automate the opening of files, folders, websites, and programs. And it's free. This is something I will definitely take an hour or two to explore further as it looks like a practical and useful program.

Effective online interaction - one piece of research.

Students in online learning engage in a variety of collaborative practices such as threaded discussions, wikis, chats, blog comments, etc. I have the most experience with asynchronous threaded discussions. They are easy to use, and if the number of threads is small, organizationally transparent. As a learner though, I've also seen that when there are multiple discussions occurring simultaneously, they all seem to get watered down a bit. Given that I tend to rely on discussion forums as a way to engage students, introduce content, and provide opportunities for analysis and reflection AND assess participation, it seemed a good time to delve into this topic a little deeper.

I went to PsycINFO to see what research has been reported on this topic. PsycINFO indexes everything that is published in the scholarly literature on psychology - it is beyond enormous. I limited my search to articles available through my institution's library-based access as full-text, downloadable PDFs so that I could read the actual articles, not just the abstracts.

Antonio Calvani, Antonio Fini, Marcello Molino and Maria Ranieri published a 2010 article called, Visualizing and monitoring effective interactions in online collaborative groups, in the British Journal of Educational Technology, Volume 41 Number 2  Pages 213–226. Their goal was to model the effectiveness of group collaboration. They wanted to use the model in a real course to give the online tutor the means to monitor and evaluate online collaboration. They also wanted to improve Moodle's standard Forum module.

The authors address the complexities inherent in trying to describe and evaluate "group effectiveness". One of the problems is that a group can be effective - they work together well, respond in a timely manner, etc - yet mundane, nonanalytical and unreflective in their thinking. Calvani et al addressed two basic dimensions of online collaboration: participation and cohesion.

Participation was composed of several factors:

• the frequency with which individual members posted to the group
• the tendency of individuals to propose discussion material by presenting cues, ideas and hypotheses
• equal participation in all group members
• rotation of roles with the group, i.e. sometimes analytical, sometimes questioning, sometimes summarizing
• regular and constant rhythm of participation

Cohesion was expressed by:

• individuals reading each other's posts
• the depth of the discussions as measured by the number of connected responses to posts
• group discussion and analysis of the arguments and proposal made in discussion
• summarizing contributions that synthesize contributions and provide closure

Calvani et al. measured these factors in the online discussions in a course taken my master's level students. Some of the features of the enhanced Moodle discussion forum that they developed were used to collect data. More on their enhanced Moodle modle and a PDF of the original article are available here.

They created an assessment model that evaluated the group on each factor. A similar scale would look like this, with higher grades closer to 5 and lower grades closer to 1:

A chart like this would provide the instructor with a graphic picture of where a group needs help. It also highlights the need for an instructor to invest time and effort into understanding group dynamics, different levels of thinking, AND assessment. 

This is just one look at the issues around the structure and evaluation of group work in online learning, but this was a useful activity for me because it reminded me that we do not necessarily have to reinvent the wheel - there is a rich empirical literature on online learning. My goal is to read more of it, and to incorporate evidence-based teaching practices into areas like evaluation.

Learning to pogo

In this post I'd like to address the notion of learning theory. I liked the clarity of M. K. Smith's infed article in which he outlines the qualitatively different layers of knowledge acquisition and understanding that characterize what adult learners experience when learning something new. Acquiring facts about something is not the same thing as changing the way you understand the world. A good analogy might be understanding how to use the remote control for your television versus understanding the costs and benefits of television programming to contemporary life, the role of TV in the development and expression of culture, or how personal and collective relationships with technology shape our society.

How we learn is an interesting question because it involves both the learner and the teacher. We could investigate a variety of learner factors: personal strategies like time management; memory strategies such as active versus passive processing; the social relationships with others in the learning community; personal motivation, self-discipline and independence; existing knowledge to connect the new learning to; even timing - sometimes we are just ready to learn something. Then there are the teacher factors: the dynamism and charisma of a teacher; the degree of support and scaffolding the teacher gives the learner; the perspective of the teaching - behavioural, humanist, cognitive, etc; the structure, pace, timing, complexity, etc of the information being taught. For a list of these and related factors see Zemke and Zemke's 30 Things We Know For Sure about Adult Learning.

The factors involved in learning are embedded not just in the individual, but also within their immediate environments (e.g., family, school, neighbourhood) and the culture at large. Technology plays an obvious role here - it can create opportunities or limitations in any of these areas. Technology in some cases IS the teacher.

Learning experiences that work really well for me don't necessarily hit all of the above factors. It may be that a really interesting or charismatic teacher has made something come alive for me, or it could be that my natural curiosity has sent me off on an information-seeking quest. If it's something I HAVE to learn but don't have an implicit interest in, then the more things that come together - both from me as the learner and from the teacher - the better. When I've been learning online, the best resources are those that are coherently designed and complete - either it points me to a set of resources or it at least summarizes a basic point. I'm most frustrated by resources that are insufficient, or poorly structured.


But I know I've learned something when I'm aware of an "aha moment". It's the same feeling I get when I've read a great book, where the author has expressed or described something - an idea, a situation, a thought - so perfectly I can't imagine that it could be done any better.


Aha moments that come to mind:

• Learning how evolution works by reading Stephen Jay Gould's books as an undergrad.
• Troubleshooting, and then fixing, some technical computer glitch (like why Vista won't put my screen to sleep or wake it up).
• Figuring out how to do more than 3 jumps in a row on a pogo stick.

Photo by Ralph Carter. Used with permission.

I used these examples because they reflect different types of learning as described by Smith. All were things I wanted to learn. For me that's key, but it reminds me that students might be learning about something that they don't find so compelling!

What is pecha kucha?

Last week I did a professional development presentation at work in pecha kucha format. What, you're asking, is pecha kucha? It's a challenge - 20 PowerPoint slides, 20 seconds each, no bullet points. The point of pecha kucha (Japanese for chit chat) is to present a big idea, in a highly visual format, and to do it efficiently. The parlour game version of PK is hitting cities everywhere - the last one in Vancouver sold out, to an audience of 1150 at the Vogue Theatre.  The next one in Vancouver - the 11th PK Night - will be held May 26 at the Vancouver Convention Centre.

Pecha kucha started in Tokyo in 2003 - it was created by Astrid Klein and Mark Dytham of Klein Dytham Architects. While it was developed for people working in architecture and design, PK is a format that has endless potential. Here's an example from youtube showing you how PK works in a presentation on signs:


">

My PK was on online learning - specifically, on the intro post in Moodle. I used some of the great ideas presented in an online seminar called Pimp Your Post - Jazzing up Introductory Posts in Online Courses moderated by our own Tracy Roberts - thanks Tracy! I showed 20 images that symbolized WHY the instructor's intro post is important and some options for HOW to make the intro post personal, informative and visual. The images are paired with narrative - the story behind the images is as important as the visual punch. There's no stopping for questions until the end - and the slides are set to move every 20 seconds so you have to be well-prepared and rehearsed. My favourite image from my PK is this one:

Attribution-Noncommercial 2.0 Generic license to stringberd

Now a challenge for you - what point was I making about WHY the instructor should make his or her online intro first, before inviting the students to dive in? What does this image represent to you?

Time . . . monkeys . . . distraction . . . the web

I liked what Julia had to say about the affordances of online learning for deep reflection - having the time to ponder what we're learning about and then responding creates the potential for deep processing. One of the ironies, of course, is that the technology that allows us the luxury of this slow, measured responding (in contrast to the immediate demands of f2f classroom learning) is also a technology that can encourage monkey brain - a term for the cognitive restlessness that results in your thoughts jumping from place to place like a monkey scrambling around in the forest without staying in one spot for more than a few seconds. One of the goals of meditation is to still the monkey mind. 


How does that relate to being online? We are connected to a multitude of places, people, and things simultaneously - how do you stem the flow of all that information that can lead you to monkey mind? Damon Young in today's BBC News Magazine calls ours the distraction society - but he argues that this is not caused by the technology, rather, our digital world aids and abets a natural tendency to seek novelty and reward. He claims that "at distraction's heart aren't silicon chips, but an unwillingness to confront very human issues: pain, boredom, anxiety. Distraction certainly has neurophysiological underpinnings - physical bottlenecks of sense, response and cognition. But these often work because we allow ourselves to be managed by machines' rhythms and logic." 


Damon argues that recognizing and then finding strategies like time management to deal with the affordances of the online world is the key to reducing distraction and confusion.


Today's young learners are natives of the distraction society. How does this affect how they learn? How might this change how we teach?