The terms assessment and evaluation are often misunderstood. For our purposes here we define assessment as simply methods for measuring learner outcomes. Quizzes or exams, for instance, are assessment tools. Evaluation, on the other hand, explains learner outcomes. One example of an evaluation tool would be an online survey that uses questions to determine how effective a certain educational technology is being utilized by online students.

In an online class, a new set of challenges come into play when faculty attempt to use a variety of assessment and evaluation techniques to measure and determine whether or not students are actually learning the material being presented to them.

The obvious fact that students are not physically present poses limitations on a faculty member’s ability to accurately gauge whether or not a student is "getting it" or not. However, there are a number of effective techniques one can use to overcome such obstacles.

Joseph Rosenbeck, associate professor in the Department of Civil Engineering Technology, Environmental Management & Safety, uses a combination of tests and quizzes, individual exercises with a focus on practical applications, peer evaluation, group activities, and threaded discussions to assess online learners.

Rosenbeck assigns point values, that ultimately determine students’ final grades, to every exercise and assignment incorporated into his online classes. He claims that such point systems assure active and significant student participation that in turn helps him understand how much his students are actually learning.

As an example of how the point system works, Rosenbeck begins by assigning the following percentage values in his online Environmental Health & Safety Auditing class:

30 percent Quizzes

20 percent Individual Assignments

40 percent Team Exercises

10 percent Class participation and discussion

The purpose of quizzes is fairly straight forward: "to test knowledge," says Rosenbeck. "I want to make sure they are reading the assignments, watching the video-taped lectures, and paying attention to the threaded discussions. I will formulate questions for the quizzes from all of these areas."

Over the course of one quarter, students in the auditing class take eight quizzes, each worth 10 points. Students can also earn one extra credit point for each quiz if they create two well-written and properly cited multiple choice questions from an assigned topic.

"For the actual practical application of the materials, students do individual assignments where they have to assess problems and come up with solutions," says Rosenbeck. "Then I’ll take some of the individual completed assignments and post them into the student discussion board (anonymously) and have the students evaluate each other’s work (also anonymously)."

Basically, in addition to Rosenbeck evaluating what students are posting to the discussion board, the students are required to submit critiques of each other’s assignments. "It makes them learn from each other and think in a critical manner," he says. "They are rewarded points for how well they provide critiques. I’ll ask several questions, for example, and they have to determine how well they (their fellow classmates) answered those questions." Based on an analysis of their critiques, Rosenbeck can tell how well the students understood the material being covered.

The team exercises portion of Rosenbeck’s class includes one face-to-face element where students come to RIT for a weekend to take part in what’s called an "Executive Leader Session," which is held near the end of the quarter. Other team exercises, which are directly related to the Executive Leader Session, are conducted prior to the students arriving at RIT. These sessions entail actual work in the field over the assigned weekend, whereby teams of four to five students conduct mock environmental health and safety audits of companies located in the Rochester area.

Prior to performing the actual mock audits, team members must, as a group, complete a series of exercises, such as develop pre-audit questionnaires, protocols and other planning formalities. Typically, to ensure good communication and organization links, the group will set up a team discussion folder as well as hold live chat sessions with the chat transcriptions posted back to the group and to Rosenbeck for review

Similar to the individual assignments, the team exercises have peer-to-peer evaluation components. Part of the reason for the evaluations is to ensure that all team members contribute their fair share to the group.

"I have them evaluate each other as far as their participation in the group," says Rosenbeck. "I have a form they complete. I give the team a grade based on what they submit to me. Then individually they can lose points depending on how well they participated in the overall activities of the group based on the team members’ evaluations, which are anonymous."

Rosenbeck adds that if a student happens to lose points on the first team exercise, he or she inevitably does not lose points thereafter.

Another important part of Rosenbeck’s online classes deals with engaging students in meaningful threaded class discussions. Students are "expected to contribute at least one substantive comment to the class discussion each week," he says.

Rosenbeck gets this process started by awarding 10 points to all students who post a "substantive" introduction that must contain the following information: name, current status in life, past work experience, past experiences with EHS audits, why you are taking this course, and what you expect to learn from taking this course.

Overall, upon reviewing all the activity occurring inside Rosenbeck’s online class, one can see that these students "do a lot of work," he says. "Without the face-to-face interaction, I won’t fee comfortable that they know all this material unless I have them do all this work."

Ó Copyright 2003. Rochester Institute of Technology. All Rights Reserved.