Have you noticed that textbook publishers are promoting web-based homework systems such as Prentice Hall’s Grade Assist (PHGA), McGraw-Hill’s Homework Manager and Wiley’s eGrade?
All 20 sections of Finance 3000 are using the McGraw-Hill product. Students do homework online and receive instantaneous feedback (with solutions), professors enjoy automated grading, and the coordinator appreciates bolstered grading fairness across sections. No two students get the same question due to randomized seed numbers (e.g., student 1: “solve X + 219 = 567”; student 2: “solve X + 98 = 673”). If a student doesn’t like his/her score, the entire problem set may be redone, with new seed numbers, and the professor’s grade report includes the score of every attempt.
I trialed PHGA with 80 MGT 3121 students, spring 2008. Students complained that they often reasoned correctly, but made errors inputting numeric answers in the software, and thus redid entire assignments (with new seed numbers) to get the points they felt they deserved. In some cases I had to agree with the students—the software is not perfect. My larger concern is that none of the types of questions that promote deep learning are available in the software. Rather, standard “textbook” questions—questions with a single correct answer such as “determine the reorder point and reorder quantity” or “forecast demand on day 150″—lulled students into deep comas. It’s about as exciting as the computerized SAT test.
Worried that web-based homework trades richness of student thinking for my convenience, I stopped using the software.
While some studies find that web-based homework leads to improvement in grades (see Heizer et al. [1] for a survey), I’m worried that improvement in grades is not the appropriate metric. Has anyone seen a study that shows that web-based homework promotes profoundly deep learning or the ability to apply the course content in unfamiliar contexts—the type of learning that is more likely with open-ended questions?
Andrea Pascarella [2] found that web-based homework encourages college physics students to abandon careful reasoning in favor of a guess-and-check approach due to the multiple-try option. She says
“Based on the feedback literature, it is known that corrective feedback combined with multiple tries can lead students to adapt a trial-and-error strategy with a focus on completing the assignment versus learning the fundamental principles underlying the material. … One way to help achieve the goal of making web-based homework systems more valuable learning tools would be to cut down the number of tries each student has to get the correct answer.” (p. 7)
Jay Heizer (the lead author of [1]) sees multiple tries as an advantage. He told me he now sees the majority of his students redoing entire homework assignments multiple times (some as many as 4-5 times) to obtain a high score. Naturally, we want our students to rework the problems they get wrong… or is this further evidence of Pascarella’s concern? Not sure.
Another study failed to show any web-based homework benefits. After controlling for teacher experience and student “academic competence” in a college statistics class, Palocsay and Stevens [3] found that “the technique used to deliver homework makes little difference in student success.” At least web-based homework doesn’t hurt our students.
I want to excite students about my discipline but perceive web-based homework as counterproductive in this regard. Am I being stubborn, ignorant, or both for refusing to adopt?
—
[1] Heizer, J., B. Render, K. Watson. 2008. Web Based Instruction Improves Learning. Decision Line, forthcoming.
[2] Pascarella, A.M. 2004. The Influence of Web-Based Homework on Quantitative Problem-Solving in a University Physics Class. Proceedings of the NARST 2004 Annual Meeting, Vancouver.
[3] Palocsay, S.W., S.P. Stevens. 2008. A Study of the Effectiveness of Web-Based Homework in Teaching Undergraduate Business Statistics. Decision Sciences J. of Innovative Ed. 6(2) 213-232.
Will, Have you spoken with anyone in the Math department about web-based homework? My sense is that they are quite happy with it and that they believe that they are seeing pass rates rise as a result. But perhaps issues in math are different? (If so, should they be?) Sherman Wong presented the Math program at a Teaching & Technology event last year. You might like to discuss this with him. DS
For what it’s worth, reference [1] above is now on the web. http://www.decisionsciences.org/DecisionLine/Vol40/40_1/dsi-dl40_1class.pdf (D.S., thanks for the comment above.)
Will, thanks for writing about this very important issue.
In my opinion, a certain amount of mechanical, get the basics, is essential foundation in many subjects. I am sensitive to this need because I have often been poor at teaching this more mechanical material, blithely assuming that students can get the mechanics quickly and easily. However, many students need time and repetition to master the basics.
But such mechanical learning is just a step along the way to the kind of in-depth analytical thinking and reflection that is the real goal. Ultimately, as Will says, the goal is to be able to apply the material in new contexts.
I looked briefly at some on-line homework (for both statistics and economics) and rejected it as coming no where close to the kinds of explaining and applied, multi-step problem-solving questions that I assign. On further reflection, however, I realized that on-line homework might be useful as a first step for some students. Provided such highly mechanical and limited assignments are viewed as stepping stones towards real problem-solving and explaining assignments, they’re fine, in my opinion.
Will’s point about motivation is also important. Challenging, real world problems are engaging. On the other hand, without providing the tools first, challenging, real world problems can be overwhelming and frightening. Some students find a clear cut list of “first do this” and “then do that” reassuring.
Overall, I share Will’s worries but think that there can be an appropriate place for such on-line learning, provided it is just a first step. It can even free up class time to focus on motivation and richer applications.
Dahlia, thanks for the comments. When my 80-person MGT 3121 class used the Prentice Hall Grade Assist in Spring 08, I found that the software definitely helps with the mechanical, get-the-basics foundation. And outsourcing the grading to a computer was brilliant. Our mutual friend & colleague S.H. raves about the McGraw Hill version in finance.
Knowing that I would also want to push the students with weekly open-ended questions and activities, assignments became a blend of on-line and on-paper content. Students who wanted to buy a used copy of the $170 textbook (i.e., most) had to pay $27 for the software, so student out-of-pocket expense plus “set up cost” was not trivial. If it’s the only homework source, it seems justified, but perhaps not in my case.