Some readers may not think of Google as a “vendor” like other providers. But Google sells much more than advertisements. Google Sites and Google Groups are two tools you can use to deliver content, support interaction and assess learning. The free version of Sites has a 100MB capacity per site, which should be enough to get you started. This article shows how I used these tools to implement a graduate course on educational assessment. It was easy to set up, including the features of an innovative approach some of you might find quite helpful.
Figure 1 shows the homepage of my course. I used Google once before to support a seminar I taught online so it was not too hard. It took me about eight hours to set up the site and post the first few assignments. It was a nice coincidence that the crimson highlights matched Indiana’s school color.
Assessment is Challenging to Teach
As summarized in the assignments listed in Figure 2, students who complete this class are expected to develop expertise with a core set of practices, principles and policies about assessment. At Indiana, this class serves a wide range of students. Most work in elementary or secondary schools but some work in colleges or elsewhere, and some are full-time students who haven’t yet worked in education. Those who teach do so in a wide range of topics.
I use a textbook written by James Popham. While it is very readable and pitched toward teachers, it contains little about assessment that students can master simply by reading the text. For example, the guidelines for developing assessment items only make sense when you apply them and actually create assessments.
The real challenge is that much of the disciplinary knowledge about assessment takes on different meanings depending on your experience, the context in which you learn and apply that knowledge. Portfolio assessment looks very different to a math teacher than a composition teacher; validity means something rather different to a classroom teacher than to an administrator. In the language of cognitive science, most of the important knowledge in this course is consequential (has direct consequences for practice) and contextual (takes on much of its meaning in the context where it is learned and used). This knowledge is most readily learned when students interact with it in a personally meaningful context and interact with other people who are doing so as well.
Personalizable Assignments in Google Sites
Rather than individualized assignments, my students work alongside each other to “personalize” the same assignment each week. This happens in what I call “wikifolios.” Technically speaking, the subpages added to Google Sites are not wikis. They are even easier to edit than wikis. You just click on the blue pencil at the top to edit the page and save it when you are done. You can easily create a template for a page you are going to create multiple times (like when I created a homepage for each student).
The first assignment is called “Introduce Yourself.” It directs students to their homepage and tells them how to edit and save it. The first assignment asks students to describe their work or research context, their academic program, their background with assessment, etc. Students are invited to include a picture and as much personal information as they wish. It is crucial that students begin “projecting” their developing knowledge of the course and how it relates to their personal context. You can’t turn someone into a true expert in a single class. But you can help that person to develop “local expertise” so he or she knows more than any of his or her peers about how the course connects with work context.
The assignment then instructs students to look at the homepages of their classmates’ wikifolios and post a comment or a question. This highlights one of the most important features of this approach (and one of the nice features of Google Sites): nearly all of the interaction between students and the instructor takes place in threaded comments placed directly on student-generated artifacts.
Once all of the students have posted the first wikifolio, I use that information to organize the class into a manageable number of “networking groups.” In this class I ended up with three: Teachers, Administrators and Researchers. As the course progresses, I encourage individuals to try to “speak for the group” as the professional affinity grows. By about the middle of the course, I expect members of one group to start anticipating the concerns or arguments of members of the other groups. But I also encourage students to speak for themselves, to develop a distinct “voice” about the topic of the course.
To support efficient communication with the entire class, I also set up a Google Group and used a gadget to give access within the site. I created separate forums where students could ask questions about assignments, technology, etc. and where I posted comments, updates and weekly feedback. However, I tried to keep all of the substantive interaction directly on the student-generated artifacts. This kept those interactions meaningful and anchored them to examples and experiences.
The Relevance of Relevance
Things really personalized in the second assignment. This assignment asks students to define a curricular aim and select an educational standard that best exemplifies the context in which they work or would like to work in order to organize their engagement across the course. Students are instructed to read the relevant chapter and are asked to make sense of several sets of key concepts. Rather than simply summarizing each set of concepts, students are asked to rank the concepts in order of their relevance to their particular curricular aim and context, and to justify their rankings.
This validity activity turns out to be a simple way to get students to make sense of concepts that would otherwise be impossibly abstract. Figure 3 shows one student’s response to the part of the assignment that involved three different types of validity-related evidence. This particular student was a classroom teacher enrolled in an educational leadership program. You can see how she concluded that content-related evidence was most relevant to her current role, but that criterion-related evidence was more relevant to her future role.
While students are encouraged to borrow ideas from classmates, the personalization of assignments makes it difficult for them to plagiarize their work. Students who are struggling can learn a lot by looking at the work of classmates from a range of contexts. One convincing bit of evidence this is working are the number of times students change their rankings after the deadline; their reflections (discussed below) sometimes confirm it was because a classmate’s work or comment shed new light on something.
The Weekly Routine
It only takes a few weeks for an efficient routine to set in. Rather than figuring out a new assignment format each week, my students spend their time interacting around the topics of the course. If there is anything new or different about an assignment you wish to introduce, put it in bold type at the top of the assignment page.
The last thing students are asked to do before posting their draft is to summarize the main ideas they took away from the assignment. Figure 4 is a good example from the validity assignment discussed above. Notice that the student quoted the textbook twice and made a point of projecting her identity as a science teacher.
As students complete their drafts, they move the link up from the bottom of their homepage (Figure 5) to signal that they are ready for feedback. Each week I give relatively extensive feedback to the first few students who post (typically among the most experienced and ambitious). This early feedback often introduces advanced content that is abstract and would be confusing if I tried to include it in the assignment. For example, this is where I point out the many places where I disagree with the textbook. I put this early feedback in the discussion forum, along with links to actual pages, so other students can refer to them when they start their wikifolio.
Leading up to the weekly deadline, I try to chime in and comment here and there, but for the most part the conversation runs itself. After the deadline, I typically review and comment on every wikifolio and engage with longer discussion threads that emerge. Finally, I draft and post a weekly summary where I aggregate the rankings and summarize the big ideas. In particular, I search for meaningful patterns within and between the groups that might shed useful new light on the big ideas of the course. I also highlight insightful exchanges and threads and insert quotes and links as needed to drive home the key points and help motivate engagement.
Motivating Engagement with Stamps and Badges
Starting in week six, I tried a new strategy for motivating engagement. Students were instructed to endorse at least three other wikifolios as being “complete” by inserting a comment that started with a $ (a “stamp”). Students were also instructed to promote one wikifolio as being “exemplary” by adding a comment starting with $$$ and explaining what was exemplary. In my weekly feedback, I told the class which students got the most endorsements. I also placed a “badge” for that assignment next to their names on the wikifolio homepage. While we did not see an increase in the length of posts or number of comments, the comments and the discussions suggested that students really liked receiving promotions and mini-badges. In the course evaluations, only one of the students reported disliking the stamps and badges.
Assess Reflections Rather than Artifacts
This structure generates a lot of text and comments. I can’t possible read it all and I certainly don’t have time to grade it all. More importantly, grading the content or quantity of the artifacts or the comments is likely to undermine natural social interaction. Of course, some students are less active than others. However, you don’t want students who don’t think they have anything to say to post comments just for the sake of receiving a grade.
Rather than grading wikifolios, I rely on three carefully-worded reflection questions in the assignment. These reflections are due one week after the original draft of an assignment and are posted at the bottom of each wikifolio. Several years ago, a colleague helped me appreciate that students have to understand a concept pretty deeply to articulate its consequences in a particular context. Thus, the first reflection question asks students to explain what they will do differently in their context because of the things they have learned (consequential engagement).
I later realized that students need even more understanding to reflect critically on their context for learning those concepts (critical engagement). Finally, students are asked to comment on the classmates whose comments and examples they found particularly helpful (collaborative engagement). This last reflection is presumed to be particularly motivating because students like to see that somebody else learned something from them. Figure 6 is a typical example, although this student also chose to cut and paste the text of the assignment to assist in writing the reflections.
I make each of the 11 graded wikifolios worth five points; this counts for 55 of the 100 class points. I give students all of their points as long as they have a draft posted by the deadline, complete all parts of the assignment and produce a reflection that makes sense. I take away points here and there for the collaborative reflection if students have clearly not been collaborating. But I am strict about the weekly deadlines, taking away one point for each day a student is late in posting, in order to keep the interaction going.
Use Exams Prudently
In Google Forms it is easy to create exams if you follow the instructions on the help site. You can even set them up to automatically score multiple choice items. I normally try to downplay exams and argue that they should primarily be used to evaluate your instruction and only secondarily to evaluate students. But exams motivate students to stay engaged and a high score can serve to reward students who have engaged successfully and consistently.
Test security is an issue in online classes because students can look up the answers. In my previous courses, the university course management system allowed me to set a timer for the test. I set the time limit short enough so unprepared students would run out of time if they had to look up many answers. I also selected items from the textbook item pool that were structured so you might have to look up all five of the responses to be sure. I included a few timed extended-response items. This worked pretty well, and I saw a direct relationship between student engagement in the wikifolios and their score on the final.
In preparation for offering this course as a big open online course the following semester, I changed the format a bit. Because I did not have an easy way of timing the exam, I elected to make the three exams longer (30 items each), give students as much time as they needed and make them worth 15 points each (out of 100 total). For security reasons, students only got their overall score and did not see which items they got wrong.
The average scores on the three exams were 14.0, 12.6 and 14.1 while the lowest scores were 13.5, 10 and 13. Two students told me they completed the exam from memory and then went back and checked answers they were unsure about. While I am confident students learned a little more by looking up the answers, the scores are not terribly valid evidence of achievement.
Conclusion
Over the past decade, private-sector service providers have made significant inroads into higher education course creation and management. With Google CourseSites, however, this power is being returned to institutions. It is now significantly easier for professors and instructors to create their own online courses and adapt them to fit specific needs.
As we saw with email, technologies that become increasingly ubiquitous and mission-critical eventually leave the realm of subject-matter experts and high-quality options become available on open forums. Course management software has now entered that realm.
You Might Also Like
