jump to navigation

Writing good discussion prompts July 8, 2020

Posted by aquillam in teaching.
Tags: , , , , , , ,
add a comment

This post assumes you’ve read my previous post, creating effective and engaging discussions.

Numerous studies have shown that students learn best when they learn socially. Learners retain more information and are better able to synthesize it, recognize assumptions, challenge misconceptions, and utilize it in novel or creative ways if they interact with other students. In distance or blended learning classes, the asynchronous discussion forum is often the tool of choice for facilitating social learning. However, simply creating a discussion board is not enough. You need to help them build  their learning community, which requires both social presence and cognitive engagement. Students need to see each other as real people, with an emotional presence in the class. If they think they matter to their peers, they are more likely to be willing to do the extra work of really engaging with their peers over the material. The rules you set and the involvement you take help to determine how involved the students will be, but you can also use prompts that encourage diverse thinking, and the sharing of ideas. The level of the prompts largely determine the level of cognitive engagement. Read more about creating effective and engaging discussions here, or check the resources at the end. I’ll focus on constructing the prompts below. 

Good discussion prompts should guide students toward higher order cognitive skills by facilitating discussion. Remember, the goal is to get students to have a conversation with each other, not to respond to the primary prompt. Bloom’s taxonomy (see below) lists the order of cognitive skills as: remembering, understanding, applying, analyzing, evaluating, and creating. When grappling with new knowledge, students tend to focus on the recall and understanding steps, but those can be assessed with something like an auto-graded quiz. The point of the discussion is to get them to engage more deeply. 

You may be able to write a single prompt that will ask them to analyze or evaluate, or you may need a scaffolded set of prompts to build toward those higher levels. Don’t forget your prompt needs to be structured in such a way that it encourages conversation, like limiting the number of people who reply directly to the prompt, so they have to reply to each other. These prompts take careful crafting to work well, but usually take less intervention to keep them going.  

You may also want to consider a structured type of prompt, such as problem solving, project based, or debate prompts. These more structured types often lead to both greater involvement (as determined by the number of posts) and  higher levels of cognition (as determined by the level of skill shown in the post content.) However, these also usually require more attention on the part of the instructor. A debate, for example, may require a thread for the initial brainstorm of ideas, followed by group assignment and facilitation of group discussions to develop a group argument, and finally moderation of the class debate. Consider how many students you have, how many GSIs can help you, and whether you can remove some other assessment (maybe the debate replaces a homework assignment.) If you can manage the workload, the enhanced student engagement is well worth it. 

So now that you have some idea about the different types of prompts, let’s talk about formulating the prompts. 

Every prompt (every piece of work, really), should be relevant to what you want students to get out of your class, so start by considering your learning goals. What learning goals do you want covered by this discussion? What outcomes would show you that they have achieved those goals? What outcomes involve higher level skills? If you were to have a conversation with friends or family about that outcome, what would it look like, and how might it start? Are there open questions or ideas up for debate? Could it be posed as a problem to be solved? Does it connect to a larger class project? Should the students have enough information when the prompt is posted to be able to generate a conversation? These questions should give you the foundation for designing your prompt(s). 

A standard structure for discussions is to require a response to the original prompt, plus two responses to peer’s posts. That works well for small classes or group discussions, and has a nice structure if you have different deadlines for the original replay and the peer responses. However, in a large class, that quickly becomes unwieldy for both you and the students. Offer them options, like responding to your prompts or to their peers, from the beginning. For example, ask a “what do you know, what do you want to know” question and give students the option to reply to a previous response with expansion or clarification of the knowledge, related questions, and thoughts about how to answer the questions. If the conversation lags, or if it isn’t at the level you want, you can join it by asking your own questions. “I see many people want to know more about [topic]. Why is that interesting to you? What knowledge do you think you’ll need? What have you learned that made you think of that?” 

Prompts with multiple requirements dependent on prior responses are useful for scaffolded discussions. Brainstorming, or general invitation questions are good starting points. They activate prior knowledge and get students started sharing ideas. However, those types of  prompts generally don’t illicit much interaction on their own, and their cognitive level is down there at the bottom of Bloom’s taxonomy. They tend to illicit a lot of “I totally agree, I like [thing] too!” peer responses.  You can push the level up by asking for value judgments or consensus, then ask for solutions or next steps. For example, “Due Tuesday: If you are one of the first 10 people to respond, post a challenge to accomplishing [thing] and explain what makes it a challenge. If there are already 10 responses, pick the response that you think poses the greatest challenge and explain why you think it is the greatest challenge. Due Friday: look at other people’s responses to what poses the greatest challenge, and pick one that you don’t agree with. Explain how you think that challenge can be overcome. Alternatively, respond to the responses to your earlier post.” By staging the steps in the original prompt, you’re laying out the plan, so students can prepare to write their responses. You’re also enforcing interaction by requiring that most of the responses actually be responses to students, not to you. Finally, you’re giving them some autonomy, by offering them options around how to engage. 

Open-ended prompts with a variety of reasonable responses are the most likely to engage higher level thinking. Prompts that ask students to reflect on course material and incorporate it to make some sort of judgement seem to be the most effective at achieving higher level thinking. You can help generate the sense of community by asking students to reflect on other posts. Follow that up again by asking them to respond to the people who responded to their posts. Make sure the discussion ties in to the course by matching it to the learning goals, or by using it as the foundation for other assignments. You might use a prompt like the previous example to assign groups for a project to come up with solutions to the challenges, or vote on which challenge is the biggest and set up a debate for whether or not we should try to overcome the challenge. 

You can also use the prompts to drive students to help each other out. Set up a standing, non-required discussion for “it’s in the syllabus” questions and encourage its use (you can refer to it, send students there when they ask questions appropriate for it, give extra credit for students who are helpful, thank the extra helpful ones…) Set up homework or exam review threads.  For example, a week (or more) before the exam, ask students for the three most difficult problems, then ask them to help each other out. Or ask them to try and write exam questions, but specify that they have to be higher on Bloom’s taxonomy than recall or understanding. Give them topic headings and ask them to fill in what they know. 

Most discussion boards will allow you to set up groups, including an option for groups of 1. If you want to use a more formal prompt type, you’ll likely want to take advantage of that. For example, if you want the class to hold a debate, you’ll need to break them into teams. You might also break them into groups to tackle a problem. For example, if you give them a really big challenge, like getting humans to Mars, you might start with a brainstorm/ biggest challenge post for the whole class, then split them into groups to tackle one individual challenge per group, then bring them back together again to make sure their solutions aren’t conflicting. You can even use discussion boards to mimic think-pair-share, or 1-2-4-all (think-pair-small group-full class sharing) techniques. One person groups offer the opportunity for journaling or metacognitive questions: “Compare your first two posts and most recent two posts. Which ones show the highest level cognitive engagement? How do you judge that? What was it about that post that engaged you on that level? Do you think this discussion board is helping you improve your critical thinking skills?”

Finally, a good prompt will be transparent, and accommodating. If you supply your students with the course learning goals, the prompt should tell them what goals you hope to reach with this discussion prompt. It should tell them what sources you expect them to pull from (book chapter(s), links to reading materials…) Requirements, including deadlines and formatting,  should be clear. Provide links to services they might need (e.g. if they’re likely to want video, include links to the video and captioning how-tos). Check your prompt for accessibility standards. 

And, don’t forget to keep a document somewhere with the prompt and any notes, like whether or not it was good, or needed improvement. Future you will thank you for it! 

Resources

Creating effective and engaging online discussions June 24, 2020

Posted by aquillam in Science, teaching.
Tags: , , , ,
add a comment

With links for Canvas Discussions and Piazza where appropriate

As we prepare for fall semester, I’ve been creating some documentation for my department. Occasionally, people outside my department find these useful, so I thought I’d blog them too.

It’s been well established that students understand and retain more from classes where they are able to engage with their peers. Additionally, working with large and diverse teams is a necessary life skill (and one employers value), so it’s well worth including as a learning goal for your class. However, building a community of learning poses a special challenge in the online classroom. Many instructors turn to the discussion board as the tool for community building. As with all tools, how you use it matters. 

If you want your students to engage with the discussion board, the first thing you need to do is establish its value. The obvious way to do that is by assigning points to it. There is some evidence to suggest that assigning points at the beginning of the class is valuable, especially during the period of group formation. However, if you set a fixed requirement (e.g. five good posts), most students will fulfill that requirement and stop. Also, if the stakes are too high, students become more concerned with matching the rubric than in genuine interactions. An alternative to points might be badges (support for Badgr for Canvas), but so far most students don’t place a high value on digital badges. To build a community, you need to share with students why you’re asking them to engage with each other. Whether you simply tell them what the research says or share the research literature and citations is up to you (see links at the end for a few of these.) Best practice may be to include points for discussion board posts with the first few assignments, but not make the discussion board a distinct element for graded assessment, and remind them of the purpose regularly. This gives them the motivation to get started, but keeps the focus on social learning over assessment.  

Once you’ve established why you want them to participate, you need to establish the expectations for how to participate. Providing clear guidelines can help you avoid some of the major pitfalls, and provide guidance for students who may be anxious about group work online. There are three things you really need to do: establish the rules of engagement, be present (but not too present), and make it manageable.

First and foremost, layout the ground rules. For example, tell them they need to write complete sentences, no personal attacks, no taking over the conversation, and no lurking. Give them examples of good posts and bad posts. How often have we seen the interaction that looks like “I totally agree with <student>. I love <fill in the blank>. I really like what they said about liking <fill in the blank>.” That’s a great interaction with copy-paste, and maybe makes <student> feel validated, but it’s not engaging with the material or furthering the conversation. Also, make sure that they know what good behavior looks like. Respondnig to someone with “that argument sounds stupid to me” really isn’t any better than “that’s a stupid idea.” Think about what you would expect of them if they were actually in class. How would you react to any of the above interactions in the classroom? Give them that guidance up front. Also, if you are going to assign points to the discussion, especially if you’ll have them do peer evaluation, provide them with a rubric. In Canvas, you can connect the rubric to a graded discussion. Best practice is to make it fairly generic so you can use the same rubric for every post. You may want to pin the instructions and rubric to the top of the discussion board. Adding a rubric to a graded discussion in Canvas, pinning a post in Canvas, pinning a post in piazza. 

You need to be present regularly, and let them see that you are there. Establish times when you’ll be online, and times when you won’t respond. Establish expectations such as when and how often they should post, whether or not they should expect you to respond, and when they should expect to respond to each other. Again, think of what you would do in person. If the students are having a good conversation, keep out of it. If the conversation is faltering, give it a push. 

If you have a very large class, you may need to break it into groups to make the conversations manageable – discussion section sized rather than lecture sized. This is especially helpful if GSIs will be grading, or if you want students to always respond to the primary prompt.  Creating group discussions in Canvas Creating groups in Piazza. It can also be helpful for you to show them how to adjust their settings. Some discussion boards default to emailing participants every interaction. You want them to be active on the discussion board, not buried under email! how to adjust settings for Canvas discussion boards How to adjust settings in Piazza 

Using an icebreaker in a discussion board is valuable. It’s the first post, so it establishes how the board will work. It helps students get to know who else is in the class, and you can use it to help students find peers with similar goals, or to find peers with different skills. Require both a primary post and a couple responses to other students. You should call out particularly good interactions and illustrate why they’re good interactions. Model the behavior yourself. Respond to several students as if you were one of the group, trying to further the conversation. In short, participate in the icebreaker the same way you expect them to participate in the regular class discussion.

Put discussion board participation on the course calendar, and send reminders the same way you would for any other assignment. Treat it like a required part of the course even if you aren’t assigning points to it.  For Canvas Discussions, make it graded or check the “Add to Student to-do” box. For Piazza, create a calendar event in Canvas and link to Piazza Check for students who aren’t participating, and follow up with them. If they have some challenge that limits their participation, do what you can to address the issue. Reiterate your reasoning for using a discussion board and its importance to the class. 

Keep in mind that plain text is not always the best tool. This is especially true if students need to do math or artwork. Go ahead and type the formula for the volume of a sphere in plaintext, you’ll see what I mean! Many discussion tools allow alternative posting options such as images, video, or files. The more flexible you can be, the more willing some students will be to participate. Make sure to model and respond to these posts yourself. 

It’s a lot easier for students to simply respond to your post, rather than interacting with peers. But you’re trying to build a community, not hold 100 personal conversations on the same topic. Force them to interact with each other by having a cut off, like 6 hours, or 5 posts, after which instead of responding to your prompt, they have to respond to their peers’ responses. Provide them with guidance, or you’ll end up with the “I like what you like” non-conversation again. Have them make value judgments, such as which response has the most important issue or hardest problem and why. Or have them look for the response that generates a new question, and respond with that question, or see if you can answer someone’s question. Make it clear that the goal is always to add something new to the conversation. Ask them to assess whether or not their response lends itself to more conversation, or stops it. 

Once you’ve established how to use the discussion board, it’s time to start teaching with the discussion board. Consider if you were meeting them in class, what would you want their interactions to look like? How would you guide them toward a genuine discussion in person? If there is a good conversation going, keep out of it. If the conversation stalls, add a new prompt. Don’t be afraid to let the conversation wonder. Remember, this is the space for students to explore with each other. Let them follow their curiosity, at least as long as it doesn’t devolve into a discussion of weekend plans. If it seems appropriate, you can give accolades in the discussion board, but again don’t interrupt the conversation. If you want to acknowledge a student’s work but don’t want to interrupt, you can always send a private message or email them. Similarly, if someone behaves inappropriately, or makes a mistake, you may want to PM/email them, unless a public reprimand really is what’s called for. 

Writing a good discussion prompt is a whole other conversation, so I’ll save that for a new post. 

Below are some good references and further reading on using discussion boards.  

Amanda Page and Miriam Abbott. A Discussion About Online Discussion. https://www.facultyfocus.com/articles/online-education/discussion-about-online-discussion/

Mark Lieberman. Discussion Boards: Valuable? Overused? Discuss. https://www.insidehighered.com/digital-learning/article/2019/03/27/new-approaches-discussion-boards-aim-dynamic-online-learning

The links in the following include the UM library proxy encoding. You may need to be logged in through the library for the links to work. 

Delaney D, Kummer T, Singh K. Evaluating the impact of online discussion boards on student engagement with group work. British Journal of Educational Technology. 2019;50(2):902-920. doi:10.1111/bjet.12614.

Kwok-Wing Lai (2015) Knowledge construction in online learning communities: a case study of a doctoral course, Studies in Higher Education, 40:4, 561-579, DOI: 10.1080/03075079.2013.831402

de Castro, V.B., Sridharan, B., Watty, K. and Safari, M. (2020), The impact of learner engagement on performance outcomes: a longitudinal study in accounting education. Account Finance. doi:10.1111/acfi.12640

ES2018 – Creative use of technology to assess students June 1, 2018

Posted by aquillam in teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on technologies that could help with student assessment.

The session started with a discussion of what is assessment. We tend to think of it as a method of assigning scores/grades (summative assessment), but the discussion showed we really mean a lot more than that. In particular, gaging understanding, getting feedback and re-evaluation (formative assessment) were topics covered in the discussion.

There was a brief discussion of Universal Design for Learning (UDL) The idea behind UDL is that not everyone comes to the classroom with the same set of skills and concepts. If we consider the whys and hows of learning, and are intentional and explicit in our assignment design, we can mitigate disadvantages caused by differing backgrounds.  Generating engagement and providing multiple pathways to achieve flexible but well defined outcomes improves learning.

We then spent some time discussing what most people do for their classes. It was not particularly surprising that most people place a lot of emphasis on reading before class, and delivering lectures in class. These are very unidirectional, and don’t encourage engagement or deeper thinking. There are technologies that can help.

The presenter, Ebony Perouse-Harvey with CRLT, provided a handout with several resources. Here are some basics from that handout (with apologies that I didn’t take the time to copy down all the links):

  • Clickers and short response tools let you do quick checks of student understanding. Some can be used to take input from students, like allowing them to ask questions during lecture. these include iClicker, centimeter, piazza, poll Everywhere, and Google forms.
  • Word clouds can help you gage student’s exposure to subject mater and vocabulary, or topics of interest. They can also be useful in brainstorming.  Tools include Mentimeter and Poll Everywhere.
  • Mid/concept mapping help student make connections. (As a side note, I’ve looked into these before since one of the biggest problems for our labs is that students see each lab as an isolated activity, divorced from all the others and from the lecture. There are a lot of astronomy class mind maps on mindmeister and most of them are terrible. They tend to look like the table of contents to the text. Concept maps should be a powerful tool to help students make connections, like linking exoplanet discoveries back to line spectra, Kepler’s laws, and the Moons of Jupiter lab, but I haven’t seen a truly effective assignment.) Tools include bubble.us, mindMeister, and MindUp.
  • Google Docs / Canvas Collaborations (Gogle Docs portal within Canvas) – collaborate tools like text documents, spreadsheets, or presentations. This requires a whole different blog post…
  • Discussions offers threaded communication between peers or between the instructor and student. The Canvas discussion tool can be set up to have threads available to the whole class, and other threads available only to a group. Piazza also offers anonymous posting.
  • Blogging can be used to capture student ideas during a class session.

ES 2018 – effective group assignments May 31, 2018

Posted by aquillam in teaching.
Tags: , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on creating effective group assignments.

Research shows that group work increases students’ perceptions of learning and their belief in their ability to learn, improves retention of information, increases learning of higher-order tasks, and improves scores. Additionally, more diverse groups usually work better than less diverse groups, both in terms of group outcome and individual success. Students are often resistant to group work, so it’s important to include your reasoning for requiring group work and the references in your syllabus or instructions. References were provided in this session, but only on a printed sheet.

Creating effective group assignments actually depends on two things: a good assignment and a good group. If you’re also using the assignment for assessment, you’ll also need a mechanism to evaluate the work.

Creating a good assignment

There are a couple elements to consider before creating your assignment.

The first is to make sure the assignment aligns with your learning goals. As you develop the assignment, be sure to go back and reassess whether or not the tasks still lead to that learning goal. It is best make the learning goal explicit in the assignment as well so that students know it isn’t just “busy work.” Students often don’t make connections between different aspects without some guidance.

Make sure the assignment is appropriate for a group. If you give a group the assignment to do a reading summary, chances are one person will do the summary and they’ll turn it in for the group. Instead, have them compare summaries and come up with a list of things everyone had in common, and a list of things someone thought was important but others missed. If it’s a task the average student should be able to complete alone, it’s probably not a good group task. Tasks that create interdependence are best, since they make group members reliant on each other.

Design assignments to promote higher-order cognitive skills. Again, lower level tasks like memorization are just as easy to do on your own. Group assignments provide the opportunity to test your thinking, uncover misconceptions or logical fallacies, and get different perspectives. Good assignments take advantage of those opportunities.

Good assignments usually have the following components

  • individual prep
  • a group task
  • outcome or deliverable
  • debrief

Generally, assignments are better if each of these is made explicit. Essentially, students need to know what the expectations are to perform well, and they are much more likely to develop a deeper level of learning if they are given the opportunity to think about it.

This is true even for small assignments. For example, you can give students a reading assignment (individual prep), with a think-pair-share activity in class (individual and group tasks). A flashcard or clicker response provides the outcome, then a class discussion about why students chose the responses they chose is the debrief. The syllabus should have formal instructions to students about the expectations of reading the material, working with peers, bringing a clicker/flashcard to class, and participation in in-class instruction. The debrief is an important aspect since it is the part that allows students to make sense of the answer. Without the debrief, students may not have an opportunity to do more than memorize the correct response. As long as you regularly practice these small or informal group assignments in class, the syllabus instructions are sufficient.

For larger assignments, the same guidelines still apply. The primary difference is that there are likely to be several stages with a different set of individual preparation and group tasks for each stage. Each stage may also have it’s own deliverable, separate from the final outcome or deliverable. It’s also important to debrief periodically to ensure all team members are meeting the expectations of both the instructor and their teammates. One of the reasons students are so resistant to teamwork is due to a bad experience in the past, where there were problems with a team that went unrecognized or unresolved.

Forming groups

Group formation depends heavily on the assignment. In general, it is better to have groups with some diversity in them, which usually means not allowing students to self-select groups. However, the assignment and group size affect this.

Small assignments with small groups, like think-pair-share in class questions, are generally fine with self selected groups. If nothing else, class time would be lost waiting for students to change seats. You can always ask students to sit in adjacent seats at the start of class, and have them turn to a different neighbor for each question.

Similarly, projects that extend thorough  a single class period but not beyond (e.g. a lab) are unlikely to benefit significantly over the time it takes to assign and organize groups. However, splitting up cliques and social groups may be beneficial. Randomly selected groups can often work well in these cases. For example, have students count off by the number of groups needed before starting on the activity (i.e. a class of 24 needs 6 groups, so students could off 1 – 6, then all 1s work together, 2s work together, etc.) Use caution with this approach. While diversity is good, you should be careful about isolating certain individuals. Students for whom English is a second language are especially at risk of isolation when placed in a group of native english speakers. Women, minorities, and those with physical or learning impairments can also be negatively impacted. You’ll have to weigh the effort of creating not-quite-random groups with the composition of your class.

Long term or major projects work best with carefully considered groups. In particular, the greatest gains are made when groups have heterogenous but not too dissimilar abilities. For example, putting a A student in with a B student forces the A student to explain, which can help clarify thinking and deepen understanding. The B student may gain strategies or insights from the A student. Similarly, a physics major can help the group understand the science, and an english major can help the group express their ideas clearly. When creating the groups, it’s important to try and match interests and motivation. If two of the four group members are deeply invested in the idea that Pluto is a planet, and the other two group members want to explore Mars, the group may not form a cohesive unit.

CATME Team generation software can be used to collect data on students and generate teams based on criteria you specify. It is NOT free, but you can contact ISS for resources to help cover the cost for a U-M course. As far as they know, this is the only software that can generate groups. You can also use Canvas surveys, Google forms, Qualtrics, SurveyMonkey, or other software to collect the data you want, then form the teams manually.

Any group that lasts more than a single class, and any assignment worth more than a minuscule amount  of credit, especially if it’s group credit, requires some form of group assessment and recourse for dysfunction. Assess early and often. You want to catch problems in time to correct them, not after the group implodes.

Assessing groups

When generating a score for group assignments, you want to measure both individual performance and group performance. This should include peer evaluation whenever possible, but keep in mind that teams of two will never have anonymity. Be sure share the assessment protocols at beginning of semester or assignment  (i.e. BE TRANSPARENT!) If you want honest evaluations, you need to employ non competitive grading. A rubric with options like “meets expectations” can be good, but you may want to consider requiring students to justify high or low responses. Evaluations should be goal oriented, so there are concrete outcomes to evaluate. Note some of the people in the session did have competitions, so the students could select things like “most visually appealing project” or “most helpful teammate”, but these did NOT enter into the assignment scores.

CATME includes peer evaluation tools and allows you to require both self evaluation and peer evaluation. Teammates is another full featured tool for peer evaluation and feedback. See http://teammatesv4.appspot.com/. There is also the peer evaluations in Canvas.

Take home points

  • assignments with meaningful interdependence foster student interaction
  • intentional structuring of teams promotes inclusion & learning
  • peer eval is a valuable skill and provides evidence to instructor about group dynamics