jump to navigation

ES2018 – Creative use of technology to assess students June 1, 2018

Posted by aquillam in teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on technologies that could help with student assessment.

The session started with a discussion of what is assessment. We tend to think of it as a method of assigning scores/grades (summative assessment), but the discussion showed we really mean a lot more than that. In particular, gaging understanding, getting feedback and re-evaluation (formative assessment) were topics covered in the discussion.

There was a brief discussion of Universal Design for Learning (UDL) The idea behind UDL is that not everyone comes to the classroom with the same set of skills and concepts. If we consider the whys and hows of learning, and are intentional and explicit in our assignment design, we can mitigate disadvantages caused by differing backgrounds.  Generating engagement and providing multiple pathways to achieve flexible but well defined outcomes improves learning.

We then spent some time discussing what most people do for their classes. It was not particularly surprising that most people place a lot of emphasis on reading before class, and delivering lectures in class. These are very unidirectional, and don’t encourage engagement or deeper thinking. There are technologies that can help.

The presenter, Ebony Perouse-Harvey with CRLT, provided a handout with several resources. Here are some basics from that handout (with apologies that I didn’t take the time to copy down all the links):

  • Clickers and short response tools let you do quick checks of student understanding. Some can be used to take input from students, like allowing them to ask questions during lecture. these include iClicker, centimeter, piazza, poll Everywhere, and Google forms.
  • Word clouds can help you gage student’s exposure to subject mater and vocabulary, or topics of interest. They can also be useful in brainstorming.  Tools include Mentimeter and Poll Everywhere.
  • Mid/concept mapping help student make connections. (As a side note, I’ve looked into these before since one of the biggest problems for our labs is that students see each lab as an isolated activity, divorced from all the others and from the lecture. There are a lot of astronomy class mind maps on mindmeister and most of them are terrible. They tend to look like the table of contents to the text. Concept maps should be a powerful tool to help students make connections, like linking exoplanet discoveries back to line spectra, Kepler’s laws, and the Moons of Jupiter lab, but I haven’t seen a truly effective assignment.) Tools include bubble.us, mindMeister, and MindUp.
  • Google Docs / Canvas Collaborations (Gogle Docs portal within Canvas) – collaborate tools like text documents, spreadsheets, or presentations. This requires a whole different blog post…
  • Discussions offers threaded communication between peers or between the instructor and student. The Canvas discussion tool can be set up to have threads available to the whole class, and other threads available only to a group. Piazza also offers anonymous posting.
  • Blogging can be used to capture student ideas during a class session.
Advertisements

ES 2018 – effective group assignments May 31, 2018

Posted by aquillam in teaching.
Tags: , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on creating effective group assignments.

Research shows that group work increases students’ perceptions of learning and their belief in their ability to learn, improves retention of information, increases learning of higher-order tasks, and improves scores. Additionally, more diverse groups usually work better than less diverse groups, both in terms of group outcome and individual success. Students are often resistant to group work, so it’s important to include your reasoning for requiring group work and the references in your syllabus or instructions. References were provided in this session, but only on a printed sheet.

Creating effective group assignments actually depends on two things: a good assignment and a good group. If you’re also using the assignment for assessment, you’ll also need a mechanism to evaluate the work.

Creating a good assignment

There are a couple elements to consider before creating your assignment.

The first is to make sure the assignment aligns with your learning goals. As you develop the assignment, be sure to go back and reassess whether or not the tasks still lead to that learning goal. It is best make the learning goal explicit in the assignment as well so that students know it isn’t just “busy work.” Students often don’t make connections between different aspects without some guidance.

Make sure the assignment is appropriate for a group. If you give a group the assignment to do a reading summary, chances are one person will do the summary and they’ll turn it in for the group. Instead, have them compare summaries and come up with a list of things everyone had in common, and a list of things someone thought was important but others missed. If it’s a task the average student should be able to complete alone, it’s probably not a good group task. Tasks that create interdependence are best, since they make group members reliant on each other.

Design assignments to promote higher-order cognitive skills. Again, lower level tasks like memorization are just as easy to do on your own. Group assignments provide the opportunity to test your thinking, uncover misconceptions or logical fallacies, and get different perspectives. Good assignments take advantage of those opportunities.

Good assignments usually have the following components

  • individual prep
  • a group task
  • outcome or deliverable
  • debrief

Generally, assignments are better if each of these is made explicit. Essentially, students need to know what the expectations are to perform well, and they are much more likely to develop a deeper level of learning if they are given the opportunity to think about it.

This is true even for small assignments. For example, you can give students a reading assignment (individual prep), with a think-pair-share activity in class (individual and group tasks). A flashcard or clicker response provides the outcome, then a class discussion about why students chose the responses they chose is the debrief. The syllabus should have formal instructions to students about the expectations of reading the material, working with peers, bringing a clicker/flashcard to class, and participation in in-class instruction. The debrief is an important aspect since it is the part that allows students to make sense of the answer. Without the debrief, students may not have an opportunity to do more than memorize the correct response. As long as you regularly practice these small or informal group assignments in class, the syllabus instructions are sufficient.

For larger assignments, the same guidelines still apply. The primary difference is that there are likely to be several stages with a different set of individual preparation and group tasks for each stage. Each stage may also have it’s own deliverable, separate from the final outcome or deliverable. It’s also important to debrief periodically to ensure all team members are meeting the expectations of both the instructor and their teammates. One of the reasons students are so resistant to teamwork is due to a bad experience in the past, where there were problems with a team that went unrecognized or unresolved.

Forming groups

Group formation depends heavily on the assignment. In general, it is better to have groups with some diversity in them, which usually means not allowing students to self-select groups. However, the assignment and group size affect this.

Small assignments with small groups, like think-pair-share in class questions, are generally fine with self selected groups. If nothing else, class time would be lost waiting for students to change seats. You can always ask students to sit in adjacent seats at the start of class, and have them turn to a different neighbor for each question.

Similarly, projects that extend thorough  a single class period but not beyond (e.g. a lab) are unlikely to benefit significantly over the time it takes to assign and organize groups. However, splitting up cliques and social groups may be beneficial. Randomly selected groups can often work well in these cases. For example, have students count off by the number of groups needed before starting on the activity (i.e. a class of 24 needs 6 groups, so students could off 1 – 6, then all 1s work together, 2s work together, etc.) Use caution with this approach. While diversity is good, you should be careful about isolating certain individuals. Students for whom English is a second language are especially at risk of isolation when placed in a group of native english speakers. Women, minorities, and those with physical or learning impairments can also be negatively impacted. You’ll have to weigh the effort of creating not-quite-random groups with the composition of your class.

Long term or major projects work best with carefully considered groups. In particular, the greatest gains are made when groups have heterogenous but not too dissimilar abilities. For example, putting a A student in with a B student forces the A student to explain, which can help clarify thinking and deepen understanding. The B student may gain strategies or insights from the A student. Similarly, a physics major can help the group understand the science, and an english major can help the group express their ideas clearly. When creating the groups, it’s important to try and match interests and motivation. If two of the four group members are deeply invested in the idea that Pluto is a planet, and the other two group members want to explore Mars, the group may not form a cohesive unit.

CATME Team generation software can be used to collect data on students and generate teams based on criteria you specify. It is NOT free, but you can contact ISS for resources to help cover the cost for a U-M course. As far as they know, this is the only software that can generate groups. You can also use Canvas surveys, Google forms, Qualtrics, SurveyMonkey, or other software to collect the data you want, then form the teams manually.

Any group that lasts more than a single class, and any assignment worth more than a minuscule amount  of credit, especially if it’s group credit, requires some form of group assessment and recourse for dysfunction. Assess early and often. You want to catch problems in time to correct them, not after the group implodes.

Assessing groups

When generating a score for group assignments, you want to measure both individual performance and group performance. This should include peer evaluation whenever possible, but keep in mind that teams of two will never have anonymity. Be sure share the assessment protocols at beginning of semester or assignment  (i.e. BE TRANSPARENT!) If you want honest evaluations, you need to employ non competitive grading. A rubric with options like “meets expectations” can be good, but you may want to consider requiring students to justify high or low responses. Evaluations should be goal oriented, so there are concrete outcomes to evaluate. Note some of the people in the session did have competitions, so the students could select things like “most visually appealing project” or “most helpful teammate”, but these did NOT enter into the assignment scores.

CATME includes peer evaluation tools and allows you to require both self evaluation and peer evaluation. Teammates is another full featured tool for peer evaluation and feedback. See http://teammatesv4.appspot.com/. There is also the peer evaluations in Canvas.

Take home points

  • assignments with meaningful interdependence foster student interaction
  • intentional structuring of teams promotes inclusion & learning
  • peer eval is a valuable skill and provides evidence to instructor about group dynamics

ES 2018 – Lightning talks – Active Learning May 11, 2018

Posted by aquillam in Astronomy, teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was a series of lightning talks on Active Learning. There were 7 presenters. Titles below are my quick reinterpretation, not the listed talk titles (lightning talks = no time to type everything, I just wanted to be sure I got the names right!)

 

Yulia Sevryugina: Strategies in Chemistry

Prof. Sevryugina described a couple strategies she uses for her large chemistry lab.

  • Syllabus game
    • Students generally don’t read the syllabus, and don’t really pay attention if you go over it during class, so this is intended to engage the students actively with the syllabus. It also models some lab classes, though she didn’t say if it modeled the way her class operates.
    • The room is set up with stations, and the class is divided into groups. The stations can be completed in any order except the last one. At each station is a printed copy of the syllabus, and question sheets. As each group visits a station, they pick up and fill out the question sheet for that station. Once they have all the sheets filled out, they go to the last station, where they meet the instructor, get a sweet, and discuss some of the questions.
    • Some of the questions are actually discussion topics, such as “find out why your other groups members are taking this class” or “share some learning strategies you think will help you succeed in this class.”
    • Questions are designed to cover FAQs, help the students get to know each other, and get to know the instructor.
    • Assessment is points for completion of the question sheets
  • Canvas Discussion / Piazza
    • The anonymous option helps make the instructor more approachable, and decreases the students’ embarrassment with asking questions in front of their peers.
  • Jigsaw classroom
    • The students are first divided into “expert” groups. Each group is given an area to study at home and become an expert in. At the next class, new groups are formed with one person from each expert group: the jigsaw. Each jigsaw group is given a task or activity to complete as a group, which can include an artifact for formal assessment.
    • final grades should generally be a group grade, but should include a component for individual preparation, so students can’t shirk the expert phase.
    • It’s usually very useful to have a decompression phase at the end. In particular, it is helpful to have a whole-class discussion to ensure students identify holes in their expertise, or weaknesses in their problem solving strategies.
    • This technique can be used to increase student preparation. It slightly decreases the workload (e.g. instead of having to understand all of Newton’s laws, each student only really has to understand one) but increases the stakes (the jigsaw group is counting on that student for the information.)

 

Eri Bell: Increasing student interest in science via a capstone research project 

Prof. Bell introduced a research project culminating in a poster session in several large astronomy classes. He showed data for Astro 101 in Fall 2017, indicating that the large scale group project did intact significantly increase student interest and confidence in science. This is a large class, 180 students, but the poster session was still manageable. He offered a few tips and takeaways.

  • leave enough time for final revisions.
  • a good presentation space is important. posters in the lecture hall stifled conversation by making it difficult to move around. Having it in a space where the department members and surrounding departments could visit gave it much more of a feeling of a real poster session.
  • require peer interactions by requiring something like two peer evaluations or reports of other posters so that students are forced to interact with the other posters
  • I’ll add that from my point of view, having a good rubric was important. In particular, we needed to make sure we were grading on content, not appearance, so having specific measurable items was important to ensuring we gave fair grades.

 

 

Jessie Lee: Gallery walk sale presentation

Previously in HRM 305, students had done a final project where they were supposed to come up with an HR plan then give presentation at end of class. However, the other students really only engaged in passive listening, so the presentations didn’t seem very valuable except as a summative assessment, not at all like the real world. Prof. Lee changed it to be more like a gallery walk (think opening night of an art exhibition, or a poster session!)

  • held in a collaborative leaning space, where there are movable tables and chairs and wall-mounted monitors. Student groups were set up at stations around each monitor, or in groups that could travel from station to station. People at each station give a 10 minute talk, then 3 min Q&A. After the Q&A, the audience can move to a new station.
  • Students provide peer feedback on the presentations. The feedback form is structured and includes specific questions, as well as an opportunity for general comments. The presenters really liked getting the feedback.

 

Carol Shannon Teach, Assess, Revise: using assessment to drive revision

~2013 the college of pharmacy noted some issue with their curriculum: students were not working effectively, reported learning skills much later in the program than they should hav, and the faculty and staff had fundamental misconceptions about what students knew when they came in to the program. For example, students have no idea how to use the internet for scholarly research.

They needed new curriculum to address these issues.They changed problem sets to more closely match real-life scenarios, tried to match the specifics of what students rally needed (e.g. pubMed & Embase search). They began using a pre-test for all students to determine what was needed, and post test to see what the retained. Students learned more in classes where they had to do pre-work, so they’ll be revising the curriculum again!

 

Ruth Lee: Google Docs in first year writing class

Prof. Li taught ENG 125 – freshmen english (required). Classes are primarily discussion sections with about 20 students per class. She used many features of Google Docs to enhance collaboration, including real-time collaboration in class. Some examples:

  • assign an essay to read with things like vocabulary, citations, other items already highlighted. The highlights were color coded. This modeled what she expected the students to learn.
  • brainstorming – groups were given a section of google doc to work on, then were able to evaluate the other groups’ work without having to share documents.
  • group commenting – everyone put in their thesis statement, then  peers comment on it
  • class reading chart – Students make comments about the readings without identifying which reading. Peers try to identify the reading and give an explanation for why they think the comment belongs with that reading.

The students interacted well together, and were more meta-aware of the writing and collaborative process. Google docs work well in the relatively small group.

Further scaffolding to develop reading and writing skills would help (there’s always more work to be done!)

 

Dave Choberka – UMMA Exchange

exchange.umma.umich.edu

UMMA has a collection database, but searching is hard because the keywords are limited (e.g. a search for Whistler or “Sea and Rain” would turn up the painting by that title, but not a search for “beach”, even though it’s a person walking on a beach.) Also, research on the collections tends to disappear after its done. The Exchange solves some of those problems.

  • you can create groups of objects, like all the things that seem to be about zombies
    • Once a group of objects is created, if you view one of the objects, it will show you all groups the object is on, and all the related objects.
    • You can select “On display” to see only objects that are actually on display in the museum.
  • You can create virtual exhibitions, which include text and resources from outside to help put the UMMA artifact in context.

 

Dominique Butler-Borruat:  “Lets talkabroad”

This is an online platform that provides trained conversation partners. It’s not just for language classes, it’s for conversation practice, including practice for professional communication. There is a fee per conversation. The conversation partners are vetted and coached before the conversations. Instructors can specify a region or the world (e.g. Prof. Butler-Borruat has students talking with people in French Guiana).

Enriching Scholarship 2017 – Coaxing Canvas to Support Gameful Pedagogy May 8, 2017

Posted by aquillam in teaching.
Tags: , , ,
add a comment

“Gameful” is a model where students build points by completing talks and assessments. In a fully gameful class class, students have an array of options with point values associated with them and they are told what score they have to achieve to earn a particular grade. They then have the freedom to choose which tasks to do and how hard they want to work to achieve a particular grade. Many classes are only partially gameful. Rather than full flexibility, students must complete certain tasks (e.g. a midterm and final) but then have options in other areas (e.g. out of 20 at-home assignments, complete 10.) Some material may even be optional (e.g. you have to know the difference between a terrestrial and jovian planet for the exam, but you have the option of completing a project to assess whether 10 exoplanets are likely to be terrestrial, jovian, or something else). This means some students will need extra resource material, and a different set of assignments. Getting Canvas to do that takes a lot of planning and preparation, but is very possible.

Before deciding to go gameful there are two very important things to consider. First is that it requires a lot of planning and preparation up front. If you’re doing this 2 weeks before the start of the semester, stop. Run a traditional class. Do your planning for next term along the way. Feel free to share the optional materials with your students and create alternate assignments along the way this term, but please do not try to go gameful on short notice. The second thing is that it works best for students with good executive skills. The easily distracted, non-prioritizers and procrastinators have a high rate of failure. They put off doing assignments and skip the exam because they think they can make up for it later in the term. Are you prepared to deal with the student who thinks it’ll be ok it they don’t do any work for the first half of the class and comes in crying and begging for a C- the day grades are due? If not, you may not want to go fully gameful. You can always sneak it in as an array of n projects that can replace up to m homework assignments, or by providing 2 or three options for each week’s assignment.

The most important step is planning out what you want, because that will determine what tools you choose. No matter which choice you make, it’s important to know how many points you will have and how they align to each letter grade. After all, the idea is to start from 0 and accumulate points, not maintain some percentile with respect to the other students. It’s common for gamified classes to have huge numbers of points, similar to a video game. Once you know how many points students need for a particular grade, you can enable a course grading scheme. If you do a custom scheme, you can add extra categories, like “enrolled” for 0 – 10%, “member” for 10-15%, “participant” for 15 – 20%… so students don’t have to see “E” for half the term.

If you want the fully gameful, rack up the points option, you may want to consider gradecraft. Gradecraft allows students to accumulate points and earn badges for completing tasks, reaching milestones, and achieving learning goals. The badges in particular can be useful motivators to keep procrastinators on track. It supposedly integrates with Canvas now, so you can still use Canvas as the single portal to everything.

If you want a mixed class where some assignments are required and others are optional, you want the regular Grades. Racking up a score works best if you don’t use weighted assignment categories. Even if you have categories, if you don’t weight them, you can choose to have Grades display points instead of percent. This makes it easier for students to tell if they’ve reached the score they want.

If you want to use weighted categories, students will see their total grade as a percent. This makes it VERY important to get all the assignments in and published at the beginning of the term, even if they aren’t available to the students. Otherwise as you add assignments it looks to the students like they are loosing points. However, weighed categories makes it easier to make sure your required assignments carry the proper weight at the end of the term. Also, if you want to provide a menu and let them pick n of m assignments to complete, you want categories. Within each category, you’ll set a rule to drop (m-n) assignments.

Once you have your gradebook / Assignments set up, you’ll have to decide how students should access them.

Assignments is the most straight forward way. For each Assignment, you can set an availability and due date. Students won’t see it until it becomes available, and it will show up it their calendar in due date order. However, it doesn’t easily show things like “do one of the following”, nor does it show upcoming big projects, or related resources. For that reason you may want to create a page or use Modules.

Pages are basically webpages. They are probably the simplest solution if you want to provide a menu of options. In particular, you can create a page with a table showing the options for this week, resources needed for each options, and the things you think students should be working on right now.

Modules allow an enormous amount of control. You can choose to do anything from a list of associated assignments and resources to a branching set based on time and performance. For example, you can set up a module so students see what the reading is, then get access to a reading quiz the day before class. If the quiz is self graded, you can set it so they have to get a specific minimum score before they can move on to the next item, maybe the in-class assignment. Alternatively, they could get some additional reading and a prompt to take a similar quiz before they get access to the next in-class assignment and homework. Or, if they get over some higher score, they skip the second reading and quiz and are given a choice of projects to do instead of the regular homework. If you want to get very complicated (like re-doing quizzes or requiring quizzes before they can get access to their project of choice) make sure you test it out in your sandbox. In particular, test to make sure students can do things like change their mind.

Finally, keep in mind that one of the goals in gameful pedagogy is to allow students the chance to fail without failing. if you’re going to offer options like a big project to replace a couple homeworks, make sure if the student has the options and time to succeed in the class if s/he fails on that project. The point is to let them take risks and explore their interests, not to punish them for not following the traditional path.