jump to navigation

ES2018 – Creative use of technology to assess students June 1, 2018

Posted by aquillam in teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on technologies that could help with student assessment.

The session started with a discussion of what is assessment. We tend to think of it as a method of assigning scores/grades (summative assessment), but the discussion showed we really mean a lot more than that. In particular, gaging understanding, getting feedback and re-evaluation (formative assessment) were topics covered in the discussion.

There was a brief discussion of Universal Design for Learning (UDL) The idea behind UDL is that not everyone comes to the classroom with the same set of skills and concepts. If we consider the whys and hows of learning, and are intentional and explicit in our assignment design, we can mitigate disadvantages caused by differing backgrounds.  Generating engagement and providing multiple pathways to achieve flexible but well defined outcomes improves learning.

We then spent some time discussing what most people do for their classes. It was not particularly surprising that most people place a lot of emphasis on reading before class, and delivering lectures in class. These are very unidirectional, and don’t encourage engagement or deeper thinking. There are technologies that can help.

The presenter, Ebony Perouse-Harvey with CRLT, provided a handout with several resources. Here are some basics from that handout (with apologies that I didn’t take the time to copy down all the links):

  • Clickers and short response tools let you do quick checks of student understanding. Some can be used to take input from students, like allowing them to ask questions during lecture. these include iClicker, centimeter, piazza, poll Everywhere, and Google forms.
  • Word clouds can help you gage student’s exposure to subject mater and vocabulary, or topics of interest. They can also be useful in brainstorming.  Tools include Mentimeter and Poll Everywhere.
  • Mid/concept mapping help student make connections. (As a side note, I’ve looked into these before since one of the biggest problems for our labs is that students see each lab as an isolated activity, divorced from all the others and from the lecture. There are a lot of astronomy class mind maps on mindmeister and most of them are terrible. They tend to look like the table of contents to the text. Concept maps should be a powerful tool to help students make connections, like linking exoplanet discoveries back to line spectra, Kepler’s laws, and the Moons of Jupiter lab, but I haven’t seen a truly effective assignment.) Tools include bubble.us, mindMeister, and MindUp.
  • Google Docs / Canvas Collaborations (Gogle Docs portal within Canvas) – collaborate tools like text documents, spreadsheets, or presentations. This requires a whole different blog post…
  • Discussions offers threaded communication between peers or between the instructor and student. The Canvas discussion tool can be set up to have threads available to the whole class, and other threads available only to a group. Piazza also offers anonymous posting.
  • Blogging can be used to capture student ideas during a class session.
Advertisements

ES 2018 – effective group assignments May 31, 2018

Posted by aquillam in teaching.
Tags: , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on creating effective group assignments.

Research shows that group work increases students’ perceptions of learning and their belief in their ability to learn, improves retention of information, increases learning of higher-order tasks, and improves scores. Additionally, more diverse groups usually work better than less diverse groups, both in terms of group outcome and individual success. Students are often resistant to group work, so it’s important to include your reasoning for requiring group work and the references in your syllabus or instructions. References were provided in this session, but only on a printed sheet.

Creating effective group assignments actually depends on two things: a good assignment and a good group. If you’re also using the assignment for assessment, you’ll also need a mechanism to evaluate the work.

Creating a good assignment

There are a couple elements to consider before creating your assignment.

The first is to make sure the assignment aligns with your learning goals. As you develop the assignment, be sure to go back and reassess whether or not the tasks still lead to that learning goal. It is best make the learning goal explicit in the assignment as well so that students know it isn’t just “busy work.” Students often don’t make connections between different aspects without some guidance.

Make sure the assignment is appropriate for a group. If you give a group the assignment to do a reading summary, chances are one person will do the summary and they’ll turn it in for the group. Instead, have them compare summaries and come up with a list of things everyone had in common, and a list of things someone thought was important but others missed. If it’s a task the average student should be able to complete alone, it’s probably not a good group task. Tasks that create interdependence are best, since they make group members reliant on each other.

Design assignments to promote higher-order cognitive skills. Again, lower level tasks like memorization are just as easy to do on your own. Group assignments provide the opportunity to test your thinking, uncover misconceptions or logical fallacies, and get different perspectives. Good assignments take advantage of those opportunities.

Good assignments usually have the following components

  • individual prep
  • a group task
  • outcome or deliverable
  • debrief

Generally, assignments are better if each of these is made explicit. Essentially, students need to know what the expectations are to perform well, and they are much more likely to develop a deeper level of learning if they are given the opportunity to think about it.

This is true even for small assignments. For example, you can give students a reading assignment (individual prep), with a think-pair-share activity in class (individual and group tasks). A flashcard or clicker response provides the outcome, then a class discussion about why students chose the responses they chose is the debrief. The syllabus should have formal instructions to students about the expectations of reading the material, working with peers, bringing a clicker/flashcard to class, and participation in in-class instruction. The debrief is an important aspect since it is the part that allows students to make sense of the answer. Without the debrief, students may not have an opportunity to do more than memorize the correct response. As long as you regularly practice these small or informal group assignments in class, the syllabus instructions are sufficient.

For larger assignments, the same guidelines still apply. The primary difference is that there are likely to be several stages with a different set of individual preparation and group tasks for each stage. Each stage may also have it’s own deliverable, separate from the final outcome or deliverable. It’s also important to debrief periodically to ensure all team members are meeting the expectations of both the instructor and their teammates. One of the reasons students are so resistant to teamwork is due to a bad experience in the past, where there were problems with a team that went unrecognized or unresolved.

Forming groups

Group formation depends heavily on the assignment. In general, it is better to have groups with some diversity in them, which usually means not allowing students to self-select groups. However, the assignment and group size affect this.

Small assignments with small groups, like think-pair-share in class questions, are generally fine with self selected groups. If nothing else, class time would be lost waiting for students to change seats. You can always ask students to sit in adjacent seats at the start of class, and have them turn to a different neighbor for each question.

Similarly, projects that extend thorough  a single class period but not beyond (e.g. a lab) are unlikely to benefit significantly over the time it takes to assign and organize groups. However, splitting up cliques and social groups may be beneficial. Randomly selected groups can often work well in these cases. For example, have students count off by the number of groups needed before starting on the activity (i.e. a class of 24 needs 6 groups, so students could off 1 – 6, then all 1s work together, 2s work together, etc.) Use caution with this approach. While diversity is good, you should be careful about isolating certain individuals. Students for whom English is a second language are especially at risk of isolation when placed in a group of native english speakers. Women, minorities, and those with physical or learning impairments can also be negatively impacted. You’ll have to weigh the effort of creating not-quite-random groups with the composition of your class.

Long term or major projects work best with carefully considered groups. In particular, the greatest gains are made when groups have heterogenous but not too dissimilar abilities. For example, putting a A student in with a B student forces the A student to explain, which can help clarify thinking and deepen understanding. The B student may gain strategies or insights from the A student. Similarly, a physics major can help the group understand the science, and an english major can help the group express their ideas clearly. When creating the groups, it’s important to try and match interests and motivation. If two of the four group members are deeply invested in the idea that Pluto is a planet, and the other two group members want to explore Mars, the group may not form a cohesive unit.

CATME Team generation software can be used to collect data on students and generate teams based on criteria you specify. It is NOT free, but you can contact ISS for resources to help cover the cost for a U-M course. As far as they know, this is the only software that can generate groups. You can also use Canvas surveys, Google forms, Qualtrics, SurveyMonkey, or other software to collect the data you want, then form the teams manually.

Any group that lasts more than a single class, and any assignment worth more than a minuscule amount  of credit, especially if it’s group credit, requires some form of group assessment and recourse for dysfunction. Assess early and often. You want to catch problems in time to correct them, not after the group implodes.

Assessing groups

When generating a score for group assignments, you want to measure both individual performance and group performance. This should include peer evaluation whenever possible, but keep in mind that teams of two will never have anonymity. Be sure share the assessment protocols at beginning of semester or assignment  (i.e. BE TRANSPARENT!) If you want honest evaluations, you need to employ non competitive grading. A rubric with options like “meets expectations” can be good, but you may want to consider requiring students to justify high or low responses. Evaluations should be goal oriented, so there are concrete outcomes to evaluate. Note some of the people in the session did have competitions, so the students could select things like “most visually appealing project” or “most helpful teammate”, but these did NOT enter into the assignment scores.

CATME includes peer evaluation tools and allows you to require both self evaluation and peer evaluation. Teammates is another full featured tool for peer evaluation and feedback. See http://teammatesv4.appspot.com/. There is also the peer evaluations in Canvas.

Take home points

  • assignments with meaningful interdependence foster student interaction
  • intentional structuring of teams promotes inclusion & learning
  • peer eval is a valuable skill and provides evidence to instructor about group dynamics

ES 2018 – New Canvas gradebook & quizzes May 27, 2018

Posted by aquillam in teaching.
Tags: , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This post is a summary of two sessions on new features in Canvas, the new gradebook and new quizzes. Both of these are in pilot phase right now, available under the Feature Options tab in Settings. They’ve been tested, but not at scale, so there may be bugs.

New gradebook

The new gradebook is available as an alternative to the default gradebook right now. If development continues as expected, the new gradebook will become the default gradebook in summer 2019. You can switch between the two as long as you don’t use the new functionality that the old gradebook can’t support: color labels, and the late policy. The new gradebook also has a number of other changes.

Incompatible functionality

Color labels, and the late policy are not supported in the current gradebook. Both change the database so if you use either of those features, you loose the ability to go back to the old gradebook.

Color labels (aka “status”) allow you to color code things like late, missing, and excused assignments to make them easy to identify in the gradebook. The labels are applied automatically or through Speedgrader.

Late policy allows you to set automatic deductions (e.g. 2% per hour) for assignments turned in electronically, and automatically apply a default grade for missing assignments. The documentation does not indicate how the late policy applies to paper assignments. You can also manually mark an assignment as late.

Other added features

  • “Treat ungraded as 0” will go away!
  • Final grade override (so you can make that 89.9% an A- without changing the grading scheme)
  • Assignment details (high, low, average) will go away, but there will be better analytics eventually
  • The Learning Mastery is easier to use.
  • Gradebook history will show you when grades were entered and by whom.
  • Notes column – visible to you only so you can add a note like a reminder that a challenge was received, or the student gets time and a half on the exam.
  • Unpublished assignments can appear in Grades (but don’t forget to publish them if you want students to do the assignment!) They won’t be included in the calculation.
  • Filter and sort options allow you to change the order of the assignments (e.g. group by Group, order by name) and limit what is shown (e.g. only show items in the homework group.)
  • Change the information displayed about the student (e.g. hide the sections)

There is an attendance tool that currently does not work well with the new gradebook.

Not coming soon: weighting the grades within a group. Instructure acknowledges that’s a feature many people want, so it’s “on their radar”, but isn’t coming in the next year.

New Quizzes

Aka quizzes 2.0 and quizzes.next, the new quizzes tool is an API, so you can use both old and new quizzes on the same site. After enabling it under Settings, you can create a new quiz from the Assignments tool. There were a LOT of bugs, including a few that affected grading. The biggest bug however is that items are not currently exportable, so you can’t move anything from out of the place you create it. They hope to have that fixed by fall, but you may want to create questions in another format, like old quizzes until they get the export function working.

It offers a host of new question types, including hot spot (register where on an image a user clicks – not accessible) and drag-and-drop questions. You can also set up question groups with a single “stimulus”, like an HR diagram followed by a set of multiple choice questions. It also allows you to shuffle answers on a per question basis.

The question bank is now called an item bank, and belongs to the creator, not the course. This should make it easier for an instructor to move questions to a new class, but makes it harder to share banks between instructors. Until the export function is working, this is irrelevant.

It automatically shows the results at the end of the quiz, so if you’re offering multiple attempts, you may want set a waiting period so students can’t re-take it with their first attempt in another browser window.

Modifying a quiz

If you make changes after a student takes an assessment, they will still see the original version unless you reset them. Don’t forget this when testing a quiz in student view! Also, as a tip, the presenters suggested assigning a quiz only to Test Student during testing, so you can publish it.

Changing the scoring is NOT retroactive. If students have already started taking the quiz, you’ll have to manually update the scores of anyone who has already submitted it.

 Scoring:

In the original quiz tool, the quiz score was the sum of the item scores, so you had to set the value for each graded item. in quizzes.next, you set the total points for the quiz when you create the quiz (just like any other assignment), then it sets the default item scores based on that. If you override the point value of one question, it should automatically adjust the other questions accordingly, but it was a little buggy.

Moderation will allow you to add extra time, including a multiplier, so you can ensure a student always gets time-and-a-half.

 

 

ES 2018 – Lightning talks – Active Learning May 11, 2018

Posted by aquillam in Astronomy, teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was a series of lightning talks on Active Learning. There were 7 presenters. Titles below are my quick reinterpretation, not the listed talk titles (lightning talks = no time to type everything, I just wanted to be sure I got the names right!)

 

Yulia Sevryugina: Strategies in Chemistry

Prof. Sevryugina described a couple strategies she uses for her large chemistry lab.

  • Syllabus game
    • Students generally don’t read the syllabus, and don’t really pay attention if you go over it during class, so this is intended to engage the students actively with the syllabus. It also models some lab classes, though she didn’t say if it modeled the way her class operates.
    • The room is set up with stations, and the class is divided into groups. The stations can be completed in any order except the last one. At each station is a printed copy of the syllabus, and question sheets. As each group visits a station, they pick up and fill out the question sheet for that station. Once they have all the sheets filled out, they go to the last station, where they meet the instructor, get a sweet, and discuss some of the questions.
    • Some of the questions are actually discussion topics, such as “find out why your other groups members are taking this class” or “share some learning strategies you think will help you succeed in this class.”
    • Questions are designed to cover FAQs, help the students get to know each other, and get to know the instructor.
    • Assessment is points for completion of the question sheets
  • Canvas Discussion / Piazza
    • The anonymous option helps make the instructor more approachable, and decreases the students’ embarrassment with asking questions in front of their peers.
  • Jigsaw classroom
    • The students are first divided into “expert” groups. Each group is given an area to study at home and become an expert in. At the next class, new groups are formed with one person from each expert group: the jigsaw. Each jigsaw group is given a task or activity to complete as a group, which can include an artifact for formal assessment.
    • final grades should generally be a group grade, but should include a component for individual preparation, so students can’t shirk the expert phase.
    • It’s usually very useful to have a decompression phase at the end. In particular, it is helpful to have a whole-class discussion to ensure students identify holes in their expertise, or weaknesses in their problem solving strategies.
    • This technique can be used to increase student preparation. It slightly decreases the workload (e.g. instead of having to understand all of Newton’s laws, each student only really has to understand one) but increases the stakes (the jigsaw group is counting on that student for the information.)

 

Eri Bell: Increasing student interest in science via a capstone research project 

Prof. Bell introduced a research project culminating in a poster session in several large astronomy classes. He showed data for Astro 101 in Fall 2017, indicating that the large scale group project did intact significantly increase student interest and confidence in science. This is a large class, 180 students, but the poster session was still manageable. He offered a few tips and takeaways.

  • leave enough time for final revisions.
  • a good presentation space is important. posters in the lecture hall stifled conversation by making it difficult to move around. Having it in a space where the department members and surrounding departments could visit gave it much more of a feeling of a real poster session.
  • require peer interactions by requiring something like two peer evaluations or reports of other posters so that students are forced to interact with the other posters
  • I’ll add that from my point of view, having a good rubric was important. In particular, we needed to make sure we were grading on content, not appearance, so having specific measurable items was important to ensuring we gave fair grades.

 

 

Jessie Lee: Gallery walk sale presentation

Previously in HRM 305, students had done a final project where they were supposed to come up with an HR plan then give presentation at end of class. However, the other students really only engaged in passive listening, so the presentations didn’t seem very valuable except as a summative assessment, not at all like the real world. Prof. Lee changed it to be more like a gallery walk (think opening night of an art exhibition, or a poster session!)

  • held in a collaborative leaning space, where there are movable tables and chairs and wall-mounted monitors. Student groups were set up at stations around each monitor, or in groups that could travel from station to station. People at each station give a 10 minute talk, then 3 min Q&A. After the Q&A, the audience can move to a new station.
  • Students provide peer feedback on the presentations. The feedback form is structured and includes specific questions, as well as an opportunity for general comments. The presenters really liked getting the feedback.

 

Carol Shannon Teach, Assess, Revise: using assessment to drive revision

~2013 the college of pharmacy noted some issue with their curriculum: students were not working effectively, reported learning skills much later in the program than they should hav, and the faculty and staff had fundamental misconceptions about what students knew when they came in to the program. For example, students have no idea how to use the internet for scholarly research.

They needed new curriculum to address these issues.They changed problem sets to more closely match real-life scenarios, tried to match the specifics of what students rally needed (e.g. pubMed & Embase search). They began using a pre-test for all students to determine what was needed, and post test to see what the retained. Students learned more in classes where they had to do pre-work, so they’ll be revising the curriculum again!

 

Ruth Lee: Google Docs in first year writing class

Prof. Li taught ENG 125 – freshmen english (required). Classes are primarily discussion sections with about 20 students per class. She used many features of Google Docs to enhance collaboration, including real-time collaboration in class. Some examples:

  • assign an essay to read with things like vocabulary, citations, other items already highlighted. The highlights were color coded. This modeled what she expected the students to learn.
  • brainstorming – groups were given a section of google doc to work on, then were able to evaluate the other groups’ work without having to share documents.
  • group commenting – everyone put in their thesis statement, then  peers comment on it
  • class reading chart – Students make comments about the readings without identifying which reading. Peers try to identify the reading and give an explanation for why they think the comment belongs with that reading.

The students interacted well together, and were more meta-aware of the writing and collaborative process. Google docs work well in the relatively small group.

Further scaffolding to develop reading and writing skills would help (there’s always more work to be done!)

 

Dave Choberka – UMMA Exchange

exchange.umma.umich.edu

UMMA has a collection database, but searching is hard because the keywords are limited (e.g. a search for Whistler or “Sea and Rain” would turn up the painting by that title, but not a search for “beach”, even though it’s a person walking on a beach.) Also, research on the collections tends to disappear after its done. The Exchange solves some of those problems.

  • you can create groups of objects, like all the things that seem to be about zombies
    • Once a group of objects is created, if you view one of the objects, it will show you all groups the object is on, and all the related objects.
    • You can select “On display” to see only objects that are actually on display in the museum.
  • You can create virtual exhibitions, which include text and resources from outside to help put the UMMA artifact in context.

 

Dominique Butler-Borruat:  “Lets talkabroad”

This is an online platform that provides trained conversation partners. It’s not just for language classes, it’s for conversation practice, including practice for professional communication. There is a fee per conversation. The conversation partners are vetted and coached before the conversations. Instructors can specify a region or the world (e.g. Prof. Butler-Borruat has students talking with people in French Guiana).