jump to navigation

ES2018 – Creative use of technology to assess students June 1, 2018

Posted by aquillam in teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was on technologies that could help with student assessment.

The session started with a discussion of what is assessment. We tend to think of it as a method of assigning scores/grades (summative assessment), but the discussion showed we really mean a lot more than that. In particular, gaging understanding, getting feedback and re-evaluation (formative assessment) were topics covered in the discussion.

There was a brief discussion of Universal Design for Learning (UDL) The idea behind UDL is that not everyone comes to the classroom with the same set of skills and concepts. If we consider the whys and hows of learning, and are intentional and explicit in our assignment design, we can mitigate disadvantages caused by differing backgrounds.  Generating engagement and providing multiple pathways to achieve flexible but well defined outcomes improves learning.

We then spent some time discussing what most people do for their classes. It was not particularly surprising that most people place a lot of emphasis on reading before class, and delivering lectures in class. These are very unidirectional, and don’t encourage engagement or deeper thinking. There are technologies that can help.

The presenter, Ebony Perouse-Harvey with CRLT, provided a handout with several resources. Here are some basics from that handout (with apologies that I didn’t take the time to copy down all the links):

  • Clickers and short response tools let you do quick checks of student understanding. Some can be used to take input from students, like allowing them to ask questions during lecture. these include iClicker, centimeter, piazza, poll Everywhere, and Google forms.
  • Word clouds can help you gage student’s exposure to subject mater and vocabulary, or topics of interest. They can also be useful in brainstorming.  Tools include Mentimeter and Poll Everywhere.
  • Mid/concept mapping help student make connections. (As a side note, I’ve looked into these before since one of the biggest problems for our labs is that students see each lab as an isolated activity, divorced from all the others and from the lecture. There are a lot of astronomy class mind maps on mindmeister and most of them are terrible. They tend to look like the table of contents to the text. Concept maps should be a powerful tool to help students make connections, like linking exoplanet discoveries back to line spectra, Kepler’s laws, and the Moons of Jupiter lab, but I haven’t seen a truly effective assignment.) Tools include bubble.us, mindMeister, and MindUp.
  • Google Docs / Canvas Collaborations (Gogle Docs portal within Canvas) – collaborate tools like text documents, spreadsheets, or presentations. This requires a whole different blog post…
  • Discussions offers threaded communication between peers or between the instructor and student. The Canvas discussion tool can be set up to have threads available to the whole class, and other threads available only to a group. Piazza also offers anonymous posting.
  • Blogging can be used to capture student ideas during a class session.
Advertisements

ES 2018 – New Canvas gradebook & quizzes May 27, 2018

Posted by aquillam in teaching.
Tags: , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This post is a summary of two sessions on new features in Canvas, the new gradebook and new quizzes. Both of these are in pilot phase right now, available under the Feature Options tab in Settings. They’ve been tested, but not at scale, so there may be bugs.

New gradebook

The new gradebook is available as an alternative to the default gradebook right now. If development continues as expected, the new gradebook will become the default gradebook in summer 2019. You can switch between the two as long as you don’t use the new functionality that the old gradebook can’t support: color labels, and the late policy. The new gradebook also has a number of other changes.

Incompatible functionality

Color labels, and the late policy are not supported in the current gradebook. Both change the database so if you use either of those features, you loose the ability to go back to the old gradebook.

Color labels (aka “status”) allow you to color code things like late, missing, and excused assignments to make them easy to identify in the gradebook. The labels are applied automatically or through Speedgrader.

Late policy allows you to set automatic deductions (e.g. 2% per hour) for assignments turned in electronically, and automatically apply a default grade for missing assignments. The documentation does not indicate how the late policy applies to paper assignments. You can also manually mark an assignment as late.

Other added features

  • “Treat ungraded as 0” will go away!
  • Final grade override (so you can make that 89.9% an A- without changing the grading scheme)
  • Assignment details (high, low, average) will go away, but there will be better analytics eventually
  • The Learning Mastery is easier to use.
  • Gradebook history will show you when grades were entered and by whom.
  • Notes column – visible to you only so you can add a note like a reminder that a challenge was received, or the student gets time and a half on the exam.
  • Unpublished assignments can appear in Grades (but don’t forget to publish them if you want students to do the assignment!) They won’t be included in the calculation.
  • Filter and sort options allow you to change the order of the assignments (e.g. group by Group, order by name) and limit what is shown (e.g. only show items in the homework group.)
  • Change the information displayed about the student (e.g. hide the sections)

There is an attendance tool that currently does not work well with the new gradebook.

Not coming soon: weighting the grades within a group. Instructure acknowledges that’s a feature many people want, so it’s “on their radar”, but isn’t coming in the next year.

New Quizzes

Aka quizzes 2.0 and quizzes.next, the new quizzes tool is an API, so you can use both old and new quizzes on the same site. After enabling it under Settings, you can create a new quiz from the Assignments tool. There were a LOT of bugs, including a few that affected grading. The biggest bug however is that items are not currently exportable, so you can’t move anything from out of the place you create it. They hope to have that fixed by fall, but you may want to create questions in another format, like old quizzes until they get the export function working.

It offers a host of new question types, including hot spot (register where on an image a user clicks – not accessible) and drag-and-drop questions. You can also set up question groups with a single “stimulus”, like an HR diagram followed by a set of multiple choice questions. It also allows you to shuffle answers on a per question basis.

The question bank is now called an item bank, and belongs to the creator, not the course. This should make it easier for an instructor to move questions to a new class, but makes it harder to share banks between instructors. Until the export function is working, this is irrelevant.

It automatically shows the results at the end of the quiz, so if you’re offering multiple attempts, you may want set a waiting period so students can’t re-take it with their first attempt in another browser window.

Modifying a quiz

If you make changes after a student takes an assessment, they will still see the original version unless you reset them. Don’t forget this when testing a quiz in student view! Also, as a tip, the presenters suggested assigning a quiz only to Test Student during testing, so you can publish it.

Changing the scoring is NOT retroactive. If students have already started taking the quiz, you’ll have to manually update the scores of anyone who has already submitted it.

 Scoring:

In the original quiz tool, the quiz score was the sum of the item scores, so you had to set the value for each graded item. in quizzes.next, you set the total points for the quiz when you create the quiz (just like any other assignment), then it sets the default item scores based on that. If you override the point value of one question, it should automatically adjust the other questions accordingly, but it was a little buggy.

Moderation will allow you to add extra time, including a multiplier, so you can ensure a student always gets time-and-a-half.

 

 

ES 2018 – Lightning talks – Active Learning May 11, 2018

Posted by aquillam in Astronomy, teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This session was a series of lightning talks on Active Learning. There were 7 presenters. Titles below are my quick reinterpretation, not the listed talk titles (lightning talks = no time to type everything, I just wanted to be sure I got the names right!)

 

Yulia Sevryugina: Strategies in Chemistry

Prof. Sevryugina described a couple strategies she uses for her large chemistry lab.

  • Syllabus game
    • Students generally don’t read the syllabus, and don’t really pay attention if you go over it during class, so this is intended to engage the students actively with the syllabus. It also models some lab classes, though she didn’t say if it modeled the way her class operates.
    • The room is set up with stations, and the class is divided into groups. The stations can be completed in any order except the last one. At each station is a printed copy of the syllabus, and question sheets. As each group visits a station, they pick up and fill out the question sheet for that station. Once they have all the sheets filled out, they go to the last station, where they meet the instructor, get a sweet, and discuss some of the questions.
    • Some of the questions are actually discussion topics, such as “find out why your other groups members are taking this class” or “share some learning strategies you think will help you succeed in this class.”
    • Questions are designed to cover FAQs, help the students get to know each other, and get to know the instructor.
    • Assessment is points for completion of the question sheets
  • Canvas Discussion / Piazza
    • The anonymous option helps make the instructor more approachable, and decreases the students’ embarrassment with asking questions in front of their peers.
  • Jigsaw classroom
    • The students are first divided into “expert” groups. Each group is given an area to study at home and become an expert in. At the next class, new groups are formed with one person from each expert group: the jigsaw. Each jigsaw group is given a task or activity to complete as a group, which can include an artifact for formal assessment.
    • final grades should generally be a group grade, but should include a component for individual preparation, so students can’t shirk the expert phase.
    • It’s usually very useful to have a decompression phase at the end. In particular, it is helpful to have a whole-class discussion to ensure students identify holes in their expertise, or weaknesses in their problem solving strategies.
    • This technique can be used to increase student preparation. It slightly decreases the workload (e.g. instead of having to understand all of Newton’s laws, each student only really has to understand one) but increases the stakes (the jigsaw group is counting on that student for the information.)

 

Eri Bell: Increasing student interest in science via a capstone research project 

Prof. Bell introduced a research project culminating in a poster session in several large astronomy classes. He showed data for Astro 101 in Fall 2017, indicating that the large scale group project did intact significantly increase student interest and confidence in science. This is a large class, 180 students, but the poster session was still manageable. He offered a few tips and takeaways.

  • leave enough time for final revisions.
  • a good presentation space is important. posters in the lecture hall stifled conversation by making it difficult to move around. Having it in a space where the department members and surrounding departments could visit gave it much more of a feeling of a real poster session.
  • require peer interactions by requiring something like two peer evaluations or reports of other posters so that students are forced to interact with the other posters
  • I’ll add that from my point of view, having a good rubric was important. In particular, we needed to make sure we were grading on content, not appearance, so having specific measurable items was important to ensuring we gave fair grades.

 

 

Jessie Lee: Gallery walk sale presentation

Previously in HRM 305, students had done a final project where they were supposed to come up with an HR plan then give presentation at end of class. However, the other students really only engaged in passive listening, so the presentations didn’t seem very valuable except as a summative assessment, not at all like the real world. Prof. Lee changed it to be more like a gallery walk (think opening night of an art exhibition, or a poster session!)

  • held in a collaborative leaning space, where there are movable tables and chairs and wall-mounted monitors. Student groups were set up at stations around each monitor, or in groups that could travel from station to station. People at each station give a 10 minute talk, then 3 min Q&A. After the Q&A, the audience can move to a new station.
  • Students provide peer feedback on the presentations. The feedback form is structured and includes specific questions, as well as an opportunity for general comments. The presenters really liked getting the feedback.

 

Carol Shannon Teach, Assess, Revise: using assessment to drive revision

~2013 the college of pharmacy noted some issue with their curriculum: students were not working effectively, reported learning skills much later in the program than they should hav, and the faculty and staff had fundamental misconceptions about what students knew when they came in to the program. For example, students have no idea how to use the internet for scholarly research.

They needed new curriculum to address these issues.They changed problem sets to more closely match real-life scenarios, tried to match the specifics of what students rally needed (e.g. pubMed & Embase search). They began using a pre-test for all students to determine what was needed, and post test to see what the retained. Students learned more in classes where they had to do pre-work, so they’ll be revising the curriculum again!

 

Ruth Lee: Google Docs in first year writing class

Prof. Li taught ENG 125 – freshmen english (required). Classes are primarily discussion sections with about 20 students per class. She used many features of Google Docs to enhance collaboration, including real-time collaboration in class. Some examples:

  • assign an essay to read with things like vocabulary, citations, other items already highlighted. The highlights were color coded. This modeled what she expected the students to learn.
  • brainstorming – groups were given a section of google doc to work on, then were able to evaluate the other groups’ work without having to share documents.
  • group commenting – everyone put in their thesis statement, then  peers comment on it
  • class reading chart – Students make comments about the readings without identifying which reading. Peers try to identify the reading and give an explanation for why they think the comment belongs with that reading.

The students interacted well together, and were more meta-aware of the writing and collaborative process. Google docs work well in the relatively small group.

Further scaffolding to develop reading and writing skills would help (there’s always more work to be done!)

 

Dave Choberka – UMMA Exchange

exchange.umma.umich.edu

UMMA has a collection database, but searching is hard because the keywords are limited (e.g. a search for Whistler or “Sea and Rain” would turn up the painting by that title, but not a search for “beach”, even though it’s a person walking on a beach.) Also, research on the collections tends to disappear after its done. The Exchange solves some of those problems.

  • you can create groups of objects, like all the things that seem to be about zombies
    • Once a group of objects is created, if you view one of the objects, it will show you all groups the object is on, and all the related objects.
    • You can select “On display” to see only objects that are actually on display in the museum.
  • You can create virtual exhibitions, which include text and resources from outside to help put the UMMA artifact in context.

 

Dominique Butler-Borruat:  “Lets talkabroad”

This is an online platform that provides trained conversation partners. It’s not just for language classes, it’s for conversation practice, including practice for professional communication. There is a fee per conversation. The conversation partners are vetted and coached before the conversations. Instructors can specify a region or the world (e.g. Prof. Butler-Borruat has students talking with people in French Guiana).

ES 2018 – keynote: Helping Students Learn In An Age Of Digital Distraction May 10, 2018

Posted by aquillam in Astronomy, teaching.
Tags: , , ,
add a comment

Enriching Scholarship is an annual series of workshops on teaching and learning at the University of Michigan. This year’s keynote address was “Helping Students Learn In An Age Of Digital Distraction” by Katie Linder. You can watch the presentation on the Enriching Scholarship website. Here is my summary of what I thought the important points were, and a little of my own commentary.

Dr Linder started the session with a website: http://www.donothingfor2minutes.com/. Try it.

Really.

There were two reasons for her use of this site. The first was to illustrate that two minutes can really feel like a long time. If you tried it, you were probably watching the count down, or tempted to check your phone, or you just gave up on it. It’s very difficult to just stop, even for two minutes. The other reason she used it was because you never know what the other people are brining into a room. Students might be worried about another class, a friend, a romantic partner, a parent, or what they’re going to wear to the bar tonight. Instructors also bring outside worries into the classroom. In fact, your list of worries might look a lot like the list of worries of your students. You don’t have to take 2 minutes of meditation at the start of every class, but an acknowledgment of the outside world and a moment to “be here now” can help everyone center and focus.

So, now that I’ve got your attention…

First and foremost, we have to get rid of the notion that “digital distraction” is a problem with “this generation.” As evidence, she offered this image of Vatican Square:

st-peters-square-before-after.jpg

It’s Vatican square.

Those people are NOT teenagers.

We need to let go of the idea that our students are the ones with a problem with digital distraction. Most of us have, at some point, experienced an event through a screen even though we were present at the event. We check email or news before getting out of bed in the morning. We text our family at lunch time. We pull out our phone during dinner to find out more about the thing we were just discussing. We feel like we’ve left an appendage behind when we forget our phone. The only real difference is that we’ve had more time to develop coping strategies and awareness.

We often focus on managing the distractions available to the students. For example, laptop bans, network restrictions, and browser lockdowns. These strategies are useful for some things, like giving online tests, but they aren’t a good model of the real world. There will be no laptop ban for meetings at the company they go work for after graduation. If we want our students to be professionals from the day they graduate, we need to prepare them to be professionals.

We can start by trying to consider what they are brining with them into the classroom. Acknowledge they have lives outside this class and try to consider that when creating assignments. Give them that moment at the beginning to re-align their minds to your class. Explain why you’re asking them to do things, especially if you implement restrictions like a laptop ban. Consider the skills they have, and the ones they need to build. And don’t assume you know what those skills are – ask them.

If you really want them to engage with your class, you have to let them find their passion. Dr. Linder asked us to think for a moment about the last time we became so engrossed in something we were learning that we lost track of the time, then share it with our neighbors. The room became very animated with discussion, and several people were anxious to share their experience. Many people did their research digitally, using resources like Google and YouTube. However, interest in the subject enabled them to ignore distractions like email, even though many people always have email open. Additionally, when discussing it, audience members became deeply engaged with each other, even if they didn’t have the same personal interest in the subject. Try to find projects that your students can become passionate about, and share your passion for your subject, and your students will be less tempted by distractions.

Of course, not everything can be that engaging, and we need to train our students for those times too. Help students reflect on learning with short assessments like minute papers and mini reflections. Design assignments incorporating grit (perseverance, especially after failure.) Praise effort over intelligence, especially in formative assessment. Talk about learning and share cognitive literature with them. Model professional behavior for them, including the use of technology like smart phones as professional tools: research shows they tend to use their phones for entertainment and communication only. If they think of their phone as entertainment, it’s harder to resist the distractions, so share strategies for distraction management. Include them in the discussion: tell them what works for you and ask what works for them.

As a first step toward this, she suggested thinking about an area of focus for your students like meta-cognition, multitasking, cognitive load, or information literacy. Pick one area to start with, and write an appendix for your syllabus that explains its importance, cites the literature, and includes class and professional etiquette. If possible, tie a brief assignment to it. See for example, this excerpt from Sharmila Murthy’s syllabus.

Additional Resources