Preparing Future Faculty in the Assessment of Student Learning (PFF-ASL) Fellow
The goal of this fellowship program is to improve the skills of graduate students in assessment of student learning, and thus, to prepare them for potential faculty positions. The program is funded by the Council of Graduate Schools and the Teagle and Sloan Foundations ( read the Announcement). In addition to Michigan State University, the other participating institutions are Cornell, Harvard, Indiana, North Carolina A&T, University of California-Merced, and University of North Carolina-Greensboro. Fellows of this program participated in workshops at MSU's Spring Institute on College Teaching and Learning. In break-out sessions, we worked on developing a syllabus for an introductory course that we either currently teach or would like to teach. Below I have summarized what I learned from the workshops, and check out my page on Developing a Course, which is the application of some of the things I learned at the PFF-ASL institute. |
Why assessment of student learning is so important
Terrel L. Rhodes. Vice President Association of American Colleges and Universities
The biggest point I took home from Dr. Rhodes session is the increasing interest from employers in e-portfolios. The idea behind e-portfolios is that it is a place for students to keep track of their skills and learning, and more importantly, it is a place for students to reflect on their skills and learning. A good portfolio would demonstrate the application of learning beyond a single course. The strongest portfolios incorporate assessments from instructors.
I was amused by the interest in e-portfolios because as an undergrad my college had, and then dropped, this requirement, and this was about a decade ago. However, as I think back to my undergrad, the emphasis was more on the website itself, rather than the content, and as website production became easier and less novel, the requirement was dropped. At the time, I was relieved (one less hurdle before graduation), but now I think how ahead of the game we could have been if we had thoughtful, reflective e-portfolios to present in job and grad school applications.
I was amused by the interest in e-portfolios because as an undergrad my college had, and then dropped, this requirement, and this was about a decade ago. However, as I think back to my undergrad, the emphasis was more on the website itself, rather than the content, and as website production became easier and less novel, the requirement was dropped. At the time, I was relieved (one less hurdle before graduation), but now I think how ahead of the game we could have been if we had thoughtful, reflective e-portfolios to present in job and grad school applications.
Moving Forward using Backward Design: Alignment of Learning Outcomes, Instructional Activities, and Assessments
Dr. Cori L. Fata-Hartley, Assistant Professor of Biology at Michigan State University
I have covered Backward Design before, so I will not go into the details of it again, but instead focus on the unique takes and ideas Dr. Fata-Hartley went into.
We were asked to think about and share our definition of curriculum. Mine was: the plan for the course, i.e. the topics covered, objectives, and plans for how to achieve the objectives and goals. Given definition: an academic plan to foster students' learning (Stark and Lattuca, 1997). Includes outcomes and goals, teaching and learning materials, and assessment.
We were asked to think about and share our definition of curriculum. Mine was: the plan for the course, i.e. the topics covered, objectives, and plans for how to achieve the objectives and goals. Given definition: an academic plan to foster students' learning (Stark and Lattuca, 1997). Includes outcomes and goals, teaching and learning materials, and assessment.
Designing your lectures: The common approach to instructional design by new faculty: choose the text, identify chapters to cover, develop lectures, create exams. Decisions need to be made about what chapters to cover, and what pages of the chapter, and which of the facts and vocabulary will we require the students to learn? This can lead to disconnects between what the students are reading and studying and what the big picture is. Students and instructors alike can lose sight of the main points. Instead, try Backward Design: Identify the desired results, determine acceptable evidence, plan learning experiences. I've learned about Backward Design before, but I thought the most important aspect that Dr. Fata-Hartley highlighted was the importance in looking at the evidence collected and deciding if the students achieved the desired results. If they didn't, take steps to refine the curricula, or determine if the way you are collecting evidence is appropriate for your needs. This self-assessment on the instructors part is really important and I think often lacking. I think it is also important to consider how the situation might be corrected for the current semester, not pushed off for the next semester. If your assessments or instruction missed the mark, why not address the situation so your current students don't miss out on learning what you expected?
|
Benefits to Backward Design: As a student myself, I have experienced the frustration of having to understand personal quirks of one professor to the next in addition to studying the material in order to do well in a class or exam. Why do we put students through this? Why do we sometimes make them guess what will be on the exam? It would be better for everyone to clearly let students know what is expected of them. This can be done when the instructor uses backward design carefully, so that the instruction, assessment, and goals/objectives all align, and then make these goals known to the students.
|
Dr. Fata-Hartley had us take a moment to think about how backward design is similar to how we do research in our own disciplines, and thus, is a process we are already familiar with. For example, when planning a paper for publication, you think about what it would explain,design experiments based on that, and you would think about what kind of figure you would want to produce to explain your data. In other words, you would think of your Research Question --> Identify your acceptable evidence --> Design experiment that would give you that evidence. You would have to assess whether your methods address the hypothesis and add or change experiments as necessary. Designing your course should work the same way!
Beginning at the end: Writing Effective Learning Outcomes and Objectives
Ann E. Austin, Professor of Higher, Adult, and Lifelong Learning at Michigan State University
Dr. Austin also talked about Backward Design, but in addition to this, she used a lot of ideas from Fink that I thought were very useful. I was really impressed when I first was introduced to Fink's Taxonomy of significant learning (see figure to the left), because it was the first time I saw a representation of all the different types of learning and experiences that occur in college. Most of the time the focus is on content, but so much of college is about learning how to learn and finding what you are passionate about. (You can easily see how Fink's Taxonomy would be useful in demonstrating skills in student e-portfolios, as well- it goes so much beyond what you learned for a test, to what you actually got out of all the experiences!)
Developing a course: Dr. Austin emphasized the importance in understanding situational factors (see figure below): Who are the students in this class (freshmen versus seniors/ biology and chemistry majors)? Who are their future employers? What are student expectations/is there something they are hoping to learn? Where does this particular class sit in the broader curriculum, i.e. is it a prerequisite or an elective? Knowing the answers to these questions can help you design the course appropriately.
Another interesting thought to help in developing your course: Shift the emphasis away from the instructor and to the students. When developing your course, don't think "I am going to teach these topics...", but instead consider "What should my students be able to do or demonstrate after learning X topic?".
Why do instructors write lower level objectives but then expect higher level outcomes? Consider, have you given your students all the tools to achieve what you expect from them? If you expect higher level outcomes, carefully construct your objectives so you know how to prepare your students AND your students know what to expect.
Writing good objectives:
Low level objective: Know all the parts of a neuron
Low level assessment: Draw all the structures of the neuron, or, label a drawing of a neuron.
Though it is good that the assessment aligns with the objective, you are not asking much of the students. Is this really an important lesson for them to master?
Higher level objective: Describe how the structures of the neuron are involved in synaptic transmission.
Higher level assessment: Give students a problem such as a disease where you are certain that there is a problem in synaptic transmission, but unsure of what structure or process specifically is disrupted. What tools would you use and what would you investigate? [this incorporates so many important lessons that you would cover if your instruction aligns with your assessments- they would need to know different parts of the neuron, their functions, various experimental methods, and experimental design. But again, don't assume they know all these things already, you should be providing them instruction if it is something you expect).
Tip: use taxonomy wheels to help you find action verbs for your objectives (so students know what to do or demonstrate, not just what to know/understand).
Developing a course: Dr. Austin emphasized the importance in understanding situational factors (see figure below): Who are the students in this class (freshmen versus seniors/ biology and chemistry majors)? Who are their future employers? What are student expectations/is there something they are hoping to learn? Where does this particular class sit in the broader curriculum, i.e. is it a prerequisite or an elective? Knowing the answers to these questions can help you design the course appropriately.
Another interesting thought to help in developing your course: Shift the emphasis away from the instructor and to the students. When developing your course, don't think "I am going to teach these topics...", but instead consider "What should my students be able to do or demonstrate after learning X topic?".
Why do instructors write lower level objectives but then expect higher level outcomes? Consider, have you given your students all the tools to achieve what you expect from them? If you expect higher level outcomes, carefully construct your objectives so you know how to prepare your students AND your students know what to expect.
Writing good objectives:
Low level objective: Know all the parts of a neuron
Low level assessment: Draw all the structures of the neuron, or, label a drawing of a neuron.
Though it is good that the assessment aligns with the objective, you are not asking much of the students. Is this really an important lesson for them to master?
Higher level objective: Describe how the structures of the neuron are involved in synaptic transmission.
Higher level assessment: Give students a problem such as a disease where you are certain that there is a problem in synaptic transmission, but unsure of what structure or process specifically is disrupted. What tools would you use and what would you investigate? [this incorporates so many important lessons that you would cover if your instruction aligns with your assessments- they would need to know different parts of the neuron, their functions, various experimental methods, and experimental design. But again, don't assume they know all these things already, you should be providing them instruction if it is something you expect).
Tip: use taxonomy wheels to help you find action verbs for your objectives (so students know what to do or demonstrate, not just what to know/understand).
Examples of lessons that fit into Fink's Taxonomy:
Integration: Class project idea: participate in an outreach program at some point in the semester where you need to communicate science to youngsters and turn in a write-up of your experience. Application/ Critical thinking: evaluate the validity of X theory. Application/ Creative thinking: design an experiment to test X theory Application/ Practical thinking: how would acceptance of the X theory change conventional treatment regiments? (Source: Fink LD (2003). Creating significant learning experiences: an integrated approach to designing college courses. San Francisco, Jossey-Bass.) Use 1 minute papers to get students to reflect on what they have learned, and clarify what they need more instruction on: "Summarize the most important idea from today's class", or "What was the murkiest thing that we learned in the X activity?" |
Figures in this section are from: Designing Courses for Significant Learning," L. Dee Fink, PhD, Director, Instructional Development Program, University of Oklahoma.
This item is offered under a Creative Commons Attribution-NonCommercial-ShareAlike license. |
Student Learning Assessments: Purposes, Methods, and Decisions
Dr. Larry Gruppen, Chair & Professor of Medical Education, University of Michigan
Dr. Gruppen was a wealth of knowledge with a lot of specific examples!
Be specific when writing your objectives. The more specific you are in what you want the outcome to be, the easier it will be to decide on the appropriate assessment and instruction. Examples: Students will be able to (1) Use important anatomical terms in order to describe three-dimensional points of reference (2) Present anatomical and clinical information in a concise and precise manner well enough to demonstrate understanding to peers in the discipline (3) Localize a neurological lesion on the basis of information you acquire from a patient's history and physical examination (4) Demonstrate respect for the cadaver during all phases of dissection. Different types of assessments depending on the type of outcomes: (1) if your outcome is knowledge/content based: Tests in various formats- multiple choice, essay, short answer, oral (2) higher level cognitive skills: problem sets, projects, creative writing, critical analysis, oral presentations (3) values: questionnaires, interviews, reflective essays (4) performance/skills: demonstrations, problem-solving, experimental design write-up, fieldwork, library/search skills, laboratory method How to make "Teaching to the test" a good thing, ie, make testing supportive of learning, rather than limiting (modified from UK Assessment Reform Group, 1999) (1) actively involve students in their own learning (2) take account of the results of the assessment and adjust teaching (3) recognize that assessments have profound effects on motivation and self-esteem- and these things influence learning (4) students should be able to assess themselves and understand how to improve |
Examples of Assessments:
Paraphrase to show understanding assignment. I really liked his example. I think I could also use this in a lesson about writing skills: The task: (1) Rephrase this definition of metacognition in your own terms: 'Metacognition is the awareness and understanding one's own thinking and cognitive processes'. (2) Explain how your definition is synonymous with the dictionary's definition, but is more effective for you. The holistic rubric: 0- Response is identical to dictionary definition 1- Response is confusing and is not explained, though there is some evidence of a definition 2- Response is synonymous but contains no explanation 3- Response is synonymous and contains a brief/unclear explanation of the definition 4- Response is synonymous and contains a clear, well-articulated explanation of the definition Developing a Pro and Con assignment: 1- focus on a decision, judgment, dilemma, or issue that has a teaching and learning implication in your discipline 2- write a prompt that will elicit thoughtful pros and cons 3- let students know how many pros/cons you expect and how they are to be expressed. This kind of assignment is useful because it requires student to go beyond their first reactions and research both sides of an issue, and weigh the values of opposing claims. For example: You have read several recent articles on the current debate about patenting human genetic material. from your viewpoint as a consumer, what are the principle pros and cons of allowing the patenting of genes? Come up with about 6 each. Presentations: We know a good presentation when we see one, but did we adequately prepare students beforehand? Use a multidimensional scoring rubric to help students judge for themselves what level they are performing at, before they give their presentation. For example, you might include: (1) accuracy and detail of information, (2) ability to properly articulate information, (3) flow of ideas, (4) intentionality of activities, (5) consideration of audience, time constraint. Your rubric should include descriptions that allow student to gauge whether they are meeting these competencies at a level that is 91) exemplary, (2) proficient, (3) marginal, (4) unacceptable |
Aligning Assessments with Institutional and Programmatic Learning Goals
Doug Estry, Kirk Kidwell, Jeff Grabill, Julie Libarkin
This workshop session was an interesting look into the workings of a university, and the kinds of things professors need to consider when they start up their courses. For example, Michigan State has Tier I and Tier II writing requirements, which is something that has always been a mystery to me. Taking the course "Writing, Rhetoric, and American Cultures" fulfills part of the Tier I writing requirements. Meeting Tier II writing requirements generally means that a student can write in the forms used in the discipline of their major.
Michigan State also has Institutional Learning Goals. Students are expected to gain experience at: analytical thinking, cultural understanding, effective citizenship, effective communication, and integrated reasoning, and the goal of our workshop was to be aware that these goals exist, and see how our class objectives fit into the university goals, or how these goals could be better incorporated into our classes. MSU created rubrics so we can decide what level we think the students should be able to perform at, given the objectives of the course or activity. Science courses are primed for getting students prepared for analytical thinking, integrated reasoning, and, sometimes, effective communication. One course would probably not be able to handle all the liberal learning goals well, but the overall curriculum for a science major might benefit from more engagement in cultural understanding and effective citizenship. For example, would there by better general understanding of science if scientists (and their trainees) volunteered more? Should we begin training students in science policy earlier? Should the curriculum include an ethics class?
Michigan State also has Institutional Learning Goals. Students are expected to gain experience at: analytical thinking, cultural understanding, effective citizenship, effective communication, and integrated reasoning, and the goal of our workshop was to be aware that these goals exist, and see how our class objectives fit into the university goals, or how these goals could be better incorporated into our classes. MSU created rubrics so we can decide what level we think the students should be able to perform at, given the objectives of the course or activity. Science courses are primed for getting students prepared for analytical thinking, integrated reasoning, and, sometimes, effective communication. One course would probably not be able to handle all the liberal learning goals well, but the overall curriculum for a science major might benefit from more engagement in cultural understanding and effective citizenship. For example, would there by better general understanding of science if scientists (and their trainees) volunteered more? Should we begin training students in science policy earlier? Should the curriculum include an ethics class?