Tests / Quizzes

[jalana]

zedx said:

So does that mean, Case isn't using a Pass/Fail system (at least for part of its curriculum) anymore?

I'm just an incoming M1, but I doubt it. The P/F system never meant that your work wasn't graded. Just that only you knew your grade on an exam, and your grade wasn't recorded except as P or F.

[starrwine]

Sara Lyn,
Well, I'm glad to know someone else will have a tablet! I like your rationale and totally agree. We'll have to compare notes once we get the computers and see if we can increase the learning curve. See you in July!

Amy

[smt15]

I did not emphazize the graded part of the multiple choice because they got rid of pass/fail. I have heard incorrect rumors that say there will be no exams at all next year, and I want incoming first years to have correct information. The grading system will continue as it is now, which is the exams are graded, you and your dean know your actual score, but only pass or fail shows up on your transcript. I hope this clears up the confusion.

[pegsie]

Updates from the latest Committee on Medical Education meeting:
-The synthesis essays that you turn in each week will be looked over by your group facilitator; although these will not be graded, he has the opportunity to help those who are having trouble.

-For each block, students will have access to approximately 200 questions from the Case SOM MCQ database. It's up to the individual how many he/she wants to do and how often.

-At the end of each block, all students will be required to take a "Cumulative Achievement Test" of retired NBME MCQs. These achievement tests are cumulative, so each subsequent block will have a slightly longer test. Although everybody is required to take the exam and the exams will be scored, the exam is not actuallly graded. Only students and their advisors will have access to the scores. Block facult will get an aggregate report.

I gotta say...having access to those NBME MCQs is key. The USMLE is a test unlike any other I've taken--er, I mean, studied for--and it's great that the incomings are going to have ample opportunity to learn how to approach the questions.

[gdr4]

Hey -- this is what was "published" today as the new evaluation system. Hope the formatting comes out ok. -Dante.

THEME OF ISSUE 10:
Student Assessment in the New Curriculum for the University Program

Assessment is designed to strengthen Dean Ralph Horwitz’s pillars of medical education and measure these elements: clinical mastery, research and scholarship, leadership, and civic professionalism. Starting with the Class of 2010 matriculating in July 2006, the University Program will adopt methods of assessment that emphasize multi-dimensional aspects of student performance relating to these pillars. Early in the curriculum, assessment will measure students’ skills in problem solving and abilities in applying medical concepts to new situations, in addition to comprehension of fundamental principles. Students are engaged in working collaboratively in small Case Inquiry Groups (IQ Groups), and their contributions to the group content and to the group process as well as their professional behaviors will be assessed in the context of these groups. Students will participate in self-assessments and, at intervals, prepare learning plans to identify areas of focus and growth. The assessment is moving to a continuous quality improvement model as we hope to strengthen skills in self-reflection and lifelong learning. Emphasis will be on longitudinal approaches tracking the student’s performance across all four years and possibly beyond medical school.

Methods of assessment planned for the Foundations of Medicine and Health in the new curriculum follow. Each will be described in turn.

Students are required to “meet criteria” for each of the following components in order to pass each block:
1. Essay Questions (both SEQs and SummSEQs)
2. Completion of formative Multiple-Choice Questions
3. Case Inquiry Group (IQ Group) assessment
4. Clinical Mastery assessment
5. Clinical Immersion assessment or End-Product for the week
6. Completion of Evidence-Based Self-Assessment of Progress (E-SAP)
Students who do not “meet criteria” may gather additional evidence of mastery of the content of the block through various means and must do so before their summative competency reviews at the end of Blocks 3 and 6.

Essay Questions (both SEQs and SummSEQS)
Synthesis Essay Questions (SEQs) were inspired by best practices during the Renal committee in the current curriculum combined with positive experiences with a similar format in the college program. These essay questions are named “synthesis” because they require that students demonstrate understanding of the material and the ability to synthesize—i.e., apply to new situations—the concepts that were taught. These are written by content specialists and are used in both formative and summative ways.
• Mid-week throughout the block, students will be given a choice between two Synthesis Essay Questions pertaining to that particular week’s learning objectives to answer in an essay by the end of the week. At the end of the week, “ideal” answers to the SEQs written by the specialists will be published. Students may compare their answers to “ideal answers.” In addition, Case Inquiry Group facilitators (“IQ Group” leaders” of the new curriculum) will review students’ answers and provide feedback. Students may confer with specialists who wrote the questions, if desired. Over the course of the block, students will be given 20 synthesis essay questions, two per week with a requirement of answering one of the two each week, as part of the formative learning experience.
• Summative SEQs (SummSEQs) are derivatives (not identical to the initial 20 SEQs). These questions require students to demonstrate conceptual understanding of the material. At the end of the block, students will take a SummSEQ test with approximately 3 to 5 SummSEQs, which will be graded by specialists.

Formative Multiple-Choice Question Examinations
• At the beginning of each block and organized by week, relevant multiple-choice questions from the Case School of Medicine test question bank will be released to students for self-assessment. Each question chosen correlates with the block’s learning objectives. In total, about 200 multiple choice-questions will be released per block, with about 20 of those assigned per week. The 20 questions will be administered online weekly to be taken at a time determined by the student. The site is unsecured and provides the student with immediate feedback as to the correct answers.

• At the end of each block, an online multiple-choice-question Cumulative Achievement Test, composed of questions from the National Board of Medical Examiners (NBME) secure test item bank, will be administered to test information that was taught during the block and help students assess their progress in preparing for the USMLE Step 1 exam. These Cumulative Achievement Tests are designed to test content drawn from the block and also include questions from previous blocks so that the exams become progressively longer. Though required, Cumulative Achievement Tests are formative and scores do not weigh into the decision as to whether a student “meets” or “does not meet” expectations for the block. Results are entered into the student’s electronic portfolio, and students may choose to disclose results. The end-of-block Cumulative Achievement Test is a timed examination administered the last Friday afternoon of the block. On June 12 and 13, NBME representatives were at Case to meet with School of Medicine faculty to create the exams for each of the six blocks of the new curriculum.

Case Inquiry Group (IQ Group) Assessment
Learner-centered Case Inquiry Groups (IQ Groups) meet three times a week for case-based inquiry and problem solving. Facilitators are required to complete assessments of small group participants during the midpoint and at the end of each of block. Students will be assessed on observable behaviors such as teamwork, preparation, quality of both questions and contributions, group dynamics/peer interaction, leadership, professionalism, attendance, etc. Peer assessment will be used formatively for feedback only and will not count toward a grade. The Case Inquiry Group (IQ Group) facilitator assessment for each student’s performance during the block will be factored into the judgment of whether or not students “meet criteria” for performance in the block.

Clinical Mastery Assessment
Beginning very early in the curriculum, students will take part in a new program initiated this past year called “Rotating Apprenticeships in Medical Practice” (RAMP). This program allows students to learn from highly respected clinical teachers and offers students learning opportunities with patients that span the life cycle. At regular intervals during the year, preceptors complete clinical evaluations charting students’ growing competence in core clinical skills.

Clinical Immersion Assessment or End-of-Block Project
Each of the six blocks in the Foundations of Medicine and Health incorporates a week of clinical experiences that include clinical assignments and activities intended to deepen and strengthen basic science concepts taught in the classroom. Students will receive credit and assessment for these end-of-immersion week projects that involve teamwork and collaboration among small groups.

Completion of Evidence-Based Self-Assessment of Progress (E-SAP)
At the end of each block, students review the learning objectives for the block and their mastery of required concepts and prepare a reflective essay. The highlights of the essay identify strengths (as perceived by the student) and areas for further learning and improvement. These are reviewed by the students’ advisers and are required as part of each block’s activities to cultivate skills in reflection and lifelong learning.

Summative Portfolio Reviews
Each student takes part in Summative Core Competency Reviews three times throughout the curriculum: at the end of Block 3, at the end of Block 6, and mid-fourth year. Students gather and assess evidence of their mastery of 9 competencies and write a reflective essay on how well they have met expectations for each competency as compared with established benchmarks. The evidence and essay are part of the student’s portfolio. The expectations, or “benchmarks,” for proficiency vary with progression through the curriculum.

Case School of Medicine Competencies for the University Program match competencies required for residency and are recast in the language of Dean Horwitz’s ‘pillars” of medical education. These are:
1. Medical Knowledge
2. Clinical Mastery
3. Interpersonal and Communication Skills
4. Civic and Personal Professionalism
5. Research and Scholarship
6. Leadership
7. Practice-Based Learning and Improvement
8. Systems-Based Practice
9. Lifelong Learning and Personal Development

The determination “meets” or “does not meet” criteria is made during each of the three review periods mentioned. Students not meeting criteria for the portfolio review are referred to the Committee on Students. This longitudinal assessment approach makes it possible to follow the student’s performance across blocks and over all four years.

Co-Chairs of the Student Assessment Committee are:

Klara K. Papp, Ph.D., [e-mail]
Director, Center for the Advancement of Medical Learning (CAML)
Senior Research Associate in the School of Medicine Office of Curricular Affairs

and

John Mieyal, Ph.D., [e-mail], Professor of Pharmacology.

The committee includes Drs. Amy Wilson-Delfosse, Terry Wolpaw, and Dan Wolpaw and Mr. Dante Roulette (Class of 2008). Together, they spearhead assessment plans in the new School of Medicine curriculum. They worked to design a system that fosters student learning and self-assessment that provides the highest quality information to students and faculty alike.