This post is the fifth and final in a five-part series drawing on Formative Assessment and Feedback in Medical Education: A Practical Guide, AMEE Guide No. 189. Each post will take a question medical educators are actively grappling with and work through what the research and the practice actually suggest.
Over the course of this series, we've covered a lot of ground: What makes an assessment genuinely formative. Why feedback often doesn't produce change. The case for a broader toolkit beyond written exams. Educators’ role in effective formative assessment in medical education. But there's a larger question at the heart of all of these ideas: Are the assessments in your program working together, or just coexisting?
It’s a crucial question to ask, because there's a big difference between a program with formative assessments versus one built around a formative assessment system.
The difference between a collection and a system
A collection of assessments is exactly how it sounds. You have OSCEs, low-stakes quizzes, structured clinical observations, peer evaluations and simulation debriefs. Each has an important part to play, but because they were designed separately, they don't necessarily build on each other. And the data they generate often lives in different places—or doesn't get used at all.
A system is different. Assessments are deliberately sequenced, and evidence from one informs the next. Feedback doesn't just close out an encounter; it sets up the conditions for the next one. When something isn't working—for an individual student or across a group of learners — the system identifies the issue early enough to act on it.
The research calls this “programmatic assessment.” The core principle is that assessment should be designed as a cohesive architecture that supports learning across the full arc of a program, not a series of isolated events that measure competence at specific times.
What it takes to build that architecture
Throughout this series we’ve addressed different components of what makes formative assessment in medical education work. Together, they describe the structural requirements of a genuine assessment system.
Continuous, aligned assessment gives you the long view. Frequent, low-stakes checkpoints tied to specific competencies let you track development over time and catch difficulties before they compound, rather than finding them at high-stakes moments.
For example, our Self-Assessments can provide curriculum-aligned evidence at key points in a student's development that faculty can use to track progress over an extended time. And NBME INSIGHTS supports that long view, giving faculty visibility into individual and group performance, and turning assessment data into actionable decisions.
Feedback that closes the loop separates monitoring from learning. Specific, timely feedback tied to learning objectives and followed by structured reflection can change performance.
A diverse assessment toolkit ensures you're getting a complete picture. Written exams measure student knowledge as an end result, while simulations and OSCEs reveal the process of how they reason and make decisions under pressure. Peer assessment builds self-evaluation skills they'll need throughout their careers.
Our Communication Learning Assessment (CLA) is an example of a purpose-built formative tool. Designed to give students structured opportunities to practice patient-centered communication when the stakes are lower, CLA allows self-reflection with built-in feedback.
Educators as facilitators, not just evaluators, make the system responsive. Instructors who model how to use feedback and treat assessment as an ongoing dialogue are the connective tissue that holds the architecture together.
The role of technology, and where it falls short
The expanding role of technology makes this kind of architecture feasible at scale. Learning management systems, feedback analytics platforms, AI-driven simulations and online assessment tools can show patterns in student performance, automate feedback on objective assessments and make longitudinal data visible in ways that weren't possible a decade ago.
Feedback analytics can find common areas where students are struggling, making it possible to intervene before a small gap becomes a persistent one. AI-driven simulations and virtual reality can simulate complex patient scenarios and adapt dynamically to a learner's performance, offering immediate, personalized feedback in a low-stakes environment.
But technology amplifies a system; it doesn't create one. Platforms and analytics tools are most powerful when they're integrated into a curriculum that’s designed around formative principles. Deployed into a collection of assessments without that underlying architecture, these technologies generate a lot of data, but not always more learning. The question isn't whether your program is using technology, but whether the technology is connected to a clear vision of students’ development.
There's also the question of equity. As digital tools become more central to assessment, it’s vital that all learners have equal access to them, and the tools are free from biases.
From individual practice to institutional commitment
The impact of formative assessment in medical education is limited when instructors are left to improvise. Faculty who invest in formative practice can make a real difference for their students, but a program built on individual efforts isn't a system.
Building a genuine formative assessment architecture requires a commitment to giving educators the tools they need to function as facilitators, embedding assessment into the curriculum, and creating a culture where faculty learn from each other as much as they do from data.
What this means in practice
Making the shift from a collection of assessments to a connected system takes sustained commitment, but most programs have more of the foundation than they realize. A good starting point is asking better questions about what you already have. That means:
Mapping your assessments against your learning objectives. Does the sequence build on itself?
Identifying where data isn’t being used. If the feedback loop isn’t closing, the system isn’t working.
Looking for toolkit competency gaps—clinical reasoning, communication and self-regulation—that your methods can’t capture.
Investing in faculty development that supports the facilitator role.
Exploring how our assessment tools and analytics can connect performance evidence across your program.
Small shifts, bigger architecture
If you're working on assessment design in your program, start by asking where your assessments connect and where they don’t. Find the places where data is generated but not used. Is feedback delivered without leading to action? Also pay attention to methods that aren’t building on each other or moving toward a clear objective. Those gaps are where you may find some of the easiest improvements.
If you’d like to build a more connected assessment architecture, our on-demand webinar, Assessment for Learning in Medical Education: Supporting Students' Growth and Development highlights the impact of formative assessment on medical education, how to implement assessment for learning into your curriculum and our research in this space.