Rapid, intensive change in the health care system requires a change in how the outcomes of medical education are measured. The post-Flexner model of evaluating the institution focuses on teaching (rather than learning) and is not relevant to ambulatory training. Medical education needs to use assessment that focuses on student learning, can respond quickly to changes, and is continuously refined by use of information about students' abilities. Course-based assessment is a systematic way to gather, analyze, disseminate, and use course data to improve student learning, and it is well suited to current trends in health professions education. The authors present and discuss six principles of effective outcome assessment in medical education and illustrate these principles from their experience of using this type of assessment in a pre-clinical course integrating basic science, social science, and clinical skills. Courses and programs for medical students and residents should use course-based assessment to ensure that medical trainees acquire the skills and attitudes needed to practice medicine well. The next challenge for medical educators is to implement course-based assessment that is intimately tied to a broader institutional plan.