Showing 1 - 2 of 2 Research Library Publications

Reported Pediatrics Milestones (Mostly) Measure Program, Not Learner Performance

Posted: | Kimberly Hu, Patricia J. Hicks, Melissa Margolis, Carol Carraccio, Amanda Osta, Marcia L. Winward, Alan Schwartz

Academic Medicine: Volume 95 - Issue 11S - Pages S89-S94

 

Semiannually, U.S. pediatrics residency programs report resident milestone levels to the Accreditation Council for Graduate Medical Education (ACGME). The Pediatrics Milestones Assessment Collaborative (PMAC) developed workplace-based assessments of 2 inferences. The authors compared learner and program variance in PMAC scores with ACGME milestones.

Category:Assessment-Oriented Research, Links to Outcomes

The Use of Multivariate Generalizability Theory to Evaluate the Quality of Subscores

Posted: | Z. Jiang, M.R. Raymond

Applied Psychological Measurement: Volume: 42 issue: 8, page(s): 595-612

 

Conventional methods for evaluating the utility of subscores rely on reliability and correlation coefficients. However, correlations can overlook a notable source of variability: variation in subtest means/difficulties. Brennan introduced a reliability index for score profiles based on multivariate generalizability theory, designated as G, which is sensitive to variation in subtest difficulty. However, there has been little, if any, research evaluating the properties of this index. A series of simulation experiments, as well as analyses of real data, were conducted to investigate G under various conditions of subtest reliability, subtest correlations, and variability in subtest means.

Category:Assessment-Oriented Research, Reliability/Validity, Scoring