Evaluating the Construct Validity of the Interprofessional Collaborator Assessment Rubric

In Physical Medicine and Rehabilitation (PM&R), interprofessional teamwork is essential for patient care. Reliable tools are needed to assess and enhance residents' interprofessional collaboration skills, ensuring they effectively lead and participate in rehabilitation teams.Goals, objectives, and purpose: This study aimed to explore the construct validity of the Interprofessional Collaborator Assessment Rubric (ICAR).Methods/Methodology: The study involved 50 faculty members from 11 PM&R residency programs. Video-based scenarios were created to simulate various proficiency levels in leading interprofessional team meetings. Participants evaluated standardized residents' performances in twelve scripted clinical videos, demonstrating four performance levels: minimal, developing, competent, and mastery. Each level was represented by three videos, shown in random order. Faculty used the ICAR, a four-point scale tool, to assess interprofessional performance with a minimum score of 31 and maximum of 124. The residents portrayed were at the postgraduate year-3 level, using the same standardized resident for all levels.Results/Findings: A total of 600 evaluations were completed. The ICAR successfully distinguished between the four performance levels (p < .0001). Although the differences in ratings for the four levels were statistically significant, there was considerable variability in scores among the participants for each video. Minimal Competency (level 1) videos had an average score of 34.5 (average response score 1.11). Developing Competence (level 2) videos averaged 70.23 (average response score 2.27), showing slightly above developing skills. Competent videos (level 3) scored 92.75 (average response score 2.99), and Mastery videos (level 4) averaged 121.3 (average response score 3.92). No level 1 video was rated as level 4, nor vice versa. These findings suggest that the ICAR effectively differentiates between levels of interprofessional collaboration skills.Conclusions, implications, and/or curiosities: This study is the first to validate the construct validity of the ICAR. While the ICAR shows promise in terms of construct validity, additional research is required to enhance faculty observation skills and minimize interrater variability.