Validation of ORCA Outcome Measure

Quick Overview

In this presentation, Bryce Reeve discusses the validation of the observed reported communication ability measure (ORCA) for children with Angelman Syndrome. The ORCA measure was developed in response to the need for a reliable and comprehensive communication measure that can be used in clinical trials. Existing measures had limitations such as the need for a speech language pathologist, reliance on verbal speech, and inability to differentiate between children with Angelman Syndrome. The development of the ORCA measure involved qualitative interviews with parents and caregivers, drafting and cognitive testing of the questionnaire, and psychometric testing for validity and reliability. The ORCA measure showed a normal distribution of scores and strong associations with other communication scales. It is currently being disseminated and implemented in clinical trials and has garnered interest from other rare neurodevelopmental disorders. The ORCA measure assesses expressive, receptive, and pragmatic communication and can be completed by caregivers independently in 15-20 minutes.

Introduction

In this talk, Bryce Reeve, a professor at the Duke University School of Medicine, discusses the validation of the observed reported communication ability measure, also known as the ORCA measure. The ORCA measure was developed in response to the need for an effective measure of communication to evaluate the effectiveness of therapies for children with Angelman Syndrome. This talk provides an overview of the limitations of existing communication measures and outlines the steps taken to develop and validate the ORCA measure.

Background

The FAST organization, in collaboration with Duke University School of Medicine, conducted a survey in 2018 to determine the outcomes that parents and caregivers of children with Angelman Syndrome considered important indicators of therapy effectiveness. The survey revealed that speech and communication ability were consistently identified as crucial outcomes. With this knowledge, the FAST organization aimed to identify an existing communication measure that met the qualifications set by the Food and Drug Administration (FDA).

Limitations of Existing Measures

A review of existing communication measures revealed several limitations. Many of these measures required the involvement of speech-language pathologists or communication experts, making them impractical for use in clinical trials. Additionally, most measures were not developed specifically for clinical trials or therapy purposes. They also relied heavily on verbal speech as an indicator of communication ability, which posed a challenge for nonverbal children with Angelman Syndrome. Furthermore, existing measures often failed to differentiate between children with Angelman Syndrome, resulting in a uniform low score for most children.

Development of the ORCA Measure

To address the limitations of existing measures, the research team, consisting of speech-language pathologists and experts in quantitative and qualitative methods, developed the ORCA measure. The development process followed established guidelines for survey development and evaluation. The team conducted qualitative interviews with parents and caregivers to gain a deeper understanding of how children with Angelman Syndrome communicate. This information was used to draft the ORCA measure, which underwent cognitive testing to ensure clarity and relevance.

Psychometric Testing and Findings

Quantitative testing, known as psychometric testing, was conducted to collect data on the ORCA measure from over 250 parents or caregivers of children with Angelman Syndrome. The testing aimed to assess the validity and reliability of the ORCA measure. The results showed a distribution of scores that indicated a range of communication abilities among children with Angelman Syndrome, addressing the floor effect observed in existing measures. The ORCA measure demonstrated strong associations with other communication scales and showed good internal consistency and test-retest reliability.

Dissemination and Implementation

The ORCA measure is now being disseminated under a non-commercial publicly available license. It has garnered interest not only within the Angelman Syndrome community but also in other rare neurodevelopmental disorders. Several organizations have already started using or expressed interest in using the ORCA measure in their research. The research team is actively working on translating the measure into multiple languages and collaborating with other organizations to collect additional data and expand its use.

Conclusion

The validation process has provided strong evidence for the validity and reliability of the ORCA measure in assessing communication abilities in children with Angelman Syndrome. The ORCA measure offers a comprehensive assessment of expressive, receptive, and pragmatic communication and accommodates various communication modalities. It can be completed by caregivers without the need for a communication expert and is designed for use in clinical trials. The development of the ORCA measure was made possible by the contributions and feedback of the Angelman Syndrome community.