Presentation
Conducting and Reporting a Summative Validation on a Complex System, with Participant Teams, in Record Time….DURING A PANDEMIC
Event Type
Oral Presentations
TimeWednesday, June 8th1:00pm - 1:30pm EDT
Location
DescriptionMedical devices that are systems of systems, such as a surgical robotic platform, are extremely complex in that they can contain hundreds of user interfaces and use steps to utilize. To add to that complexity, some require teams of clinicians to use, who must work (and communicate) together to achieve safe and effective outcomes. In and of itself, this would present an extreme challenge to complete usability validation that would make many human factors engineers anxious. Now add to that complexity – a global pandemic has occurred that will adversely affect your ability to recruit teams; a delay to start the validation due to that pandemic has now shortened the schedule that your team has to complete testing and reporting in time for submission; and members of your test team become intermittently unavailable due to COVID quarantines.
But that’s exactly what the collaborative team of human factors engineers at Medtronic and Cambridge Consultants achieved when they validated the Hugo RAS Surgical Robotics platform.
There were many challenging lessons and innovative ideas we experienced to achieve this monumental goal that we feel would greatly benefit the human factors community to learn from our mistakes and our successes. In this presentation, we propose to discuss:
1) How the complexity of the system (and its complex use cases) drove study design considerations to inject realism, identify and develop critical tasks, and interconnect specific tasks to provide naturalistic flow and testing efficiency.
2) Approaching study data collection and observation when working with surgical teams, utilizing supplemental data collectors and observers to ensure all participants were being watched, real-time data review to help reduce missed events, and real-time video review as needed.
3) The utilization of non-human factors practitioners and engineers to supplement intermittent study team shortages due to quarantines, and how we leveraged their skills to help us complete testing and reporting.
4) To accommodate a condensed project schedule, we developed a novel way to author the validation report WHILE we were still conducting usability testing (rather than the traditional method of test then report). We’ll share how we created silos of teams responsible for managing and collecting huge amounts of data, authoring, triaging issues, and risk acceptability reviews. This posed many challenges and learnings of what to do and what not to do.
But that’s exactly what the collaborative team of human factors engineers at Medtronic and Cambridge Consultants achieved when they validated the Hugo RAS Surgical Robotics platform.
There were many challenging lessons and innovative ideas we experienced to achieve this monumental goal that we feel would greatly benefit the human factors community to learn from our mistakes and our successes. In this presentation, we propose to discuss:
1) How the complexity of the system (and its complex use cases) drove study design considerations to inject realism, identify and develop critical tasks, and interconnect specific tasks to provide naturalistic flow and testing efficiency.
2) Approaching study data collection and observation when working with surgical teams, utilizing supplemental data collectors and observers to ensure all participants were being watched, real-time data review to help reduce missed events, and real-time video review as needed.
3) The utilization of non-human factors practitioners and engineers to supplement intermittent study team shortages due to quarantines, and how we leveraged their skills to help us complete testing and reporting.
4) To accommodate a condensed project schedule, we developed a novel way to author the validation report WHILE we were still conducting usability testing (rather than the traditional method of test then report). We’ll share how we created silos of teams responsible for managing and collecting huge amounts of data, authoring, triaging issues, and risk acceptability reviews. This posed many challenges and learnings of what to do and what not to do.
Authors
