Presentation
Virtual Family-Centered Rounds During the COVID-19 Pandemic – Technology Usability Analysis
Event Type
Oral Presentations
TimeThursday, June 9th11:00am - 11:30am EDT
Location
DescriptionINTRODUCTION
The American Academy of Pediatrics and the Institute for Patient and Family-Centered Care recommend family-centered rounds (FCR) is fundamental to inpatient pediatric care. FCRs consist of daily multidisciplinary care team meetings (e.g., physicians, nurses, patient, caregiver(s), pharmacist, health care professionals and learners) that take place inside, or just outside, the patient’s room. Stakeholders discuss patient status, involve patients and caregivers in decision-making and ensure everyone has a common understanding of the care plan. FCRs are standard practice in many pediatric hospitals resulting in improved patient outcomes, patient and caregiver satisfaction, care team communication, and patient safety.
In-person FCRs have been used by the Children’s Hospital of Eastern Ontario (CHEO) since 2014. FCRs take place daily and involve teams of 8-10 healthcare providers and learners meeting with patients and caregivers. With the onset of the COVID-19 pandemic, entering patient rooms was no longer advisable to maintain physical distance and preserve personal protective equipment (PPE). In response, CHEO assembled a multidisciplinary team to develop a virtual process for FCRs. A pilot project (April – July 2020) applying participatory design and rapid improvement cycles helped the team identify suitable (and readily available) technology to support Virtual Family-Centred Rounds (vFCRs). Rounds are now conducted virtually using a secure videoconferencing service (Zoom). Every morning, a rounding schedule is prepared by the unit clerk. Nurses deliver tablets and earbuds to patients and caregivers when it is their turn for vFCR and disinfect the tools between users to support infection prevention and control. Meanwhile, physicians, learners, and healthcare professionals (e.g., pharmacists) join virtually from private offices or shared spaces (e.g., meeting rooms) where physical distancing can be maintained. During this project, performance data and feedback from patients, caregivers and staff suggested the process was effective, accepted by stakeholders, and potentially useful post-pandemic.
The vFCR process continues to be used at CHEO more than 16 months later. In April 2021, a process evaluation was launched to evaluate vFCRs against the accepted standard of in-person FCRs to ensure high-quality care. Because vFCRs are dependent on technology (devices, software, infrastructure), this evaluation also considered technology usability requirements and technical factors required to facilitate implementation and adoption.
The objectives of the vFCR technology evaluation included identifying and documenting:
1. technology usability requirements through observation and evaluation of vFCR technology based on human factors principles.
2. key characteristics of vFCR technology required to support ease of use and adoption.
PARTICIPANTS & METHODS
This study involved a review of FCR frameworks and readily available virtual healthcare technologies. A human factors approach was used to identify user requirements and specific characteristics that the system should include to support human performance. Naturalistic observation was conducted to collect data on real performance in context. The study received clearance from CHEO and Carleton University’s Research Ethics Boards.
A researcher observed vFCRs conducted by pediatric medicine teams over 5 weekdays. Three days were on the unit and focused on clerk, patient, caregiver, and nurse interaction with vFCR technology. Two days were focused on observing technology interactions with physician teams joining vFCR virtually from meeting rooms. Participants included: unit clerks; patients and their caregivers; attending and learning physicians on 3 pediatric medicine teams; nurses; and healthcare professionals contributing to patient care.
The researcher did not disrupt the process and approached staff afterwards to ask questions as required. Observations focused on interactions with the technology used to support vFCR and were categorized using the AEIOU (Activities, Environment, Interactions, Object and Users) framework. Specifically, the researcher looked at what worked (i.e., requirements met) and what did not work (i.e., un-met or new requirements) in the context of the 5 quality components of usability: learnability, efficiency, memorability, errors and satisfaction.
RESULTS
Over a 5-day period, a researcher observed 35 of 45 rounds with multidisciplinary care teams. There was a total of 73 participants: 5 patients, 23 caregivers, 14 learners, 2 healthcare providers, 5 attending staff physicians, and 24 nurses. According to existing documentation, rounds should take approximately 10 minutes per patient, with a 2-minute window to collect and clean equipment, hand over the technology and transition to the next patient. Observation data revealed an average rounding time of 8.4 minutes (min: 2 minutes; max: 21 minutes), and an average of 3.7 minutes (min: 0 minutes; max: 12 minutes) to hand over technology to the next nurse/patient. Time delays were most often related to scheduling and nurse availability, not technology.
A number of technology requirements were validated or identified through observation of successful interactions (what did work). Specifically, ease of use and learnability of vFCR technology were mostly supported by the consumer technologies implemented to support vFCR (e.g., Zoom and iPads). Application settings including ’Always Display Participant Names’ and ‘Always Show Meeting Controls’ helped ensure the system was ready for ‘pickup and use’ following familiar conventions with minimal instructions. A consistent setup of vFCR tablet carts with all required equipment (e.g., headsets), cleaning supplies, and instructional tip sheets stuck directly to the carts, supported learnability and memorability. Other aspects of the vFCR equipment design including mobility and a height adjustable work surface supported efficient delivery of devices to patients and caregivers and nurse note taking.
Technology requirements were also validated or identified through observation of issues or errors (what did not work). For example, the configuration of multiple devices joined to the same videoconference in proximity (either on the unit or in shared conference rooms) was unintuitive and led to use errors, poor sound quality (when the prescribed technology setup was not followed) and confusion. Tip sheets affixed directly to the rounding carts served as effective reminders but were insufficient in the absence of an initial verbal explanation of vFCR technology setup and use. This was observed repeatedly as new and/or infrequently scheduled staff cycled through the care teams (e.g., residents and nursing students on rotation).
Observing the impact of met and unmet technology usability requirements helped identify key characteristics required to support ease of use and adoption, including: secure and reliable software and wireless network infrastructure, hands on training, and expert users within every team who can support new staff as they become familiar with less intuitive aspects of technology setup.
DISCUSSION
VFCRs depend on technology (devices, software, infrastructure). Findings on user interactions with vFCR technology suggest familiar consumer technology can support ease of use and learnability. However, when the implementation varies from familiar configurations, error and confusion can arise. Characteristics of the organization, including space constraints and limited availability of private offices, led to a more complicated and unfamiliar technical setup for physician learners joining the same videoconference from multiple devices in the same meeting room. This, in addition, to constantly changing patients, caregivers, nursing and physician learners and rotating shift schedules highlighted the importance of simple technology configurations. Signage and tips sheets can support recognition vs. recall, but hands-on training is required to support vFCR participants when optimal learnability and usability requirements are not met by technology alone. These findings suggest the impact of small usability issues should not be underestimated when implementing vFCR technology.
This case study focused on observing one site which limits generalizability and highlights difficulties in the evaluation of interventions to improve complex aspects of health care delivery. While naturalistic observation yielded clear results in terms of technology usability from the perspective of learnability, efficiency, memorability and errors, user satisfaction was not thoroughly examined. Work is underway to compare these findings with results of a patient and provider satisfaction survey, distributed to vFCR participants during the study.
The American Academy of Pediatrics and the Institute for Patient and Family-Centered Care recommend family-centered rounds (FCR) is fundamental to inpatient pediatric care. FCRs consist of daily multidisciplinary care team meetings (e.g., physicians, nurses, patient, caregiver(s), pharmacist, health care professionals and learners) that take place inside, or just outside, the patient’s room. Stakeholders discuss patient status, involve patients and caregivers in decision-making and ensure everyone has a common understanding of the care plan. FCRs are standard practice in many pediatric hospitals resulting in improved patient outcomes, patient and caregiver satisfaction, care team communication, and patient safety.
In-person FCRs have been used by the Children’s Hospital of Eastern Ontario (CHEO) since 2014. FCRs take place daily and involve teams of 8-10 healthcare providers and learners meeting with patients and caregivers. With the onset of the COVID-19 pandemic, entering patient rooms was no longer advisable to maintain physical distance and preserve personal protective equipment (PPE). In response, CHEO assembled a multidisciplinary team to develop a virtual process for FCRs. A pilot project (April – July 2020) applying participatory design and rapid improvement cycles helped the team identify suitable (and readily available) technology to support Virtual Family-Centred Rounds (vFCRs). Rounds are now conducted virtually using a secure videoconferencing service (Zoom). Every morning, a rounding schedule is prepared by the unit clerk. Nurses deliver tablets and earbuds to patients and caregivers when it is their turn for vFCR and disinfect the tools between users to support infection prevention and control. Meanwhile, physicians, learners, and healthcare professionals (e.g., pharmacists) join virtually from private offices or shared spaces (e.g., meeting rooms) where physical distancing can be maintained. During this project, performance data and feedback from patients, caregivers and staff suggested the process was effective, accepted by stakeholders, and potentially useful post-pandemic.
The vFCR process continues to be used at CHEO more than 16 months later. In April 2021, a process evaluation was launched to evaluate vFCRs against the accepted standard of in-person FCRs to ensure high-quality care. Because vFCRs are dependent on technology (devices, software, infrastructure), this evaluation also considered technology usability requirements and technical factors required to facilitate implementation and adoption.
The objectives of the vFCR technology evaluation included identifying and documenting:
1. technology usability requirements through observation and evaluation of vFCR technology based on human factors principles.
2. key characteristics of vFCR technology required to support ease of use and adoption.
PARTICIPANTS & METHODS
This study involved a review of FCR frameworks and readily available virtual healthcare technologies. A human factors approach was used to identify user requirements and specific characteristics that the system should include to support human performance. Naturalistic observation was conducted to collect data on real performance in context. The study received clearance from CHEO and Carleton University’s Research Ethics Boards.
A researcher observed vFCRs conducted by pediatric medicine teams over 5 weekdays. Three days were on the unit and focused on clerk, patient, caregiver, and nurse interaction with vFCR technology. Two days were focused on observing technology interactions with physician teams joining vFCR virtually from meeting rooms. Participants included: unit clerks; patients and their caregivers; attending and learning physicians on 3 pediatric medicine teams; nurses; and healthcare professionals contributing to patient care.
The researcher did not disrupt the process and approached staff afterwards to ask questions as required. Observations focused on interactions with the technology used to support vFCR and were categorized using the AEIOU (Activities, Environment, Interactions, Object and Users) framework. Specifically, the researcher looked at what worked (i.e., requirements met) and what did not work (i.e., un-met or new requirements) in the context of the 5 quality components of usability: learnability, efficiency, memorability, errors and satisfaction.
RESULTS
Over a 5-day period, a researcher observed 35 of 45 rounds with multidisciplinary care teams. There was a total of 73 participants: 5 patients, 23 caregivers, 14 learners, 2 healthcare providers, 5 attending staff physicians, and 24 nurses. According to existing documentation, rounds should take approximately 10 minutes per patient, with a 2-minute window to collect and clean equipment, hand over the technology and transition to the next patient. Observation data revealed an average rounding time of 8.4 minutes (min: 2 minutes; max: 21 minutes), and an average of 3.7 minutes (min: 0 minutes; max: 12 minutes) to hand over technology to the next nurse/patient. Time delays were most often related to scheduling and nurse availability, not technology.
A number of technology requirements were validated or identified through observation of successful interactions (what did work). Specifically, ease of use and learnability of vFCR technology were mostly supported by the consumer technologies implemented to support vFCR (e.g., Zoom and iPads). Application settings including ’Always Display Participant Names’ and ‘Always Show Meeting Controls’ helped ensure the system was ready for ‘pickup and use’ following familiar conventions with minimal instructions. A consistent setup of vFCR tablet carts with all required equipment (e.g., headsets), cleaning supplies, and instructional tip sheets stuck directly to the carts, supported learnability and memorability. Other aspects of the vFCR equipment design including mobility and a height adjustable work surface supported efficient delivery of devices to patients and caregivers and nurse note taking.
Technology requirements were also validated or identified through observation of issues or errors (what did not work). For example, the configuration of multiple devices joined to the same videoconference in proximity (either on the unit or in shared conference rooms) was unintuitive and led to use errors, poor sound quality (when the prescribed technology setup was not followed) and confusion. Tip sheets affixed directly to the rounding carts served as effective reminders but were insufficient in the absence of an initial verbal explanation of vFCR technology setup and use. This was observed repeatedly as new and/or infrequently scheduled staff cycled through the care teams (e.g., residents and nursing students on rotation).
Observing the impact of met and unmet technology usability requirements helped identify key characteristics required to support ease of use and adoption, including: secure and reliable software and wireless network infrastructure, hands on training, and expert users within every team who can support new staff as they become familiar with less intuitive aspects of technology setup.
DISCUSSION
VFCRs depend on technology (devices, software, infrastructure). Findings on user interactions with vFCR technology suggest familiar consumer technology can support ease of use and learnability. However, when the implementation varies from familiar configurations, error and confusion can arise. Characteristics of the organization, including space constraints and limited availability of private offices, led to a more complicated and unfamiliar technical setup for physician learners joining the same videoconference from multiple devices in the same meeting room. This, in addition, to constantly changing patients, caregivers, nursing and physician learners and rotating shift schedules highlighted the importance of simple technology configurations. Signage and tips sheets can support recognition vs. recall, but hands-on training is required to support vFCR participants when optimal learnability and usability requirements are not met by technology alone. These findings suggest the impact of small usability issues should not be underestimated when implementing vFCR technology.
This case study focused on observing one site which limits generalizability and highlights difficulties in the evaluation of interventions to improve complex aspects of health care delivery. While naturalistic observation yielded clear results in terms of technology usability from the perspective of learnability, efficiency, memorability and errors, user satisfaction was not thoroughly examined. Work is underway to compare these findings with results of a patient and provider satisfaction survey, distributed to vFCR participants during the study.