Presentation
PART 2: In the Face of New Challenges; Effective Human Factors Engineering for Software as Medical Device Products
Event Type
Oral Presentations
TimeThursday, June 9th3:00pm - 3:22pm EDT
Location
DescriptionThe pharmaceutical and device industries are shifting focus from pure drug delivery devices to drug delivery systems, in order to meet the complex needs of patients and their care network. For us at Novo Nordisk A/S (NN), this means that looking at a wider range of medical devices than ever before, and dedicated development of Software as Medical Devices (SaMD).
At Novo Nordisk, we are on an comprehensive journey to ensure we have the competencies and processes to effectively deliver on the promise of these new systems. Among other things, this means investment in skillset across design, system engineering, risk management, and of course; human factors engineering (HFE). This presentation highlights some of the learnings and successes within the NN HFE process.
The fundamentals of HFE (and HFE regulations) applies to SaMD as well as medical devices/combination products. Even so, new challenges arise that require new processes and methods to effectively create SaMD products that are useful and meaningful to our end users, as well as safe and effective.
Design and Human Factors across operating systems:
SaMD products are defined by not being part of a hardware (medical device) [MDRF/SaMD WG/N10FINAL:2013], with the hardware prescribing the operating system (e.g. iOS or Android). While this distinction is relevant from a technical and regulatory perspective, end users should not experience a split between digital elements related to the operating system and digital elements related to the SaMD. Instead a holistic experience of both the SaMD and the operating system it is used with, should be considered to ensure a safe and enjoyable experience.
To effectively deal with this challenge, Novo Nordisk has prioritized a 2-part design strategy:
• The design affordances related to features specific to the SaMD (e.g. look and feel of App buttons) are created to resonate with users across operating systems. Features are evaluated in formative testing on and with users of the various operating systems supported by the SaMD. This strategy has proven very effective, and users with various operating systems experience note (unprompted) that the NN SaMD products are “easy” and “straightforward” to use.
• To ensure a smooth user experience, it is prioritized to utilize the native features (e.g. number pickers and system message format) that users are used to (and would expect) when using the SaMD. This is a careful consideration between creating a product that lives up to users’ expectations as consumers of digital products while being able to control that it is safe to use. NN ensures that all hazard-related use scenarios associated with a native features include appropriate product specific risk mitigations (e.g. a confirmation screen following all critical number input using a native number-picker).
Representative Human Factors validation test setup and stimuli:
HF validation test conditions should be sufficiently realistic to represent actual conditions of use and the stimuli must be production equivalent. At the same time the testing should be sufficiently generalizable to demonstrate that the results can demonstrate that the product is safe and effective.
The actual condition of use requires users to use both the SaMD product as well as their personal hardware (e.g. personal smartphone). Moreover, digital products naturally change over time (e.g. a log of activities or specific error messages related to use of the product). This presents new challenges towards meeting the test condition and stimuli requirements needed for HF validation testing.
To effectively deal with these challenges, NN has implemented new principles and processes:
• To ensure generalizable data, participants in the HF validation study are asked to use specific test phones. Data around their personal phone and phone settings will be captured and used for root cause analysis. Formative data shows that this setup is effective, and that any errors caused by this setup (i.e. study artefact) are relatively easy to identify and properly debrief.
• Some of the user interface related to hazard-related use scenarios which are difficult to trigger organically (e.g. specific error messages, or long-time use scenarios) will be presented to participants via a product simulator. I.e. not the verified product, but a simulated version of that verified product. NN will create an equivalency assessment of the product simulator compared to the verified product, to ensure the product simulator can be considered production equivalent.
Meeting expectations of both digital (consumer) products and health solutions:
This rapid shift to digital solutions means meting the expectations users may already have to both digital consumer products and health solutions – and doing so simultaneously. Manufacturers must consider the impact this has on their digital health solutions.
In NN, we tackle this in a cross-disciplinary fashion. One of the key challenges from an HFE perspective, are the pervasive (incorrect) mental models based on experience of using digital consumer products. E.g. users might have specific expectations to anything “connected” based on experiences with Bluetooth connected headphones, or internet connected streaming services, but may (unknowingly) not understand the differences between these different types of connections. While this may not be an issue with a Bluetooth headset, it may present potential for harm in health technology, and must be considered by manufacturers. Incorrect mental models have also been observed based on experience with conventional (non-digital) treatment experiences. E.g. conventional HCP-facing initiating on new treatments may be experienced vastly differently than a digital drug titration solution. Users may not fully understand key differences in what they can and cannot expect from the digital health solutions.
NN is dealing with this using an extensive formative evaluation process and by leveraging the new (user communication) opportunities in digital products.
• To ensure safe products, NN employs comprehensive formative data collection. Including contextual research towards both patients and HCPs to explore their expectation to the digital as well as health space. The results are fed into the risk management process, and risk mitigations are implemented to ensure adequately safe products.
• To set users up for success when using the products, NN use the formative data insights to scope products that are better aligned with users existing expectations (i.e. speak into existing mental models as opposed to overcome them). When this is possible, NN leverages the opportunities with digital products to facilitate an ongoing communication with users. As such, NN are continuously mapping the user journey and touchpoints with the product to provide users (focused) information when they need it.
At Novo Nordisk, we are on an comprehensive journey to ensure we have the competencies and processes to effectively deliver on the promise of these new systems. Among other things, this means investment in skillset across design, system engineering, risk management, and of course; human factors engineering (HFE). This presentation highlights some of the learnings and successes within the NN HFE process.
The fundamentals of HFE (and HFE regulations) applies to SaMD as well as medical devices/combination products. Even so, new challenges arise that require new processes and methods to effectively create SaMD products that are useful and meaningful to our end users, as well as safe and effective.
Design and Human Factors across operating systems:
SaMD products are defined by not being part of a hardware (medical device) [MDRF/SaMD WG/N10FINAL:2013], with the hardware prescribing the operating system (e.g. iOS or Android). While this distinction is relevant from a technical and regulatory perspective, end users should not experience a split between digital elements related to the operating system and digital elements related to the SaMD. Instead a holistic experience of both the SaMD and the operating system it is used with, should be considered to ensure a safe and enjoyable experience.
To effectively deal with this challenge, Novo Nordisk has prioritized a 2-part design strategy:
• The design affordances related to features specific to the SaMD (e.g. look and feel of App buttons) are created to resonate with users across operating systems. Features are evaluated in formative testing on and with users of the various operating systems supported by the SaMD. This strategy has proven very effective, and users with various operating systems experience note (unprompted) that the NN SaMD products are “easy” and “straightforward” to use.
• To ensure a smooth user experience, it is prioritized to utilize the native features (e.g. number pickers and system message format) that users are used to (and would expect) when using the SaMD. This is a careful consideration between creating a product that lives up to users’ expectations as consumers of digital products while being able to control that it is safe to use. NN ensures that all hazard-related use scenarios associated with a native features include appropriate product specific risk mitigations (e.g. a confirmation screen following all critical number input using a native number-picker).
Representative Human Factors validation test setup and stimuli:
HF validation test conditions should be sufficiently realistic to represent actual conditions of use and the stimuli must be production equivalent. At the same time the testing should be sufficiently generalizable to demonstrate that the results can demonstrate that the product is safe and effective.
The actual condition of use requires users to use both the SaMD product as well as their personal hardware (e.g. personal smartphone). Moreover, digital products naturally change over time (e.g. a log of activities or specific error messages related to use of the product). This presents new challenges towards meeting the test condition and stimuli requirements needed for HF validation testing.
To effectively deal with these challenges, NN has implemented new principles and processes:
• To ensure generalizable data, participants in the HF validation study are asked to use specific test phones. Data around their personal phone and phone settings will be captured and used for root cause analysis. Formative data shows that this setup is effective, and that any errors caused by this setup (i.e. study artefact) are relatively easy to identify and properly debrief.
• Some of the user interface related to hazard-related use scenarios which are difficult to trigger organically (e.g. specific error messages, or long-time use scenarios) will be presented to participants via a product simulator. I.e. not the verified product, but a simulated version of that verified product. NN will create an equivalency assessment of the product simulator compared to the verified product, to ensure the product simulator can be considered production equivalent.
Meeting expectations of both digital (consumer) products and health solutions:
This rapid shift to digital solutions means meting the expectations users may already have to both digital consumer products and health solutions – and doing so simultaneously. Manufacturers must consider the impact this has on their digital health solutions.
In NN, we tackle this in a cross-disciplinary fashion. One of the key challenges from an HFE perspective, are the pervasive (incorrect) mental models based on experience of using digital consumer products. E.g. users might have specific expectations to anything “connected” based on experiences with Bluetooth connected headphones, or internet connected streaming services, but may (unknowingly) not understand the differences between these different types of connections. While this may not be an issue with a Bluetooth headset, it may present potential for harm in health technology, and must be considered by manufacturers. Incorrect mental models have also been observed based on experience with conventional (non-digital) treatment experiences. E.g. conventional HCP-facing initiating on new treatments may be experienced vastly differently than a digital drug titration solution. Users may not fully understand key differences in what they can and cannot expect from the digital health solutions.
NN is dealing with this using an extensive formative evaluation process and by leveraging the new (user communication) opportunities in digital products.
• To ensure safe products, NN employs comprehensive formative data collection. Including contextual research towards both patients and HCPs to explore their expectation to the digital as well as health space. The results are fed into the risk management process, and risk mitigations are implemented to ensure adequately safe products.
• To set users up for success when using the products, NN use the formative data insights to scope products that are better aligned with users existing expectations (i.e. speak into existing mental models as opposed to overcome them). When this is possible, NN leverages the opportunities with digital products to facilitate an ongoing communication with users. As such, NN are continuously mapping the user journey and touchpoints with the product to provide users (focused) information when they need it.
Author
