Presentation
The Role UX Design Attributes Play in the Perceived Persuasiveness of COVID-19 Contact Tracing Apps
Event Type
Oral Presentations
TimeThursday, June 9th12:00pm - 12:30pm EDT
Location
DescriptionINTRODUCTION
Contact tracing apps (CTAs) were deployed worldwide in 2020 to combat COVID-19. Due to their low uptake, a growing amount of empirical research is being conducted to understand the factors that drive their adoption. We argue that for them to be adopted, users, first and foremost, must find them persuasive. However, there is little research to understand the role user experience (UX) plays in their perceived persuasiveness. Consequently, we conducted an online study on Amazon Mechanical Turk among Canadian and American residents (n = 446) to investigate the most important UX design attributes associated with the perceived persuasiveness of CTAs. The study was based on two app designs (control and persuasive), each of which comprises three use cases: no-exposure, exposure, and diagnosis report interfaces. One interface (screenshot) was randomly presented to a participant to view and provide their responses on the perceived UX design attributes and perceived persuasiveness of the interface.
In the overall model, we found that perceived usefulness is the most important and consistent UX design attribute that influences perceived persuasiveness (β = 0.29, p < 0.001), followed by perceived trustworthiness (β = 0.24, p < 0.001), and perceived privacy protection (β = 0.16, p < 0.05). Respectively, the three predictors were consistently significant in two-third, half, and one-third of the 12 submodels based on app design, use case, adoption status, and country of residence. Overall, the relationships regarding the persuasive designs are more likely to be significant, with the variance of the target construct explained by the predictors ranging from 71% to 89% compared with 54% to 69% for the control designs.
The three significant attributes will help designers know which UX design attributes to focus on when designing CTAs for future pandemics/epidemics. More importantly, in predictive modeling, if their ratings are known, they hold potential in predicting new users’ responsiveness to a number of persuasive strategies/messages featured in a behavior change support system.
OVERVIEW OF COVID ALERT
The study was based on COVID Alert, Canada’s official CTA deployed in July 2020. In this study, we implemented two app designs (persuasive and control), with each design comprising the three key interfaces.
No-Exposure Status Interface. The control version (C1) lets the user know that they have not been exposed by being in close contact with someone infected by COVID-19. In addition to this functionality, the persuasive version (P1) supports a self-monitoring feature that enables the user to track their daily exposure level: number and duration of contacts.
Exposure Status Interface. The control version (C2) notifies the user that they may have been exposed to COVID-19 and provide information on what to do next. In addition to this functionality, the persuasive version (P2) supports a self-monitoring feature that enables the user to track their exposure level: the total number and duration of contacts within the last 14 days.
Diagnosis Report Interface. This control version (C3) allows the user to report their one-time key given to them by the public health authority after testing positive. In addition to this functionality, the persuasive version (P3) supports a social-learning feature that enables the user to see the number of other users who have reported their COVID-19 diagnosis using the app.
METHOD
The six interfaces were randomly administered to the participants on Amazon Mechanical Turk, with each participant seeing one interface only. A total of 446 participants took part in the study: 204 Canadian residents and 242 American residents.The distribution of the interfaces among the participants is as follows: C1 (n = 74), P1 (n = 75), C2 (n = 77), P2 (n = 80), C3 (n = 72), and P3 (n = 68). The constructs measured include perceived usefulness, perceived trust, perceived security, perceived ease of use, perceived privacy protection, perceived compatibility, perceived enjoyment, and perceived persuasiveness. The first seven are predictors and the last one is the target construct. Each construct was measured using three items ranging from “strongly disagree – 1” to “strongly agree – 7.”
RESULTS
We used Partial Least Square Path Modeling to explore the relationships between each of the seven predictors and perceived persuasiveness. The overall and subgroup models are as follows:
1. Overall Model: usefulness (β = 0.29, p < 0.001), trustworthiness (β = 0.24, p < 0.001), privacy protection (β = 0.16, p < 0.05), ease of use (β = 0.12, p < 0.01).
2. Submodels based on Country of Residence:
a) Canada: usefulness (β = 0.19, p < 0.01), trustworthiness (β = 0.24, p < 0.01), privacy protection (β = 0.25, p < 0.01).
b) United States: usefulness (β = 0.43, p < 0.001), trustworthiness (β = 0.28, p < 0.05).
3. Submodels based on Adoption Status:
a) Adopters: privacy protection (β = 0.42, p < 0.01), ease of use (β = 0.10, p < 0.05).
b) Non-Adopters: usefulness (β = 0.43, p < 0.01), trustworthiness (β = 0.22, p < 0.01), ease of use (β = 0.13, p < 0.05).
4. Submodels based on App Design:
a) Control: usefulness (β = 0.30, p < 0.01), trustworthiness (β = 0.20, p < 0.01).
b) Persuasive: usefulness (β = 0.32, p < 0.01), trustworthiness (β = 0.23, p < 0.05), privacy protection (β = 0.12, p < 0.05).
5. No-Exposure Submodels based on App Design:
a) C1: No attribute is significant.
b) P1: trustworthiness (β = 0.79, p = 0.055).
6. Exposure Submodels based on App Design:
a) C2: usefulness (β = 0.41, p < 0.05).
b) P2: usefulness (β = 0.29, p < 0.05), trustworthiness (β = 0.22, p < 0.01), enjoyment (β = 0.27, p < 0.05).
7. Design Report Submodels based on App Design:
a) C3: usefulness (β = 0.31, p < 0.01).
b) P3: privacy protection (β = 0.75, p < 0.001).
DISCUSSION
Perceived usefulness and perceived trustworthiness are significantly and strongly associated with perceived persuasiveness. These associations are (near) strong, despite controlling for country of residence and app design. However, when app design and use case are controlled for, some of the significant associations become non-significant, especially regarding C1 and P1. Nevertheless, regarding C2 and P2 the association between perceived usefulness and perceived persuasiveness remains significant. Moreover, the association between perceived trust/perceived enjoyment and perceived persuasiveness remains strong regarding P2 and non-significant regarding C2. Secondly, regarding the diagnosis report interface, the association between perceived usefulness and perceived persuasiveness remain significantly strong for C3, while that between perceived privacy and perceived persuasiveness becomes strong for P3. Finally, the control for adoption status shows that perceived usefulness and perceived trustworthiness remain strong in the non-adopter models, while perceived privacy protection becomes significantly strong in the adopter model.
Overall, perceived usefulness has a more consistent strong relationship with perceived persuasiveness, followed by perceived trust, perceived privacy protection, and perceived enjoyment. The other constructs (perceived ease of use, perceived compatibility, and perceived security) have no strong or significant relationship with perceived persuasiveness.
The significantly strong relationships hold potential for building machine-learning predictive models by using the significant UX design attributes to predict first-time users’ responsiveness to persuasive strategies implemented in CTAs and other persuasive apps aimed at motivating behavior change. In other words, the significant relationships have the potential to address the cold start problem experienced by persuasive health technologies. The cold start problem is the inability for a persuasive app to know what persuasive strategies or messages that will be effective in changing a first user’s behavior. This means by asking first-time users few UX-related questions prior to using the app, their responsiveness to certain persuasive strategies and messages may be predicted.
Contact tracing apps (CTAs) were deployed worldwide in 2020 to combat COVID-19. Due to their low uptake, a growing amount of empirical research is being conducted to understand the factors that drive their adoption. We argue that for them to be adopted, users, first and foremost, must find them persuasive. However, there is little research to understand the role user experience (UX) plays in their perceived persuasiveness. Consequently, we conducted an online study on Amazon Mechanical Turk among Canadian and American residents (n = 446) to investigate the most important UX design attributes associated with the perceived persuasiveness of CTAs. The study was based on two app designs (control and persuasive), each of which comprises three use cases: no-exposure, exposure, and diagnosis report interfaces. One interface (screenshot) was randomly presented to a participant to view and provide their responses on the perceived UX design attributes and perceived persuasiveness of the interface.
In the overall model, we found that perceived usefulness is the most important and consistent UX design attribute that influences perceived persuasiveness (β = 0.29, p < 0.001), followed by perceived trustworthiness (β = 0.24, p < 0.001), and perceived privacy protection (β = 0.16, p < 0.05). Respectively, the three predictors were consistently significant in two-third, half, and one-third of the 12 submodels based on app design, use case, adoption status, and country of residence. Overall, the relationships regarding the persuasive designs are more likely to be significant, with the variance of the target construct explained by the predictors ranging from 71% to 89% compared with 54% to 69% for the control designs.
The three significant attributes will help designers know which UX design attributes to focus on when designing CTAs for future pandemics/epidemics. More importantly, in predictive modeling, if their ratings are known, they hold potential in predicting new users’ responsiveness to a number of persuasive strategies/messages featured in a behavior change support system.
OVERVIEW OF COVID ALERT
The study was based on COVID Alert, Canada’s official CTA deployed in July 2020. In this study, we implemented two app designs (persuasive and control), with each design comprising the three key interfaces.
No-Exposure Status Interface. The control version (C1) lets the user know that they have not been exposed by being in close contact with someone infected by COVID-19. In addition to this functionality, the persuasive version (P1) supports a self-monitoring feature that enables the user to track their daily exposure level: number and duration of contacts.
Exposure Status Interface. The control version (C2) notifies the user that they may have been exposed to COVID-19 and provide information on what to do next. In addition to this functionality, the persuasive version (P2) supports a self-monitoring feature that enables the user to track their exposure level: the total number and duration of contacts within the last 14 days.
Diagnosis Report Interface. This control version (C3) allows the user to report their one-time key given to them by the public health authority after testing positive. In addition to this functionality, the persuasive version (P3) supports a social-learning feature that enables the user to see the number of other users who have reported their COVID-19 diagnosis using the app.
METHOD
The six interfaces were randomly administered to the participants on Amazon Mechanical Turk, with each participant seeing one interface only. A total of 446 participants took part in the study: 204 Canadian residents and 242 American residents.The distribution of the interfaces among the participants is as follows: C1 (n = 74), P1 (n = 75), C2 (n = 77), P2 (n = 80), C3 (n = 72), and P3 (n = 68). The constructs measured include perceived usefulness, perceived trust, perceived security, perceived ease of use, perceived privacy protection, perceived compatibility, perceived enjoyment, and perceived persuasiveness. The first seven are predictors and the last one is the target construct. Each construct was measured using three items ranging from “strongly disagree – 1” to “strongly agree – 7.”
RESULTS
We used Partial Least Square Path Modeling to explore the relationships between each of the seven predictors and perceived persuasiveness. The overall and subgroup models are as follows:
1. Overall Model: usefulness (β = 0.29, p < 0.001), trustworthiness (β = 0.24, p < 0.001), privacy protection (β = 0.16, p < 0.05), ease of use (β = 0.12, p < 0.01).
2. Submodels based on Country of Residence:
a) Canada: usefulness (β = 0.19, p < 0.01), trustworthiness (β = 0.24, p < 0.01), privacy protection (β = 0.25, p < 0.01).
b) United States: usefulness (β = 0.43, p < 0.001), trustworthiness (β = 0.28, p < 0.05).
3. Submodels based on Adoption Status:
a) Adopters: privacy protection (β = 0.42, p < 0.01), ease of use (β = 0.10, p < 0.05).
b) Non-Adopters: usefulness (β = 0.43, p < 0.01), trustworthiness (β = 0.22, p < 0.01), ease of use (β = 0.13, p < 0.05).
4. Submodels based on App Design:
a) Control: usefulness (β = 0.30, p < 0.01), trustworthiness (β = 0.20, p < 0.01).
b) Persuasive: usefulness (β = 0.32, p < 0.01), trustworthiness (β = 0.23, p < 0.05), privacy protection (β = 0.12, p < 0.05).
5. No-Exposure Submodels based on App Design:
a) C1: No attribute is significant.
b) P1: trustworthiness (β = 0.79, p = 0.055).
6. Exposure Submodels based on App Design:
a) C2: usefulness (β = 0.41, p < 0.05).
b) P2: usefulness (β = 0.29, p < 0.05), trustworthiness (β = 0.22, p < 0.01), enjoyment (β = 0.27, p < 0.05).
7. Design Report Submodels based on App Design:
a) C3: usefulness (β = 0.31, p < 0.01).
b) P3: privacy protection (β = 0.75, p < 0.001).
DISCUSSION
Perceived usefulness and perceived trustworthiness are significantly and strongly associated with perceived persuasiveness. These associations are (near) strong, despite controlling for country of residence and app design. However, when app design and use case are controlled for, some of the significant associations become non-significant, especially regarding C1 and P1. Nevertheless, regarding C2 and P2 the association between perceived usefulness and perceived persuasiveness remains significant. Moreover, the association between perceived trust/perceived enjoyment and perceived persuasiveness remains strong regarding P2 and non-significant regarding C2. Secondly, regarding the diagnosis report interface, the association between perceived usefulness and perceived persuasiveness remain significantly strong for C3, while that between perceived privacy and perceived persuasiveness becomes strong for P3. Finally, the control for adoption status shows that perceived usefulness and perceived trustworthiness remain strong in the non-adopter models, while perceived privacy protection becomes significantly strong in the adopter model.
Overall, perceived usefulness has a more consistent strong relationship with perceived persuasiveness, followed by perceived trust, perceived privacy protection, and perceived enjoyment. The other constructs (perceived ease of use, perceived compatibility, and perceived security) have no strong or significant relationship with perceived persuasiveness.
The significantly strong relationships hold potential for building machine-learning predictive models by using the significant UX design attributes to predict first-time users’ responsiveness to persuasive strategies implemented in CTAs and other persuasive apps aimed at motivating behavior change. In other words, the significant relationships have the potential to address the cold start problem experienced by persuasive health technologies. The cold start problem is the inability for a persuasive app to know what persuasive strategies or messages that will be effective in changing a first user’s behavior. This means by asking first-time users few UX-related questions prior to using the app, their responsiveness to certain persuasive strategies and messages may be predicted.
