Lessons learned from evaluating eight password nudges in the wild

Karen Renaud, Vera Zimmerman, Joseph Maguire, Steve Draper

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Downloads (Pure)

Abstract

Background. The tension between security and convenience, when creating passwords, is well established. It is a tension that often leads users to create poor passwords. For security designers, three mitigation strategies exist: issuing passwords, mandating minimum strength levels or encouraging better passwords. The first strategy prompts recording, the second reuse, but the third merits further investigation. It seemed promising to explore whether users could be subtly nudged towards stronger passwords.

Aim. The aim of the study was to investigate the influence of visual nudges on self-chosen password length and/or strength.

Method. A university application, enabling students to check course dates and review grades, was used to support two consecutive empirical studies over the course of two academic years. In total, 497 and 776 participants, respectively, were randomly assigned either to a control or an experimental group. Whereas the control group received no intervention, the experimental groups were presented with different visual nudges on the registration page of the web application whenever passwords were created. The experimental groups’ password strengths and lengths were then compared that of the control group.

Results. No impact of the visual nudges could be detected, neither in terms of password strength nor length. The ordinal score metric used to calculate password strength led to a decrease in variance and test power, so that the inability to detect an effect size does not definitively indicate that such an effect does not exist.

Conclusion. We cannot conclude that the nudges had no effect on password strength. It might well be that an actual effect was not detected due to the experimental design choices. Another possible explanation for our result is that password choice is influenced by the user’s task, cognitive budget, goals and pre-existing routines. A simple visual nudge might not have the power to overcome these forces. Our lessons learned therefore recommend the use of a richer password strength quantification measure, and the acknowledgement of the user’s context, in future studies.
Original languageEnglish
Title of host publicationProceedings of LASER 2017
Subtitle of host publicationLearning from Authoritative Security Experiment Results, Arlington, VA, USA, October 18-19, 2017
Place of PublicationBerkeley, CA
PublisherUSENIX Association
Pages25-37
Number of pages13
ISBN (Print)9781931971416
Publication statusPublished - 31 Oct 2017
EventThe LASER Workshop: Learning from Authoritative Security Experiment Results - Arlington, United States
Duration: 18 Oct 201719 Oct 2017
https://www.usenix.org/conference/laser2017

Workshop

WorkshopThe LASER Workshop: Learning from Authoritative Security Experiment Results
Abbreviated titleLASER 2017
CountryUnited States
CityArlington
Period18/10/1719/10/17
Internet address

Fingerprint

Design of experiments
Students

Cite this

Renaud, K., Zimmerman, V., Maguire, J., & Draper, S. (2017). Lessons learned from evaluating eight password nudges in the wild. In Proceedings of LASER 2017: Learning from Authoritative Security Experiment Results, Arlington, VA, USA, October 18-19, 2017 (pp. 25-37). Berkeley, CA: USENIX Association.
Renaud, Karen ; Zimmerman, Vera ; Maguire, Joseph ; Draper, Steve. / Lessons learned from evaluating eight password nudges in the wild. Proceedings of LASER 2017: Learning from Authoritative Security Experiment Results, Arlington, VA, USA, October 18-19, 2017. Berkeley, CA : USENIX Association, 2017. pp. 25-37
@inproceedings{b5464e718a5d4ceca4205b4ca1ef5958,
title = "Lessons learned from evaluating eight password nudges in the wild",
abstract = "Background. The tension between security and convenience, when creating passwords, is well established. It is a tension that often leads users to create poor passwords. For security designers, three mitigation strategies exist: issuing passwords, mandating minimum strength levels or encouraging better passwords. The first strategy prompts recording, the second reuse, but the third merits further investigation. It seemed promising to explore whether users could be subtly nudged towards stronger passwords.Aim. The aim of the study was to investigate the influence of visual nudges on self-chosen password length and/or strength.Method. A university application, enabling students to check course dates and review grades, was used to support two consecutive empirical studies over the course of two academic years. In total, 497 and 776 participants, respectively, were randomly assigned either to a control or an experimental group. Whereas the control group received no intervention, the experimental groups were presented with different visual nudges on the registration page of the web application whenever passwords were created. The experimental groups’ password strengths and lengths were then compared that of the control group.Results. No impact of the visual nudges could be detected, neither in terms of password strength nor length. The ordinal score metric used to calculate password strength led to a decrease in variance and test power, so that the inability to detect an effect size does not definitively indicate that such an effect does not exist.Conclusion. We cannot conclude that the nudges had no effect on password strength. It might well be that an actual effect was not detected due to the experimental design choices. Another possible explanation for our result is that password choice is influenced by the user’s task, cognitive budget, goals and pre-existing routines. A simple visual nudge might not have the power to overcome these forces. Our lessons learned therefore recommend the use of a richer password strength quantification measure, and the acknowledgement of the user’s context, in future studies.",
author = "Karen Renaud and Vera Zimmerman and Joseph Maguire and Steve Draper",
year = "2017",
month = "10",
day = "31",
language = "English",
isbn = "9781931971416",
pages = "25--37",
booktitle = "Proceedings of LASER 2017",
publisher = "USENIX Association",
address = "United States",

}

Renaud, K, Zimmerman, V, Maguire, J & Draper, S 2017, Lessons learned from evaluating eight password nudges in the wild. in Proceedings of LASER 2017: Learning from Authoritative Security Experiment Results, Arlington, VA, USA, October 18-19, 2017. USENIX Association, Berkeley, CA, pp. 25-37, The LASER Workshop: Learning from Authoritative Security Experiment Results, Arlington, United States, 18/10/17.

Lessons learned from evaluating eight password nudges in the wild. / Renaud, Karen; Zimmerman, Vera; Maguire, Joseph; Draper, Steve.

Proceedings of LASER 2017: Learning from Authoritative Security Experiment Results, Arlington, VA, USA, October 18-19, 2017. Berkeley, CA : USENIX Association, 2017. p. 25-37.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Lessons learned from evaluating eight password nudges in the wild

AU - Renaud, Karen

AU - Zimmerman, Vera

AU - Maguire, Joseph

AU - Draper, Steve

PY - 2017/10/31

Y1 - 2017/10/31

N2 - Background. The tension between security and convenience, when creating passwords, is well established. It is a tension that often leads users to create poor passwords. For security designers, three mitigation strategies exist: issuing passwords, mandating minimum strength levels or encouraging better passwords. The first strategy prompts recording, the second reuse, but the third merits further investigation. It seemed promising to explore whether users could be subtly nudged towards stronger passwords.Aim. The aim of the study was to investigate the influence of visual nudges on self-chosen password length and/or strength.Method. A university application, enabling students to check course dates and review grades, was used to support two consecutive empirical studies over the course of two academic years. In total, 497 and 776 participants, respectively, were randomly assigned either to a control or an experimental group. Whereas the control group received no intervention, the experimental groups were presented with different visual nudges on the registration page of the web application whenever passwords were created. The experimental groups’ password strengths and lengths were then compared that of the control group.Results. No impact of the visual nudges could be detected, neither in terms of password strength nor length. The ordinal score metric used to calculate password strength led to a decrease in variance and test power, so that the inability to detect an effect size does not definitively indicate that such an effect does not exist.Conclusion. We cannot conclude that the nudges had no effect on password strength. It might well be that an actual effect was not detected due to the experimental design choices. Another possible explanation for our result is that password choice is influenced by the user’s task, cognitive budget, goals and pre-existing routines. A simple visual nudge might not have the power to overcome these forces. Our lessons learned therefore recommend the use of a richer password strength quantification measure, and the acknowledgement of the user’s context, in future studies.

AB - Background. The tension between security and convenience, when creating passwords, is well established. It is a tension that often leads users to create poor passwords. For security designers, three mitigation strategies exist: issuing passwords, mandating minimum strength levels or encouraging better passwords. The first strategy prompts recording, the second reuse, but the third merits further investigation. It seemed promising to explore whether users could be subtly nudged towards stronger passwords.Aim. The aim of the study was to investigate the influence of visual nudges on self-chosen password length and/or strength.Method. A university application, enabling students to check course dates and review grades, was used to support two consecutive empirical studies over the course of two academic years. In total, 497 and 776 participants, respectively, were randomly assigned either to a control or an experimental group. Whereas the control group received no intervention, the experimental groups were presented with different visual nudges on the registration page of the web application whenever passwords were created. The experimental groups’ password strengths and lengths were then compared that of the control group.Results. No impact of the visual nudges could be detected, neither in terms of password strength nor length. The ordinal score metric used to calculate password strength led to a decrease in variance and test power, so that the inability to detect an effect size does not definitively indicate that such an effect does not exist.Conclusion. We cannot conclude that the nudges had no effect on password strength. It might well be that an actual effect was not detected due to the experimental design choices. Another possible explanation for our result is that password choice is influenced by the user’s task, cognitive budget, goals and pre-existing routines. A simple visual nudge might not have the power to overcome these forces. Our lessons learned therefore recommend the use of a richer password strength quantification measure, and the acknowledgement of the user’s context, in future studies.

M3 - Conference contribution

SN - 9781931971416

SP - 25

EP - 37

BT - Proceedings of LASER 2017

PB - USENIX Association

CY - Berkeley, CA

ER -

Renaud K, Zimmerman V, Maguire J, Draper S. Lessons learned from evaluating eight password nudges in the wild. In Proceedings of LASER 2017: Learning from Authoritative Security Experiment Results, Arlington, VA, USA, October 18-19, 2017. Berkeley, CA: USENIX Association. 2017. p. 25-37