Unbreakable:
Designing for Trustworthiness in Private Messaging
In the digital realm, private messaging platforms play an important role in upholding individual rights to privacy and free speech. But these digital environments are not without many harms that undermine user trustworthiness.
This report provides a balanced view of the Harms experienced by a diverse selection of participants from Colombia, Nigeria and the USA, and offers user-driven Design Opportunities to address these harms so that users feel a greater sense of safety and security.
Objectives
What is at stake? For users, private messaging can deliver offensive and inappropriate content, it can channel disinformation and “fake news”, and it can be used by nefarious actors to defraud unsophisticated or unsuspecting users.
This research looked to surface and test a preliminary set of design concepts that are likely to reduce the deleterious potential of private messaging platforms. As civil society organizations continue to push for more responsible technology, we hope our findings can be used by private messaging providers and other third party players to build on the emerging ideas and test and implement potential solutions to protect users and increase trustworthiness.
When you download an app you never check what instructions or terms and conditions agreement it has, you just install it and that's it because you need it.
A password in this day and age feels rudimentary, because if someone really wants to, they can still access your account.
Country Insights
Colombia
Participants in Colombia were most worried about the security of their accounts and personal information from hackers, fraudulent contacts, emergency scams, blackmailing and other schemes. A smaller group was also concerned about government surveillance due to an increased crackdown on dissenters and government critics.
Concerned
Activist
At Risk
Adolescent
Low Tech
Entrepreneur
Nigeria
Participants in Nigeria were acutely worried about their personal security due to widespread cases of hacking of accounts and emergency scams. They were less worried about surveillance and data mining by governments or corporations. Some users were concerned about risks related to spamming, exposure to offensive/ graphic content, and misinformation.
Authoritative
Admin
Citizen
Journalist
Low Tech
Influencers
The USA
Participants in the USA were very concerned about their privacy. While the level of concern varied, users generally agreed that the privacy should be a basic expectation. So, many users distrust messaging apps that have a clear commercial practices that infringe upon their privacy (e.g., selling advertisements).
Advantaged
Activist
Globe
Trotter
Ceremonial
Admin
Understanding Harms
The use of private messaging platforms can expose users to a range of harms that impact trustworthiness.
In this section we cover six harms that were most frequently mentioned by the users we spoke to in Nigeria, Colombia and the United States. For each one, we explore the different ways that these impact the user experience as well as the platform design gaps that can amplify these harms.
Vulnerability to adverse
mental health impacts
The negative psychological impacts that arise from using private messaging platforms.
1.
Vulnerability to manipulation (misleading montent, misinformation) or exposure to offensive content
The use of private messaging platforms to knowingly or unknowingly spread content that can be perceived as being hateful, offensive and/ or misleading.
4.
Vulnerability to hacking, scamming, blackmailing, extortion, fraud, and harassment
The direct misuse of private messaging platforms by adults as distinct from those affecting youth and children.
2.
Vulnerability to encryption and data breaches via modified and third party supporting platforms
The different ways that encrypted messaging platforms security, privacy and encryption features are bypassed by users through the use of modified and third party supporting encrypted messaging platform apps.
5.
Vulnerability to targeted harassment for youth and young adults
The use of private messaging platforms to exploit the vulnerability of youth and minors.
3.
Vulnerability to digital surveillance and monitoring
The potential use of private messaging platforms by governments and corporations to survey or monitor users.
6.
01
Securing and/ or Modifying
Account Information
Addresses:
Easy access to personal identifying data
03
Providing Accessible & Tailored Security & Privacy Controls
Addresses:
Generalized and hidden privacy & security controls for contacts & groups
05
Providing User
Support Mechanisms &
Emergency Controls
Addresses:
Limited user support and lack of adequate reporting mechanisms
06
Improving Administrative
& Management Tools
Addresses:
Limited content management tools
02
Improving Verification
& Permission Mechanisms
Addresses:
Limited verification and consent-focused features for contacts and groups
04
Managing Access to Modified & Third-Party Supporting Platforms
Addresses:
Existence of modified (MOD) & third party supporting apps ecosystem
07
Providing Data Use Transparency & Managing Data Access
Addresses:
Lack of transparency regarding access to user data
Who this report is for
Platform owners and providers
To negotiate competing product priorities and adjust product planning to address user concerns and diminishing perceptions of trust within private messaging platform experiences.
Researchers
To equip the trust and safety research field with actionable user data, and offer mixed methods methodologies focused on user experiences of private messaging platforms.
Policymakers
To better assess the risks that matter to residents and citizens related to security, democracy and information integrity, and prioritize the harms that occur on private messaging platforms to inform meaningful policy solutions.
UX
Designers
To augment their own user research and data analytics, and influence product priorities in line with user trustworthiness.
Advocacy,
Civil society
To buffer advocacy efforts with evidence of the harms a diverse set of users experience on private messaging platforms and examples of concrete changes that could improve trustworthiness.
Platform players,
Governments
To assess the potential impact of user concerns on the trustworthiness of services they offer on private messaging platforms. User perceptions will shape their trust in services offered by governments and other players