top of page

Unbreakable:
Designing for Trustworthiness in Private Messaging

In the digital realm, private messaging platforms play an important role in upholding individual rights to privacy and free speech. But these digital environments are not without many harms that undermine user trustworthiness.

This report provides a balanced view of the
Harms experienced by a diverse selection of participants from Colombia, Nigeria and the USA, and offers user-driven Design Opportunities to address these harms so that users feel a greater sense of safety and security.

banner-illustration (1).gif

Objectives

What is at stake? For users, private messaging can deliver offensive and inappropriate content, it can channel disinformation and “fake news”, and it can be used by nefarious actors to defraud unsophisticated or unsuspecting users. 

This research looked to surface and test a preliminary set of design concepts that are likely to reduce the deleterious potential of private messaging platforms. As civil society organizations continue to push for more responsible technology, we hope our findings can be used by private messaging providers and other third party players to build on the emerging ideas and test and implement potential solutions to protect users and increase trustworthiness.

When you download an app you never check what instructions or terms and conditions agreement it has, you just install it and that's it because you need it.

A password in this day and age feels rudimentary, because if someone really wants to, they can still access your account.

Country Insights

Colombia

Participants in Colombia were most worried about the security of their accounts and personal information from hackers, fraudulent contacts, emergency scams, blackmailing and other schemes. A smaller group was also concerned about government surveillance due to an increased crackdown on dissenters and government critics.

IMG-2091e-01.png

Concerned
Activist

IMG-2082e-01.png

At Risk
Adolescent

IMG-2083e-01.png

Low Tech
Entrepreneur

Nigeria

Participants in Nigeria were acutely worried about their personal security due to widespread cases of hacking of accounts and emergency scams. They were less worried about surveillance and data mining by governments or corporations. Some users were concerned about risks related to spamming, exposure to offensive/ graphic content, and misinformation.

Authoritative
Admin

IMG-2080e-01.png

Citizen
Journalist

IMG-2081e-01.png

Low Tech
Influencers

The USA

Participants in the USA were very concerned about their privacy. While the level of concern varied, users generally agreed that the privacy should be a basic expectation. So, many users distrust messaging apps that have a clear commercial practices that infringe upon their privacy (e.g., selling advertisements).

IMG-2090e-01.png

Advantaged
Activist

IMG-2084e-01.png

Globe
Trotter

IMG-2085e-01.png

Ceremonial
Admin

Understanding Harms

The use of private messaging platforms can expose users to a range of harms that impact trustworthiness.

ā€‹

In this section we cover six harms that were most frequently mentioned by the users we spoke to in Nigeria, Colombia and the United States. For each one, we explore the different ways that these impact the user experience as well as the platform design gaps that can amplify these harms.

Vulnerability to adverse

mental health impacts

The negative psychological impacts that arise from using private messaging platforms.

1.
Vulnerability to manipulation (misleading montent, misinformation) or exposure to offensive content
The use of private messaging platforms to knowingly or unknowingly spread content that can be perceived as being hateful, offensive and/ or misleading. 
4.
Vulnerability to hacking, scamming, blackmailing, extortion, fraud, and harassment
The direct misuse of private messaging platforms by adults as distinct from those affecting youth and children.
2.
Vulnerability to encryption and data breaches via modified and third party supporting platforms
The different ways that encrypted messaging platforms security, privacy and encryption features are bypassed by users through the use of modified and third party supporting encrypted messaging platform apps.
5.
Vulnerability to targeted harassment for youth and young adults
The use of private messaging platforms to exploit the vulnerability of youth and minors.
3.
Vulnerability to digital surveillance and monitoring
The potential use of private messaging platforms by governments and corporations to survey or monitor users. 
6.

Design Opportunities

Users helped us uncover a range of design opportunities that private messaging platforms should consider to improve trustworthiness.

01

Securing and/ or Modifying 

Account Information

Addresses: 

Easy access to personal identifying data

ONI_Icons_Outline_26 copy.png

03

Providing Accessible & Tailored Security & Privacy Controls

Addresses:

Generalized and hidden privacy & security controls for contacts & groups

ONI_Icons_Outline_28 copy.png

05

Providing User

Support Mechanisms &

Emergency Controls

Addresses:

Limited user support and lack of adequate reporting mechanisms

ONI_Icons_Outline_29 copy.png

06

Improving Administrative 

& Management Tools

Addresses:

Limited content management tools

ONI_Icons_Outline_25 copy.png

02

Improving Verification

& Permission Mechanisms

Addresses:

Limited verification and consent-focused features for contacts and groups

ONI_Icons_Outline_27 copy.png

04

Managing Access to Modified & Third-Party Supporting Platforms

Addresses:

Existence of modified (MOD) & third party supporting apps ecosystem

ONI_Icons_Outline_30 copy.png

07

Providing Data Use Transparency & Managing Data Access

Addresses:

Lack of transparency regarding access to user data

Contact

Who this report is for

Platform owners and providers

To negotiate competing product priorities and adjust product planning to address user concerns and diminishing perceptions of trust within private messaging platform experiences.

Researchers

To equip the trust and safety research field with actionable user data, and offer mixed methods methodologies focused on user experiences of private messaging platforms.

Policymakers

To better assess the risks that matter to residents and citizens related to security, democracy and information integrity, and prioritize the harms that occur on private messaging platforms to inform meaningful policy solutions.

UX
Designers

To augment their own user research and data analytics, and influence product priorities in line with user trustworthiness.

Advocacy,
Civil society

To buffer advocacy efforts with evidence of the harms a diverse set of users experience on private messaging platforms and examples of concrete changes that could improve trustworthiness.

Platform players, 
Governments

To assess the potential impact of user concerns on the trustworthiness of services they offer on private messaging platforms. User perceptions will shape their trust in services offered by governments and other players
bottom of page