Website Builder Subscriptions

Research Methods

Usability Testing
Remote Unmoderated A/B Testing
Surveys
‍

Timeframe

December 2023, (2 months)
‍

Role

UX Researcher
‍

Problem

SaaS vendors need to understand how their pricing practices affect their user's subscription experiences.
‍

Background

SaaS vendors that utilize subscription models have to decide how they will display their pricing and consciously make a decision to continue or remove using dark patterns.
‍

Goal(s)

Decrease instances of canceling subscriptions during free trials after interacting with pricing design; Understand usability issues and effects on user from different pricing designs.
‍

Background

SaaS vendors that utilize subscription models have to decide how they will display their pricing and consciously make a decision to continue or remove using dark patterns.
‍

Goal(s)

Decrease instances of canceling subscriptions during free trials after interacting with pricing design; Understand usability issues and effects on user from different pricing designs.
‍

How did I tackle this problem?

This was my process:

Secondary Research

To understand the space of subscriptions, pricing designs, and SaaS, I gathered research articles to identify patterns, problems to tackle, and goals.

Research Article
A 2021 research paper titled Price Salience and Product Choice reported that users who weren’t shown ticket fees upfront spent about 21% more money and were 14.1% more likely to complete a purchase.
‍Price Salience and Product Choice article

Online Article
Design articles from 2022 and 2023 have spoken against SaaS companies utilizing dark pattern practices that hide information regarding their subscription pricing, finding that these negatively affect revenue in the long run.
‍Userpilot article
UXCollective article

Participant Criteria

Identified participant criteria via secondary research + connections to various online UX channels.

Recruitment

Outreached and chose participants that met criteria using online sign-up and screening surveys, focusing on participants with experience with SaaS on either desktop or mobile.

For each study, I aimed for the following participants:

Qualitative study - 5 to 10 participants
Quantitative study - at least 40 participants

Creating research studies

After identifying participants, I finalized my approach to be mixed-methods.

I decided I wanted to find "why" my proposed design met the research goals through the qualitative study and show "how" it does this through the quantitative study.

Quantitative Study

My study was structured as this:

Between-Subjects design, Remote Unmoderated A/B Tests + After Scenario Questions using Maze and Qualtrics.
‍
1. Remote Unmoderated A/B Tests
(control and treatment groups were posted in different online channels)
2. After scenario questions

This was to accomplish observing what participants chose to do between each design, collect data without having to fully moderate the process, and also collect information on general usability issues like the Qualitative Study.

Qualitative Study

My study was structured as this:

Within-Subjects design, Usability Tests + After Scenario Questions using Zoom.
(alternating order of designs with each new participant)
‍
1. Original design
2. After scenario questions
3. New design
4. After scenario questions

This was to accomplish revealing mental models of each design, while also getting the information of what users think and overall say about the design and usability issues.

Running studies

Ran both studies for 7 weeks and collected a total of:

Qualitative Study - 1 participant
Quantitative Study - 7 participants

Challenges

While running both studies, I encountered the following challenges:

1. Time lag in recruiting participants online and lack of responses
2. Participants using varying platforms (i.e. desktop, mobile), making it difficult to gather enough relevant data for analysis and synthesis
3. Lower budget to compensate participants and accomodating more participants

Conclusion

At the end of my timeframe for 2 months, I had insufficient amount of data in both studies to form meaningful conclusions on the design, extracting new insights, and determining if would have decreased the number of un-subscriptions or reveal additional effects on users.

As a researcher, I have gotten to grow in my understanding of research from start to end, run my project, gather data, and find limits to running a project in this context.

What would I have done?

If I were to have continued this for longer, the following points describe my process:

1. Revised strategy for recruiting participants
2. Ran statistical testing (Chi-square McNemar) to demonstrate difference between origial and new pricing design
3. Analyzed and synthesized data to obtain insights towards making product decisions, utilizing factors around business, and used information to adjust research study

What would I have changed?

If I were to have continued my research, the following points describe my process:

1. Pivoted from online recruitment to participant recruitment services sooner
2. Refine definition of target participants for identifying who is unsubscribing, how, and why (B2B, B2C, etc.)
3. Test for designs that use various kinds of marketing language

Website Builder Subscriptions

Contact