Zoopany

Role

UX Researcher
‍

Timeline

March 2023, (1 year)

Research Methods

Competitive Analysis
User Interviews
Literature Reviews
Moderated Usability Testing
Moderated A/B Testing

Background

Social abuse issues have increasingly been brought up in the games industry workspaces, and they have been difficult to prevent, identify, and navigate.

Problem

Game development students need a more engaging way to understand how to meaningfully navigate these issues when they first enter the games industry, especially when the abuser’s status is higher than the abused.

Goal(s)

Zoopany is an applied game that guides students entering the games industry to:
1. Identify gaslighting
2. Understand the emotional effects of gaslighting on people
3. Provide professional resources for navigating gaslighting

Image for describing research and development process.

Summary of outcomes

Quantitative Study
We achieved our product and project goals and used t-tests to compare the applied game to an industry-standard sensitivity training module. Even with our limited data, these indicated early signs of the following:
1. The game's increase in knowledge is not different from the sensitivity training module
2. The game's increase in empathy is trending towards being different from the sensitivity training module
3. The game's increase in engagement is different from the sensitivity training module

Improving the usability with each build of the applied game gave us these results found from t-tests. While we could not form confident conclusions with our sample size, with more participants, we would be able to prove that the approach of utilizing gameplay for these problems is an effective solution.
‍
Qualitative Study
‍
We iteratively developed an applied game and used user interviews, stakeholder interviews, usability tests, and surveys to reveal that users could successfully:
1. Identify gaslighting
2. Understand its emotional impact
3. Gain access to professional resources to navigate gaslighting

Secondary Research

Comparative Analysis
‍To get a better understanding of how this problem has been addressed, I researched the following products from other companies to understand their strengths, weaknesses, and what could be improved:
‍
1. HR University's sensitivity training; Had detailed information but hard to view abused perspective

2. Vantage Point's VR sensitivity training; Facilitated discussion but is expensive to setup

3. Gamer Girl's Visual Novel Simulation; Easy to access but has shallower choices to understand people

4. Smalltalk's Abuse Awareness Simulation; Data driven but not interactive

Designing for perspectives

User Flow

We saw that each of our competitors had less opportunity for accessing interactive material for better understanding these topics. We converged with the idea to present different character perspectives to teach how to identify and emotionally connect with these topics.

I created a user flow for presenting this as a narrative experience.

Concept Development

Prototype
‍Our team arrived at two concepts, which were:‍
1. Moving your character, talking to other characters, and navigating character interactions
2. Presenting images, text, and textual choices to navigate through events

We combined concepts that would accomplish: ‍
A. Presenting a narrative by having the user control a singular character
B. Talk to other characters
C. Discover images as they move their character and explore their workspace

This prototype was a combination of physical and digital. The physical portion represented character navigation, while the digital presented the narrative and UI.

Primary Research

Stakeholder Interviews
‍With our resources at the time, we referred to our classmates and our own experiences in undergraduate courses for determining our target audience.

We gathered existing secondary research sources around gaslighting and compared them to our user's experiences of our game builds in usability tests. Our game did not output the same results as our secondary research sources, and we held meetings with our stakeholders to figure out the next best steps. Through multiple stakeholder interviews, we identified core stakeholder problems and product problems to close the performance gap.

Stakeholder problems:
‍A. Clarify who the experience is for
B. Create an experience that meets both the goals of increasing knowledge and empathy instead of solving the problem of helping people navigate these issues successfully
C. Maximize the quality of features

Product problems:
‍A. Our problem and target audience were not clear.
B. We need to assess and adjust our content to ensure meeting our goals.
C. Too many new initiatives exist to focus on working on the quality of features.

These defined our overall goals to have game development students experience:
‍1. Identify gaslighting in the games workplace
2. Generally understand how it makes people feel
3. Being provided professional resources they can use for navigating gaslighting

Evaluating our project's progress

User Interviews
‍After aligning our goals and starting to create new builds, we wanted to understand our product's progress towards meeting our goals. I created questions to gather qualitative data through user interviews. This helped us gauge how far we are from reaching our defined goals. With recruiting online and in-person on the UC Santa Cruz campus, we were able to work with 5 participants.
‍
Quotes:
‍• "I started associating feelings of hesitancy whenever I saw that character because I would only see my own player with a bit of stress whenever they were around."
• "He comes off as weirdly friendly and chill but then as soon as you get to work, he’s demanding all things that you’re supposed to be responsible for but he didn’t communicate with you well."
• "It becomes more apparent when they’re talking to someone that doesn’t make them anxious, so that’s why when I was playing, the first thing I wanted to do was talk to the other— the character that was nice to me, because it made my character, my player character also not seem like scared or anything..."

Maintaining focus on stakeholder problems

Distinguishing our goals
‍As we worked towards meeting our overall goals, I wanted to make sure we were on track to meeting our stakeholder's problems who oversaw our project. It helped to distinguish how our overall goals addressed our stakeholder's problems and product's problems. The following table distinguishes between our stakeholder problems, product problems, and how our overall goals relate to both.
‍

Stakeholder

Identify target audience
Create experience that increases knowledge and empathy

Product

Identify gaslighting
Generally understand how it makes people feel

Both

Maximize quality of features
Show improvement of knowledge and empathy

Product Features
‍We organized how to develop features that would make the most impact for meeting all these goals.
‍

Research

Literature Review
Usability Testing
‍
Literature Review
Stakeholder Interviews
‍
Competitive Analysis
‍
User Interviews
‍
Stakeholder Interviews
‍
Usability Testing
‍

Priority

Must Haves / Highest

‍
‍

Nice to Haves / Lowest

Feature

Chapter 1 Fully Implemented
Create Character Placement System
Write all dialogue
Animate all Chapter 1 NPC models
Create navigable office
‍
Import background music for all interactions
Cutscene Animations
‍
Reflection Choices

Benefit

Increase quality of experience
Give users choice to understand content
Teach gaslighting
Demonstrate emotional effects
Enable users to learn about people
Emotionally connect to moments with characters
Build emotional connection
Build knowledge about events

New Additions

New Features
‍1. Character Placement System: Users find new or same characters in different places with different dialogue depending on progress of narrative
2. Reflection Questions: Users answers questions to recall events and what happened to characters

Final Designs
‍We settled with prioritizing that the 3D models be shown first, then being able to see the 2D sprites, and seeing the name and dialogue right after.

Latest Build
‍The following pictures show our final product.

Before

After

Efficacy Testing

A/B Testing
‍Through in-person and online recruiting, we conducted in-person and online studies with 23 participants, formed closed-ended surveys, and conducted pre-post surveys with the following structure:

1. Pre-survey examining the knowledge of gaslighting and empathy towards victims
2. Module of either playing the applied game or going through the industry standard gaslighting sensitivity training module
3. Post-survey examining changes in knowledge, empathy, and engagement with topics and each module
4. Post-survey questions to gather in-depth qualitative data and last comments about each respective module

Managing participant expectations:
‍A. Communicated that the session would be recorded and that they were going to be interviewed alone
B. Reassured participants of their role in the studies but did not include specific details

Findings

Quantitative Study
‍To analyze the pre-surveys and post-surveys from each of the studies, we ran the following t-tests:

A. Paired-Samples T-Test: Data around change in knowledge and engagement were normally distributed
B. Wilcoxon Signed-Rank Test: Data around change in empathy was not normally distributed

With 23 participants, we found that these trends still aligned with our defined goals. As seen in the tables, we can still use more data to make more confident conclusions. We found early signs that:

1. The game's increase in knowledge is not different from the sensitivity training module
2. The game's increase in empathy is trending towards being different from the sensitivity training module
3. The game's increase in engagement is different from the sensitivity training module

Qualitative Study
‍Even though we already had a significant difference in engagement for both modules, many participants from the gaslighting module seemed to understand the engagement question as a question focusing on the topic of gaslighting more than the sensitivity training module.

Conclusion

After 1 year, we created an applied game that accomplished helping users to:
1. Identify gaslighting in the games workplace
2. Generally understand how it makes people feel
3. Gain access to professional resources they have to navigate gaslighting
I designed the research studies with my team to evaluate the progress and efficacy of our product. While we achieved our overall goals and answered our product and stakeholder problems, we had limited data to form final conclusions between our game's performance against the industry standard gaslighting sensitivity training module. Our current data indicated early signs of the following:

1. The game's increase in knowledge is not different from the sensitivity training module
2. The game's increase in empathy is trending towards being different from the sensitivity training module
3. The game's increase in engagement is different from the sensitivity training module

Improving the usability with each build gave us the results from our t-tests, and with more participants, we would be able to prove that the approach of utilizing gameplay for this problem is an effective solution.

What would I have changed?

1. Empathize with user earlier
I would also spend more time trying find other users that we could have addressed and perform my primary research earlier to find the most important problems and solutions sooner.
2. Perform research earlierI would make more effort to concisely define who our users are and the problems to address. After ideating right away, I see that it was difficult determining the purpose of our project. If we had started this earlier, I think this would have clarified our purpose, simplified our design, sped up our development, and had more time to gather participants. Starting this later affected our efficacy testing and this could be improved to have more data and show more confident results.