Unmoderated Testing
As a member of the Rapid Research team, the majority of studies I worked on were moderated qualitative studies: however, from time to time I was assigned to create and conduct some unmoderated studies. These studies were requested from multiple research portfolio leads comprising of 1-2 research objectives for each study.
Tools
UserTesting - I utilized UserTesting for remote, unmoderated usability studies, card sorting, and A/B tests with users. Demographics of users were determined study by study. UserTesting was the primary platformed utilized for non-business customers.
UserZoom - I utilized UserZoom for remote, unmoderated surveys with users. Demographics of users were determined study by study. UserZoom was the primary platform utilized for business customers or specifically for survey for non-business customers.
Figma - Figma is the design platform the Verizon designers and researchers were the ones to test those prototypes.
Google Office Suite - I utilized Google Office Suite to write test plans and to create findings presentations.
Miro - I utilized Miro for notetaking during sessions and keeping track of insights and themes.
Timeline
As a member of the rapid research team, all studies submitted to our team must be completed within a week's time.. Since there is a limited timeframe each study is limited to 8 study participants, 30 minute sessions each, and 2 research objectives. Depending on the complexity of those objectives, I would either work solo or with a partner on a project. Below is the weekly timeline detailing the average week.
Day 1
-
Studies are assigned to rapid researchers.
-
Researchers send out invites for Kick-off meetings with stakeholders
Day 2
-
Researchers meet with stakeholders during the Kickoff meeting to walk through prototypes and other materials, and to go over objectives
-
After the kickoff, researchers write the test plan.
-
Researchers submit test plan to manager by end of the day.
Day 3
-
After manager reviews test plan, researchers send the test plan to stakeholders for further approval
Day 4
-
Researcher finalize any changes made to test plan in the morning
-
Researcher sends out a pilot test.
-
Unmoderated testing begins at 1pm EST
Day 5
-
Test sessions are conducted all day
Day 6
-
Researchers synthesize findings and then create finding presentation
Day 7
-
Researchers hold a Read-out meeting with stakeholders to present findings.
Challenges
-
Technical Skill: Although I was familiar with UserTesting and UserZoom for remote moderated studies, I encountered some difficulties with these platforms when using them for unmoderated studies. My initial unfamiliarity with these platforms for unmoderated studies prompted me to spend a considerable amount of time figuring out how they worked and how to set up a study. Through trial and error and hands-on exploration, I developed a comprehensive understanding of these platforms, enabling me to use them just as proficiently for unmoderated studies as for moderated studies.
-
Communication skills: Effective communication with participants is crucial in remote unmoderated studies as they complete tasks independently without additional guidance from the researcher. Initially, I faced challenges in formulating direct and concise questions that participants would interpret as intended. To overcome this, I conducted pilot tests with three participants in my first few unmoderated studies to ensure the clarity of the questions. With feedback from my manager and the pilot tests, I honed my skills and developed the ability to write effective question formats for unmoderated studies
Key Takeaways
During my time at Verizon so far, I have been fortunate to work on 7 unmoderated studies, each providing invaluable feedback that have helped informed the design of the product or service it was testing. Each study had very specific and unique findings, so rather than giving key takeaways for projects, bellow you can find some key lessons or skills that I have developed after working on these projects.
-
Data analysis: Remote unmoderated studies often involved larger participant groups, leading to a significant amount of data that demanded exceptional data analysis and interpretation skills. To ensure I was doing the best I could, I meticulously monitored participant behavior, recorded and synthesized data, and paid careful attention to detail to ensure accurate reporting of findings. The sheer volume of data and feedback from these studies provided an opportunity to refine my data analysis and interpretation skills. This evidently improved my analytical skills immensely, especially with more quantitative data
-
Improved communication: As highlighted earlier, formulating precise and concise questions was one of the significant challenges I faced during unmoderated studies. However, mastering this skill not only helped me with unmoderated studies but also enabled me to create more precise questions for moderated studies. My questions used to be quite verbose before I engaged in unmoderated studies, but I have observed a remarkable improvement in their clarity and conciseness, which has led to a better overall research experience.