Moderated Testing
As a member of the Rapid Research team, the majority of studies I worked on were moderated qualitative studies. These studies were requested from multiple research portfolio leads comprising of 1-2 research objectives for each study.
Tools
-
Fable - I utilized Fable to conduct remote, moderated usability studies with screen reader and screen magnification users.
-
UserTesting - I utilized UserTesting for remote, moderated usability studies and interviews with users. Demographics of users were determined study by study. UserTesting was the primary platformed utilized for non-business customers.
-
UserZoom - I utilized UserZoom for remote, moderated usability studies and interviews with users. Demographics of users were determined study by study. UserZoom was the primary platform utilized for business customers.
-
Figma - Figma is the design platform the Verizon designers and researchers were the ones to test those prototypes.
-
Google Office Suite - I utilized Google Office Suite to write test plans and to create findings presentations.
-
Miro - I utilized Miro for notetaking during sessions and keeping track of insights and themes.
Timeline
As a member of the rapid research team, all studies submitted to our team must be completed within a week's time.. Since there is a limited timeframe each study is limited to 8 study participants, 30 minute sessions each, and 2 research objectives. Depending on the complexity of those objectives, I would either work solo or with a partner on a project. Below is the weekly timeline detailing the average week.
Day 1
-
Studies are assigned to rapid researchers.
-
Researchers send out invites for Kick-off meetings with stakeholders
Day 2
-
Researchers meet with stakeholders during the Kickoff meeting to walk through prototypes and other materials, and to go over objectives
-
After the kickoff, researchers write the test plan.
-
Researchers submit test plan to manager by end of the day.
Day 3
-
After manager reviews test plan, researchers send the test plan to stakeholders for further approval
Day 4
-
Researcher finalize any changes made to test plan in the morning
-
Testing begins with 1st participant at 1pm EST
Day 5
-
Test sessions are conducted all day
Day 6
-
Researchers synthesize findings and then create finding presentation
Day 7
-
Researchers hold a Read-out meeting with stakeholders to present findings.
Challenges
-
Technical difficulties: One of the primary challenges in remote moderated testing is technical difficulties, which may arise due to issues with the participant's equipment or the video conferencing software being used. To mitigate these challenges, I always attempt to troubleshoot the issues with the participant. However, given the time constraints of rapid research sessions, if the issue persists beyond 15 minutes, I would inform the participant that the session cannot proceed. This occurred only twice in my experience, and in those cases, I apologized to the participant and referred them to a research operations specialist to reschedule the session if possible.
-
Distractions: Sometimes Participants would get distracted by their environment or other obligations during remote testing sessions, which would impact their level of engagement and focus during the testing process. Usually when this happened, I would politely ask the participant if they need a quick break and if they did need a break I would ask if it would be okay to go over our scheduled time together. Admittedly, participants were not the only one with distractions at times and the work from home environment has it fair shar of distractions. Random noise interruptions from outside or my dog barking occasionally occurred during sessions with users. When this would happen I would always apologize to users and I would place myself on mute whenever the participant was talking.
-
Difficulty building rapport: Remote testing made it a little more difficult to build rapport since I was not not physically present with the participant; in addition, often times participants preferred not to be on camera either. To combat this I usually would ask some questions such as asking how their day is going, what's the weather like where they are, or where are they from. This always helped me build that rapport with participants.
Key Takeaways
During my time at Verizon so far, I have been fortunate to work on over 50 moderated studies, each providing invaluable feedback that have helped informed the design of the product or service it was testing. Each study had very specific and unique findings, so rather than giving key takeaways for projects, bellow you can find some key lessons or skills that I have developed after working on all these projects.
-
Adaptability: Due to the amount and wide range of project types I've learned to be extremely flexible and able to adjust my approach to accommodate for the limitations and challenges of each individual study. The nature of rapid research is agile, being able to move from a new project weekly has required me to learn how to adjust quickly while also looking at all projects holistically to provide stakeholders with insights across multiple studies..
-
Effective communication: Clear and concise communication has crucial in remote moderated studies. Having to communicate with participants, stakeholders, and team members all virtually, sometimes without seeing each others faces, is something I've had to learn to develop further since I often worked with a broader range of types of people than I ever have before.
-
Observation skills: Remote moderated studies have required me to adopt different methods for data collection, such as observing participants' behavior through their screens and relying on verbal cues and feedback. As a result, I have honed my observation skills to ensure the accurate capture of participants' experiences. During my tenure at Verizon, I have also noticed that remote researchers exhibit heightened sensitivity to tone and verbal cues compared to their hybrid counterparts. In several instances, I have noted nuances that in-person researchers have missed, while taking notes remotely for a live, in-person session. This observation has sparked engaging discussions among remote researchers who have experienced this as well during post-study debriefs.