Airpals case study
A usability study to enhance the order form experience for the Airpals website.
My role
Usability expert
Team
5 usability experts including me
Timeline
6 weeks
Focus
Designing and conducting the usability test to improve the order form experience
Tools used
Figma, Google Forms, Google Sheets, Google Docs, Zoom
Project Overview
Airpals is a B2B courier service that offers same-day and scheduled delivery in NYC and parts of Jersey City. Their goal is to simplify courier delivery and make work life easier for users by adding modern updates and empathy to the industry.
To improve the pick-up and delivery scheduling order form experience on their website, our team of 5 usability experts conducted 11 moderated user tests. We used the findings to provide recommendations to enhance user experience and improve the order form's flow.
My contribution
-
Analyzed Google analytics data to identify pain points on the Airpals website.
-
Worked closely with the team to design and script a user test that aligned with research objectives.
-
Conducted participant recruitment, moderated 2 remote user testing sessions and took notes for other 2 sessions.
-
Collaborated with the team to analyze findings and develop recommendations.
-
Created mockup and prototype, and drafted findings and recommendations for the "Review" page of the order form and the error messages displayed on the form.
-
Actively took part in drafting the final presentation and report.
-
Presented the findings and recommendations to the client.
Our Process
Our process started with a collaborative kick-off session to clarify the client’s goals and define the project scope. Based on these insights, we designed and conducted a usability test, then shared our findings and recommendations through a detailed presentation and report.
#1
Project kickoff meeting
During the kickoff meeting, we acquainted ourselves with the Airpals team and took the time to understand their goals and objectives for the project. We gathered crucial insights into their target users and business strategy.
The Airpals team requested that we prioritize usability testing on specific key areas of their website and review their Google Analytics data to identify any pain points that should be included in our study.
Meeting outcome
-
Received information regarding existing user personas created by the Airpals team which would feed into our target participant definition.
-
Determined 2 key areas of focus for our study: the pick-up and delivery scheduling form and the order confirmation email.
#2
Preparation
Pre-test questions, task, scenario, post-task questions and post-test questions
Recruited participants' demographics
Defining target participants
Using the information obtained during the kick-off meeting and the existing personas developed by the Airpals team, we established our target participant profile to be - a person who is employed and 18 years or older.
Developing a recruitment strategy
Given that our target participants needed to be employed and over 18 years old, we determined that the most effective recruitment strategy would be to email a screener to the Pratt Students distribution list and post it on our social media accounts.
Defining the research objective
Our research objective was to obtain both qualitative insights into user perceptions and behaviors, as well as quantitative data, with the intention of generating robust evidence to support our project goal.
Designing the test
To conduct our usability tests, we created a script that included a scenario, 2 tasks, post-task questions and questionnaire, and post-test questions and questionnaire.
The post-task and post-test questions enabled us to collect qualitative data, while the post-task and post-test questionnaire facilitated the gathering of quantitative insights, as specified by the client and defined in our research objective.
Conducting pilot sessions
To refine the script and questions, we conducted 5 pilot tests. During these tests, we closely observed participant behavior and identified any issues or areas for improvement in the script. For instance, we discovered that some of the task instructions were vague or repetitive and required revisions to enhance clarity.
Moreover, we recognized that the second task outlined in the script did not provide significant value to the research and made the test longer than the desired duration. As a result, we chose to eliminate the second task for the usability sessions conducted with the recruited participants.
Recruiting participants
The screener was carefully crafted to ensure that only participants who matched the target demographic would qualify for the study. Out of 18 responses received, we selected 11 participants who best matched the target participant profile to participate in the usability test. Although 11 participants were recruited, 8 were retained as the best fit for the user group, and the results reflect the responses of those retained.
#3
Data collection
Screenshot from a user testing session conducted on Zoom
Scheduling sessions
We reached out to the 11 selected participants with additional information and provided them with a Doodle link to indicate their availability. Using the participants' preferred time slots, we scheduled Zoom meetings for the remote usability testing sessions. To ensure the smooth running of each 30-minute session, we assigned a moderator and a notetaker to oversee the testing.
Conducting sessions + Collecting data
We conducted 11 remote moderated user testing sessions over zoom. This allowed us to observe and record participant behavior as they shared their screen and navigated through the order form to complete the task. As previously stated, we used a combination of verbal questions and a Google Form questionnaire to collect both qualitative and quantitative data.
#4
Data analysis
Review + Group data based on usability issues
After completing the usability testing sessions, we reviewed the data collected through the notetaker Google Form. To analyze the feedback, we utilized a digital grouping spreadsheet and applied a thematic analysis approach to classify the data based on common user frustrations, behaviors, or pain points observed during the testing.
Identify key usability issues
We assessed the issues identified in the previous step based on their severity and frequency, considering how badly the issue affected the usability of the form and how often it occurred during the test sessions. We then prioritized them accordingly to identify key issues and determine solutions for them.
Overall Findings
Key recommendations
#1
Streamline the flow for inputting parcel details
#2
Clarify pick-up and drop-off instructions
Testimonial by Joshe Ordonez, Airpals founder
I want to express my sincere gratitude for all the hard work and dedication you put into this project. Your contributions will be of tremendous help, and we're excited to implement your feedback.
Key Learnings
-
Client Management - Initially, our client had agreed to help us recruit 30% of the participants and had assumed that they could attend the sessions for the participants they had recruited and conduct a mock test session with our team. We discovered this misunderstanding during our mid-project check-in meeting, which was likely due to inadequate communication of the project's scope and limitations. To address the issue, we sent an email that clarified the project's scope and limitations, which helped to clear up the confusion. This experience taught me the importance of clarifying project scope and limitations at the outset of a project.
Next Steps
-
The option to place a one-time order as a guest, rather than requiring users to sign in.
-
Expanding the service to include regular users instead of only catering to businesses.