Overview
In an eight-month project, my team and I spearheaded a comprehensive overhaul of Pandemic Professors, a non-profit offering free tutoring to low-income students. We introduced a functional data center, updated forms, and established new session guidelines. Additionally, we crafted a detailed manual to lead the implementation of this new system.
Our primary focus was to ensure efficient, high-quality support for low-income students, both educationally and emotionally by delivering a high quality functional solution.
This comprehensive solution has been implemented at Pandemic Professors.
.
My Team: (From Left to Right) Brady Baldwin, Gabbi Laborwit, Rachel Jones (me), Jenny Xin, Ije Okafor
.
My Role: Lead Designer Researcher & Strategist
CONTRIBUTION HIGHLIGHTS
Technical - Led data center creation, utilizing Google Sheets fully from API integration to Z-tests. Gaining a greater understanding of how to collaborate with developers from taking a walk in their shoes.
Design - Led the visual, and functional design of the data center from the data flow to the functional interactive interfaces.
Research - Headed student and tutor onboarding forms development. Including conducing user tests and for English and Spanish versions. Teaching me how to strategize long-term projects
Strategy - Utilized business background to conduct extensive competitive analysis and make strategic suggestions to the board of directors.
Context
Education Inequity
Educational inequality has long been a major problem in America, and the pandemic did not help.
Pandemic Professors is one of many non-profits tackling this issue founded in the spring of 2020. They tackle this issue by providing college-educated tutors to low income students twice a week.
Challenge
Pairing Bottleneck
Pandemic Professors came to our team because their pairing system was acting as a bottleneck and they wanted to grow. Our research confirmed this was an issue as their 3 week pairing time was the longest of all 500 organizations researched.
.
.
As a growing startup they need a solution to their pairing bottleneck issue that was effective cheap and fast. With this in mind, we began answering the question:
How might we speed up the tutor-student pairing process given the constraints of the organization?
Methods
To address this question, I collaborated with my team to employ a variety of research methods that uncovered the reasons behind the sluggish pairing process and the most effective ways to address it, and evaluate the success of our implemented solution.
Methods Used By Process
User Testing - User, A/B, Diary Study, Speed Dating, Mass Surveys, Think-Aloud, Contextual Interviews, Longitudinal Studies
Analyzing - Affinity Diagramming, Regression, Competitive Analysis, Sentiment Analysis
Modeling - Journey Mapping, Current State Modeling, Personas, Empathy Mapping, Stakeholder Mapping
Brainstorming - Storyboarding, Crazy 8's
Prototyping - Wizard of Oz, Conceptual Prototyping, Functional Prototyping, Wireframes
.
Key Pains
Our research revealed three key pains that slowed down the pairing process.
Incomplete & Confusing Data
Our initial interviews revealed the large pairing time difference was primarily caused incomplete and confusing data that forced pairers to reach out for clarification, extending the pairing process.
"We reach out every time a student says "yes" in the accommodation section since we never know what they mean...that adds at least a week.”
A pairer explaining the ways in which their process is extended
Inhuman Computation Expectations on Pairers
Moreover, in walking through the process we found that pairers were expected to analyze and compare a lot of unorganized data independently. Making the process, inconsistent and incredibly mentally taxing. Leaving them with little time to do anything else.
"(When pairing) I compare all of the data on these 3 sheets, and keep in mind an idea of what is available and needed in both pools. I have a system that I use personally, but it is still hard to keep track of everything.”
A pairer explaining on the pairing process process
Lack of Empathetic Connection Between Tutors & Students
Finally, further research revealed, that a lack of empathy and emotional connection between tutors and students caused pairs to fail quickly.
With every family we spoke to mentioning a lack of willingness to understand as the reason for pair failure, and every email requesting pair termination echoing that sentiment.
"He was late to one session and (his tutor) let me know he was requesting a different student. My son really needs help. While we wait for a new tutor, he is falling behind even more. I'm worried about him.”
A parent of a student who experienced pair failure
By putting students back in the pairing process frequently, it not only disrupts that students education, but it makes harder to add new students into the system.
Solution Overview
We addressed the pairing process challenge by:
Reducing pairing duration through effective data collection and automation.
Decreasing pair volume by enhancing pair longevity through predictors for tutor behavior, new relational support systems, and simplified pair health monitoring and intervention.
Functional Tools Delivered
- Implemented a working data center to streamline processes.
- Introduced a comprehensive set of forms for efficient data input and updates.
- Developed a 93-page manual* for understanding and operating each system.
* If you want to grasp the amount of thought that went into every tiny aspect of this system, I highly recommend reading this.
.
Flow of Information in thier new ecosystem highlighting the tools we delivered.
.
Comprehensive Impact
These improvements collectively addressed the pair time bottleneck, resulting in a more efficient, empathetic, and effective system. Notably, the entire suite of tools is designed to operate seamlessly through the Google Suite, offering flexibility and scalability as the organization grows
Features
Clear & Effective Data Collection Removes the Need for Follow-Ups and Enables Automation Speeding up the Process
Consistently Phrased Questions across forms
Previously, student and tutor forms were mismatched due to inconsistent phrasing. This issue was resolved by introducing consistent wording across analogous questions on both onboarding forms.
.
.
Standardized Question Inputs
The absence of standardized inputs caused confusion among those completing forms for students, leading to submissions that Pandemic Professors couldn't effectively use. This confusion, particularly in accommodation cases, necessitated frequent follow-ups with parents in the old system.
Standardized inputs were implemented across forms, incorporating common accommodation checkboxes and automatic language translation.
"This makes sense, I actually don't think I would need to reach out for more context.”
A tutor, reading a pairer's response in the accommodation section with standardized question input revisions.
.
.
DESIGNATED Spaces for Elaboration
In initial forms, people were putting information that they wanted Pandemic Professors to know in random places making understanding difficult and automating impossible.
By adding directed questions, and spaces for elaboration we were able to present a clearer picture of students and presentation of their information in the final data center. Moreover, in tests, these questions satisfied the needs of most parents.
.
Easy Pairing with Automation Reduces Pairing Stress & Speeds Up the Process Significantly
Our initial thought when streamlining this process was to streamline automate it entirely. However, in interviewing pairing staff we found that there was a need to analyze some of these factors with a more personal touch. As such, we created a two step system.
In user tests this new system yielded a 1:34 second process for complex pairings in initial tests, which is a substantial improvement (2X as fast) as pairers took in the existing system. Additionally, every member of the pairing staff at pandemic professors said they would be excited to use this new system.
Across the board, this two-process system for input presentation serves to make data less overwhelming.
,
.
The TWO Step pAIRING Process
(Automated) Step 1 - Algorithm Ranks Tutors
Because humans cannot process all the data we collected for all of the tutors at once, we created a prioritization system that utilizes collected data and weights factor compatibility importance by ratios determined via interviews with pairing staff.
This is made possible by the Clear & Effective Data Collected Earlier
(Manual) Step 2 - Pairers Compare Nuanced Context
For contextual information that can not be automized, but should be accounted for, we worked to improve information placement to make it more usable. That is all student information is above the analogous tutor information.
.
Pre-Selection for Empathetic Tutors Improves Chances of Pair Success Reducing Re-Pair Time
Pandemic Professors helps students who have unstable backgrounds, which bleed into their learning environment. The Uncertainty Tolerance Assessment measures a tutor's ability to adapt or empathize with their student.
Tests with over 100 participants proved the assessment was statistically significant in measuring its intended target. This is important for failure to predict this behavior was the leading cause for most repairs.
This statistic is used as a part of the automated pairing process.
The Uncertainty Tolerance Assessment (UT)
Participants take a series of reflection and forced-choice questions, questions that force them into making a true choice by presenting a number of socially acceptable answers, about how they would respond in two different scenarios.
.
.
These scenarios were taken from real situations we observed in our research and were designed to measure tolerance of attendance, proactiveness in reaching out, and empathy for student life.
The assessment yields two scores:
Uncertainty Tolerance (UT) - This is the the primary measure based on the model we created. It provides a score from 0-12
Qualitative Response - This is a secondary measure intended to provide context to uncertainty tolerance. It is the result of a keyword analysis of the reasoning given by each tutor. It can be infinitely positive or negative.
.
.
In-Session Empathy Building Increases Pair Resilience & Tutoring Effectiveness Reducing Re-Pair Time
Therapy Based Feedback Forms
Derived from clinically tested child questionnaires used to evaluate therapists, these forms underwent a three-month trial on a quarter of Pandemic Professors' base. With two sections— Listening Session and Session Evaluation —the forms aim to improve pair relations and session effectiveness.
Listening Session - The listening session requires pairs to employ empathetic listening skills of reflection to better understand their students in the session.
In our study, all tutors learned something new or surprising about their students and felt a stronger bond with them because of it.
.
.
Session Evaluation - Designed to enhance understanding of session processes, our study revealed a positive shift in tutors' perspectives, leading to improved lesson planning.
"...She just thinks it’s the most boring thing ever, which really surprised me...I didn’t realize she just didn’t get anything out of it.”
Tutor of 6th Grader realizing she needed to change her approach
.
.
Continuous Pair Fail Prevention Supports Longevity Reducing Re-Pair Time & Strengthening Pandemic Professors Value to Tutors
EASY PAIR MONITORING
Before our intervention asks for help from tutors were often lost in emails and pair endings always came as a shock. Making Pandemic Professors simply a paring system instead of a supportive home organization.
With our new system, recent feedback form data (Past 6 weeks) along with an option to request attention are used to create a support action prioritization system for Pandemic Professors workers, so that they know who to reach out to and when to do it.
This data set is interactive so that workers can deprioritize pairs that someone has already attended too. Enabling independent action, further reducing the effort necessary to provide support.
,
Impact
.
A more efficient system
2X faster pairing of complex pairs in our new system as compared to simple pairs in the old system, and at 1:34 pairing is significantly faster than the 3 weeks that use to be expected of complex pairs.
A more empathetic system
With the introduction of a new, therapy-based feedback form that resulted in all of the participants studied learning something new about their student and their teaching, our system supports empathy.
Additionally, with the introduction of the Uncertainty Tolerance Assessment and associated metrics, which were proven statistically significant in a 100+ person study, we can now attempt to place students who need more empathy with a more empathetic tutor with our system.
A more effective system
By streamlining information from all parts of the ecosystem into one place, with prioritized information, Pandemic Professors should be able to better distribute their time to focus on the needs of struggling pairs, rather than incoming ones. Enabling this shift in priority will enable more effective learning from existing pairs and a more effective organizational structure as a whole.
Reflection
Looking back, the efforts described above depict the full utilization of a truly incredible opportunity to use design to make a positive impact on the world.
By not only listening to surface users but digging deep into the companies core, we were able to develop and deploy a well-tested solution that will enable more low-income students to receive free tutoring and increase the quality of tutoring they receive.
Delivering an effective implementable solution was a technically challenging and emotionally weighty task which pushed my group to be better designers, researchers, and teammates.