Overview
Pandemic Professors is a non-profit that provides free tutoring to low-income students. Over an eight-month period, my team and I worked to revise their ecosystem with a functional data center, extended forms, and manual that enabled them to speed up and improve the quality of their pairing process. Enabling them to better serve their mission.
My Role
Lead Designer Researcher Technologist & Strategist
(Highlights at bottom of page
My Team
(From Left to Right)
Brady Baldwin, Gabbi Laborwit, Rachel Jones (me), Jenny Xin, Ije Okafor
Context
Educational inequality has long been a major problem in America, and the pandemic did not help. Pandemic Professors is a non-profit that provides free tutoring from college-educated people to low-income students.
Challenge
Pandemic Professors came to us because their pairing system was significantly slowing down their ability to take in students and was putting them behind their peers (at 3 weeks they had the longest pairing time of all 500 organizations I researched).
They were a growing startup and needed a solution that was effective cheap and fast. With this in mind, we began working to answer the question:
How might we speed up pandemic professors' pairing process, given the constraints of the organization?
.
,
Methods
To answer that question and ultimately create the Data Center, the suite of tools, and the user manual for Pandemic Professors. I worked with my team through many different user research methods. Conducting this research helped us understand why the pairing process was slow, what was the most effective ways to fix it, and if our solutions were working.
The methods used are listed below:
- Affinity Diagraming
- Contextual Interviewing
- Storyboarding
- Speed Dating
- Rapid Prototyping
- Crazy 8's
- User Testing
- Think-A-Loud Interviews
- Stakeholder Mapping
- A/B testing
- Mass Surveys
- Longitudinal Studies
- Interactive Prototyping
- Critique
- Expert Interviews
- Competitive Analysis
- Conceptual Prototyping
- Journey Mapping
.
4 Key Pains & Remedies that Define our Solution
Through this process up with four key findings built on insights and evidence that shaped our Final Project.
(1/4)
PAIN: The Pairing Process is at the center of the organization, and is affected by all aspects of the organization.
REMEDY: Create a comprehensive ecosystem solution that attacks related pains, and ensures full-functionality.
.
While we were initially brought on to solve the slowness of the pairing process, it quickly became clear that the pairing process delays causes were not contained to that one step in the process, as such, we developed a comprehensive solution that tackled all parts of the organization's functions.
Since we intended to deliver a functional solution to our client we had to develop a solution that addressed the key pain points, discussed below, but also other key functions of the company
Our solution needed to:
- Track all new tutor applicants
- Automatically generate pair recommendations between tutors and students
- Collect all feedback form data on a pair
- Alert Pandemic Professors when a pair is struggling based on feedback from data
- Maintain a repository of all current and past pairs, including historical information on why a tutor or student’s past pairing was disbanded.
This comprehensiveness allowed our team to hand it off to the company with confidence that it would be usable.
.
Flow of Inputs into Data Center
(2/4)
PAIN: Pairers are presented with large amounts of unorganized data that is often incomplete when making a pair. This makes pairing mentally taxing, and means follow-ups are often required.
REMEDY: Give pairers better resources.
Alter tutor and student onboarding forms to include consistent phrasing, standardized input types, and appropriate spaces for elaboration to ensure that received information is complete and understandable.
Then present these altered inputs in the most usable way possible.
.
ensuring inputs are complete and understandable (3 types of input alterations)
Consistent Phrasing of Questions
In the past, student and tutor forms were difficult to match up because the information literally didn't match up. This has been solved by introducing consistent phrasing across analogous questions on onboarding forms for students and tutors.
.
.
Standardized Question Inputs
Without standardized inputs, those filling out forms for students were confused about what pandemic professors wanted. This confusion led to Pandemic professors receiving inputs that they could not use. Especially in the case of accommodations, the confusion here forced pairers to follow up with parents nearly every time in the old system.
The concept of standardized inputs was implemented in various ways across the forms, from the inclusion of common accommodation checkboxes here to whole form automatic language translation.
"This makes sense, I actually don't think I would need to reach out for more context.”
A tutor reading a pairer's response in the accommodation section with standardized question input revisions.
.
.
Spaces for Elaboration
In initial forms, people were putting information that they wanted Pandemic Professors to know in random places making understanding difficult and automating impossible.
By adding directed questions, and spaces for elaboration we were able to present a clearer picture of students and presentation of their information in the final data center. Moreover, in tests, these questions satisfied the needs of most parents.
.
,
MAKING INPUTS Usable
To ensure maximum visibility of these new and improved inputs, a two-process make a pair system was created.
In pairing, this new system yielded a 1:34 second process for complex pairings in initial tests, which is a substantial improvement (2X as fast) as pairers took in the exsisting system. Additionally, every member of the pairing staff at pandemic professors said they would be excited to use this new system.
Across the board, this two-process system for input presentation serves to make data less overwhelming.
,
.
Automated Process: Automate and Rank Matching of Standardized Inputs
Information that is not standardized and consistent was automated in the new system!
For pairing, I created an automated compatibility rank system based on research on what information makes a good pair. This enables pairers to sort through information easily, and know that they are making a good match.
Manual Process: Increase Visibility and Understandability of Nuanced Context Points
For contextual information that can not be automized, but should be accounted for, we worked to improve information placement to make it more usable. That is all student information is above the analogous tutor information.
(3/4)
PAIN: Poor understanding of who tutors were or what they expected created unempathetic matches that required repairs, clogging the pairing process.
REMEDY: Add questions and create assessments with usable reliable metrics to enable the organization to better understand tutors and students.
,
ADDING LESSON TYPE EXPECTATION QUESTION
In initial interviews, we found that tutors were being asked to go beyond what they expected to have to do. That is they were being asked to write lesson plans when they expected to just help with homework. By adding clear questions about expectations of either creating lesson plans or helping with homework, and implementing this into the automized process, we were able to eliminate this issue for future tutors.
"I didn't expect to have to make a lesson plan, I thought I was just going to help with homework.”
A tutor who had to create a french curriculum for a student.
.
.
Adding Behavior Prediction Metrics (Uncertainty Tolerance Assessment)
Pandemic Professors helps students who have unstable backgrounds, which bleed into their learning environment. The Uncertainty Tolerance Tolerance Assessment measures a tutor's ability to adapt or empathize with their student. Tests with over 105 participants proved the assessment was statistically significant in measuring its intended target. This is important for failure to predict this behaviour was the leading cause for most repairs.
How it works
Participants take a series of reflection and forced-choice questions, questions that force them into making a true choice by presenting a number of socially acceptable answers, about how they would respond in two different scenarios.
.
.
These scenarios were taken from real situations we observed in our research and were designed to measure tolerance of attendance, proactiveness in reaching out, and empathy for student life.
The assessment yields two scores:
Uncertainty Tolerance: this is the 0-12 score that results from the forced-choice questions, and is the primary measure.
Qualitative Response: this score can be infinitely negative or positive. It is the result of a keyword analysis of the reasoning given by each tutor. This is a secondary measure intended to provide context to uncertainty tolerance.
For example, if a tutor scored a 2/12 (very low) on uncertainty tolerance but had a +3 (very high) for their qualitative response, then we would suggest that Pandemic Professors check this tutor's reasoning, for their responses are not matching up.
.
(4/4)
PAIN: Tutor-Student pair relationships were not being supported by pandemic professors once they were created, and the company did not have time to check up on concerns leading to more repairs
REMEDY: Create a new support system feedback form that encourages empathetic conversations, and makes it easy to track problems.
,
Therapy Based Feedback Forms
To better support pairs, a new feedback form designed to build empathy has been created for tutors and students. It provides simple scores for easy pair tracking and analysis, enabling faster reactions to failing pairs in addition to the improvements in pair health.
This feedback form was derived from clinically tested questions and adapted for tutors. It has two sections: Session Evaluation and Listening Session.
Session Evaluation: builds understanding about session processes, allowing for tutors to better adjust their teaching methods. In our tests, we found that this successfully changed tutor's perspectives on their actions enabling them to make better lessons.
"...She just thinks it’s the most boring thing ever, which really surprised me...I didn’t realize she just didn’t get anything.”
Tutor of 6th Grader realizing she needed to change direction
.
.
Listening Session: The listening session requires pairs to employ empathetic listening skills of reflection to better understand their students in the session. In tests, all of the tutors we tested learned something new or surprising about their students and felt a stronger bond with them because of it.
.
.
EASY PAIR MONITORING
Centralizing the information coming from these forms into an actionable space ensures that when pairs were having trouble someone would care and take action
"I don't like filling out the (OLD) feedback form, because no one ever does anything.”
Tutor on the ineffectiveness of the old system.
Highlighting, scores across a set period of 6 sessions ensures that recent pair information is accounted for, and allowing tutors to mark if attention is needed, even if things are going well adds a necessary level of transparency.
,
Solution Outcomes
To solve this problem we delivered a suite of functional tools including:
- A working data center for them to run all of their processes
- A 90 pg. manual for how to work and understand each system
- A comprehensive set of forms to run new inputs and updates into their system
Overall this suite of tools serves to form...
A more efficient system
2X faster pairing of complex pairs in our new system as compared to simple pairs in the old system, and at 1:34 pairing is significantly faster than the 3 weeks that use to be expected of complex pairs.
A more empathetic system
With the introduction of a new, therapy-based feedback form that resulted in all of the participants studied learning something new about their student and their teaching, our system supports empathy.
Additionally, with the introduction of the Uncertainty Tolerance Assessment and associated metrics, which were proven statistically significant in a 105+ person study, we can now guarantee students who need more empathy, are more likely to be placed with a more empathetic tutor with our system.
A more effective system
By streamlining information from all parts of the ecosystem into one place, with prioritized information, Pandemic Professors should be able to better distribute their time to focus on the needs of struggling pairs, rather than incoming ones. Enabling this shift in priority will enable more effective learning from existing pairs and a more effective organizational structure as a whole.
Contribution Highlights
As a member of this capstone team, I had the pleasure of using my versatility to aid in the development of all parts of this solution.
It was a long project so I have just listed some highlights below.
Technical
I led the creation of the data center serving as technical lead. While this was an intimidating task, after completion, I can confidently answer any Google Sheets question you can think of from API integration to Z-tests.
Research
I was head of the development of the student and tutor onboarding forms. In this key role, I led user interviews and usability tests on the English and Spanish versions of these forms to ensure maximum usability. Leading this process on my own taught me how to better reach out and strategize long-term projects.
Strategy
I was able to utilize my business background to lead an extensive competitive analysis of the company.
Design
Built out initial data flow for all of the information in the system, which was presented to the board of directors to get the approval of our direction.
I led the visual and functional design of our future version prototype using my expertise from building the functional prototype and user research to ensure success.