Sesame Street English 2.0
Managing the whole product, with focus on User Experience and User Research, of an eLearning digital product with Sesame Street
Timeline
• November 2015 - August 2017
My Role
• UX Design and Research, Content
introduction
Japanese education giant Nagase Brothers, in collaboration with Sesame Street Workshop, aimed to launch an updated version of their English Education eLearning app. This app was designed for use in schools and homes across Japan, with plans for gradual expansion into other parts of Asia. The application was operated through a tablet or laptop connected to a TV, utilizing a specially designed touch screen that seamlessly fit over most televisions.
My role during this period was 課長, which translates to Section or Department Manager. Upon reflection, I recognized that my responsibilities encompassed UX Research and Design aspects.
Throughout the project, I pursued multiple UX goals:
My role during this period was 課長, which translates to Section or Department Manager. Upon reflection, I recognized that my responsibilities encompassed UX Research and Design aspects.
Throughout the project, I pursued multiple UX goals:
- Guaranteed Instructors could use the new product properly and smoothly during lessons.
- Ensured that students could navigate the at-home self-study component of the program smoothly, contributing to improved test scores.
- Conducted research to ensure a cohesive content flow that accommodates both instructors and students, both in the classroom and at home.
Although I took the lead from the Japan side, the developers and designers at Sesame Workshop held the design files and responsibilities initially. However, upon my involvement, I was able to shape and contribute significantly. It was a collaborative effort, with instructors providing valuable data for ongoing testing, and students graciously allowing me to observe their lessons, sometimes evoking a mix of excitement and anxiety.
problem
Given the scale of the project, I encountered several challenges. Firstly, I aimed to clearly define all our users. Secondly, ensuring a seamless user experience for each participant was a priority. Thirdly, I needed to assess whether all students could successfully complete the 12 levels of material in the home study program. Lastly, I had to ensure the integration of new material within the allotted lesson time.
My approach
Upon joining the company, my initial focus was identifying our primary users. This understanding of the product's users empowered me to provide user-centric design suggestions to the Sesame Workshop team, which were incorporated into the final product. Dealing with multiple stakeholders both in Japan and internationally, I navigated the challenge of ensuring alignment with everyone's goals before the product launch.
initial questions
How could I help identify our users better by researching and analyzing gathered data that will allow them to have a smoother experience?
How could I help our students improve by testing and iterating the new content that will allow them to achieve higher grades to get into the schools they want?
How could I help instructors and students with their lessons by observing the lessons and making adjustments to the design, allowing them to finish within the lesson time?
initial questions
Who’s voice am I representing?
The initial step involved identifying the primary users of the app and devising strategies to enhance their overall experience. Given the predominantly male-oriented Japanese work culture, implicit biases existed towards the main users, including Japanese instructors at internal and partner schools, Owner Senseis (Japanese women teaching children at home), and foreign instructors both in Japan and abroad. As most instructors were female, an internal shift was needed to connect with them and address their unique challenges instead of just assumptions made previously.
Students were categorized into 12 levels corresponding to their public school grade. A big task for me involved integrating the new SSE 2.0 content with the existing material. It became apparent that the new content alone wouldn't suffice for a complete lesson, necessitating thoughtful incorporation after considerable investment.
The final challenge was balancing the amount of content for each lesson to fit the designated time. This required comprehensive testing with all users to gauge their feedback on pacing, flow, and difficulty of the added content.
Throughout this process, numerous meetings with various team members and stakeholders were conducted to establish a clear understanding of the needs and expectations surrounding SSE 2.0.
Students were categorized into 12 levels corresponding to their public school grade. A big task for me involved integrating the new SSE 2.0 content with the existing material. It became apparent that the new content alone wouldn't suffice for a complete lesson, necessitating thoughtful incorporation after considerable investment.
The final challenge was balancing the amount of content for each lesson to fit the designated time. This required comprehensive testing with all users to gauge their feedback on pacing, flow, and difficulty of the added content.
Throughout this process, numerous meetings with various team members and stakeholders were conducted to establish a clear understanding of the needs and expectations surrounding SSE 2.0.
Research
We’re educators, we love research!
Having accumulated over 9 years of experience in education, continuous research on students and lessons was a standard practice both at a personal level and within the company. Armed with a deep understanding of each student's goals, motivations, current English level, and pain points, acquired through one-on-one interviews, my extensive teaching background proved invaluable for this project.
To kick off the research phase, I conducted lesson observations with 21 Japanese and foreign instructors in the Greater Tokyo area. Additionally, I observed students across our 9 courses in various schools. To enrich the dataset, I surveyed Japanese instructors and gathered feedback during monthly training meetings. Recognizing my leadership in the project, I was invited to join weekly meetings with Sesame Street and other stakeholders.
The plethora of quantitative and qualitative insights obtained from observations and feedback during training sessions became the foundation for future design decisions. Identifying technical issues, including spelling and grammar, led me to conduct quality checks on all 481 lesson segments within SSE 2.0, involving meticulous examination of PDFs for each screen. This process consumed around 48 working hours but provided an unparalleled understanding of the entire product.
Given my in-depth knowledge of both new and existing content, coupled with the assumption of training duties, I became THE specialist for our product. Testing various iterations of content for each level was imperative to ensure alignment with lesson times. This expertise was crucial in enhancing educational benefits, ultimately justifying an increase in our monthly pricing.
As our product was an eLearning tool, I navigated through user interactions—from logging in to lesson completion, direct input, review, and logging the lesson or at-home review. While there weren't intricate user flows due to the predetermined lesson order, each step played a vital role in refining the overall user experience.
To kick off the research phase, I conducted lesson observations with 21 Japanese and foreign instructors in the Greater Tokyo area. Additionally, I observed students across our 9 courses in various schools. To enrich the dataset, I surveyed Japanese instructors and gathered feedback during monthly training meetings. Recognizing my leadership in the project, I was invited to join weekly meetings with Sesame Street and other stakeholders.
The plethora of quantitative and qualitative insights obtained from observations and feedback during training sessions became the foundation for future design decisions. Identifying technical issues, including spelling and grammar, led me to conduct quality checks on all 481 lesson segments within SSE 2.0, involving meticulous examination of PDFs for each screen. This process consumed around 48 working hours but provided an unparalleled understanding of the entire product.
Given my in-depth knowledge of both new and existing content, coupled with the assumption of training duties, I became THE specialist for our product. Testing various iterations of content for each level was imperative to ensure alignment with lesson times. This expertise was crucial in enhancing educational benefits, ultimately justifying an increase in our monthly pricing.
As our product was an eLearning tool, I navigated through user interactions—from logging in to lesson completion, direct input, review, and logging the lesson or at-home review. While there weren't intricate user flows due to the predetermined lesson order, each step played a vital role in refining the overall user experience.
insights
The insights amplified their voices.
Following the extensive research, the next phase involved analyzing the gathered data to derive insights for introducing the product and ensuring satisfaction for all users. Armed with this data, I possessed compelling evidence to advocate for improved design decisions among stakeholders.
Instructors
Instructors could then be broken into these groups:
Older Instructors:
This group tended to encounter more challenges in using the program during lessons despite their extensive teaching experience and skills.
Their unique concerns included:
Older Instructors:
This group tended to encounter more challenges in using the program during lessons despite their extensive teaching experience and skills.
Their unique concerns included:
- They exhibited a strong attachment to the lessons they were initially trained for, facing initial challenges in adapting to the new content.
- Technical issues proved to be a significant source of frustration, often resulting in lessons exceeding the allotted time for these instructors.
- They frequently held the belief that their students would fare well, but the challenge lay in teaching the new content, particularly the Middle School (M Course) levels. These levels posed a significant challenge to the English proficiency of a majority of our Japanese Instructors.
Younger Instructors:
The majority of these instructors were receptive to the idea of adapting to new content. The ongoing mention of SSE 2.0 in every training session, facilitated by a hiring boom shortly after my arrival, kept the concept at the forefront of their minds.
Despite this openness, they still faced certain pain points, including:
The majority of these instructors were receptive to the idea of adapting to new content. The ongoing mention of SSE 2.0 in every training session, facilitated by a hiring boom shortly after my arrival, kept the concept at the forefront of their minds.
Despite this openness, they still faced certain pain points, including:
Students
Concerning our students, our lessons adhered to various ESL testing guidelines such as TOEIC, TOEFL, and CEFR, simplifying the alignment of content to their proficiency levels.
However, a notable pain point for them was:
However, a notable pain point for them was:
- While our Kindergarten to Primary School courses aligned with existing content, our M Course was entirely new, constructed from the ground up. Observing the best students and instructors, it became evident that the content, in its current form, would be too challenging.
Parents
With a significant expansion in home study content, parental involvement became more crucial. While the majority of parents were willing to engage in this process, especially if it meant better test readiness for their students, they still encountered a pain point.
That being:
That being:
- A very small number of parents, regardless of circumstances, were unwilling to engage in home study with their children. Given that the majority of the new content emphasized this aspect, these particular parents failed to perceive the value justifying a price increase for the new 2.0 content.
iterating
Our students weren’t the only ones that needed testing.
Given Sesame Street's existing product and constraints on visual UI changes, this stage primarily involved internal testing with a focus on Tokyo-based Owner Senseis. Similar to earlier stages, testing encompassed both product usability and content evaluation during lessons and home reviews.
This phase was notably time-consuming as the 2.0 content underwent constant iteration, introducing new challenges that required multiple checks. Continuous feedback from instructors, students, and stakeholders in weekly meetings ensured prompt implementation of solutions within reasonable limits.
With product design finalized, the next step involved putting it into the hands of users for fine-tuning. Two critical areas tested outside of user experience were ensuring the content's impact on test scores and evaluating the timing of the new material within the classroom setting.
Test Scores
Japanese education's inclination towards testing and data collection facilitated easy access to this area. A clear understanding emerged regarding the alignment of all students within their classes and where these classes fell within ESL guidelines. The new 2.0 content, particularly the home study component, provided students with increased English exposure, enhancing the potential for improved scores.
A pain point for testing was:
A pain point for testing was:
- Acknowledging that testing had its imperfections, conversations with students in lower classes who could speak near-fluent English, often due to being born abroad, brought attention to a unique challenge. Tokyo had a significant number of these "Returnees," and due to the set level system, these students wouldn't see a significant increase in their skill level until years later while being in a class matched to their school grade. While suggesting that the M course would be an ideal fit for them, it was recognized that it wouldn't align with their busy schedules. The company, however, maintained a stance against allowing students to jump too many levels to ensure a longer duration of engagement with the program.
Timing
For Kindergarten and Primary School classes, introducing the new 2.0 content as it was would have exceeded the set time limits for these classes. I examined other elements in the current lessons across all levels that could be trimmed and rearranged to accommodate the new content. After additional testing, final timings were established and successfully fit within the specified time limits.
A common issue with timing was:
A common issue with timing was:
- The M Courses posed another challenge. Initially designed to provide two hours of content, this clashed with the existing lesson structure where the longest lesson was one hour. Considering the additional time spent on questions and supplementary study materials, an instructor would invest an extra hour, depending on class size. Recognizing that the two-hour course, coupled with challenging content, would be impractical for Japanese instructors to manage, I made the decision to break down each course into two days.
For Day One, I restructured parts of the lesson meant for preview and review. Additionally, I introduced new activities to create a comprehensive one-hour lesson.
Day Two focused on going over the lesson's content and allocated time for the concluding segment, typically involving prepared speeches that required additional time from students.
impact and afterthoughts
After multiple iterations and rounds of testing, the SSE 2.0 project finally received the president's approval with a few requested tweaks, paving the way for its launch. It was a notable achievement considering the president's usual stringent approval timelines for projects.
Concluding the training sessions for staff, stakeholders, and Japanese and Foreign Instructors, I handed over the responsibility for SSE 2.0. Although the impact post my departure remains unseen, I'm aware that 3,589 students in Japan alone will benefit from the work done. The expansion of the Sesame Street English program to China, Hong Kong, Taiwan, and Vietnam, along with a subsequent price increase, further solidifies the project's success.
Remaining in contact with the trained instructors, I've received feedback on the positive effects of the content on students. While I wish I had the opportunity to start the project from its inception, I acknowledge the substantial accomplishments and recognize the potential for further improvements in a hypothetical SSE 3.0.
This project, despite challenges and miscommunication, stands out as a testament to my ability to make user-centered design decisions even without prior experience in certain tasks. Reflecting on a hypothetical future with more control, I would love to have recreated the User Interface, even in the absence of access to Sesame Street's internal artwork.
Overall, my contribution to SSE 2.0 highlights my resilience, adaptability, and commitment to delivering impactful products in the field of education.
Concluding the training sessions for staff, stakeholders, and Japanese and Foreign Instructors, I handed over the responsibility for SSE 2.0. Although the impact post my departure remains unseen, I'm aware that 3,589 students in Japan alone will benefit from the work done. The expansion of the Sesame Street English program to China, Hong Kong, Taiwan, and Vietnam, along with a subsequent price increase, further solidifies the project's success.
Remaining in contact with the trained instructors, I've received feedback on the positive effects of the content on students. While I wish I had the opportunity to start the project from its inception, I acknowledge the substantial accomplishments and recognize the potential for further improvements in a hypothetical SSE 3.0.
This project, despite challenges and miscommunication, stands out as a testament to my ability to make user-centered design decisions even without prior experience in certain tasks. Reflecting on a hypothetical future with more control, I would love to have recreated the User Interface, even in the absence of access to Sesame Street's internal artwork.
Overall, my contribution to SSE 2.0 highlights my resilience, adaptability, and commitment to delivering impactful products in the field of education.
Want to work together? Get in touch!
collab@tommydesigns.me