Best Practices from 15+ Organizations Collaboratively Testing Generative-AI Tooling in Educational Contexts
January 29, 2024
The AI Career Coach Coalition, a group of over 15+ nonprofit and educational institutions, is pleased to share the following statement on its findings after 5 months of testing the application of AI in career development settings.
Context: The AI Career Coach aims to bridge the career support access gap by providing a scalable, personalized, and comprehensive career guidance tool that streamlines the path to one’s career aspirations. Coach is a chat-based online platform offering a menu of pre-built career development activities backed by a coalition of large-scale education and workforce institutions. The AI Career Coach Coalition partners are involved with testing, providing feedback, co-designing the platform’s features and capabilities, and contributing resources to embed into the platform’s knowledge database. Since July 2023, 500+ students, educators, program staff, and other stakeholders have been testing the beta version of Coach. The following key insights are drawn from our collective experience testing so far.
Key insights:
- AI enables personalization at scale, which is crucial in career development contexts. The power of personalization in career support services cannot be overstated. The majority of testers appreciated Coach for its ability to tailor responses based on the information they had provided, commending its adaptability. Coach’s strength in personalization demonstrates a huge opportunity for generative AI to deliver high-quality, personalized solutions at scale.
- The necessity for large-scale, multi-organization collaboration is critical in the realm of AI tool testing. To thoroughly understand and push the boundaries of these tools, it’s essential to engage in testing across a variety of contexts. Utilizing coalition models proves to be an effective strategy for obtaining widespread and robust feedback, essential for refining AI capabilities.
- Ensuring the monitoring, bias checking, and cultural sensitivity of AI tools is of paramount importance. The major differentiator in accessible and quality AI tools lies within the responsibility framework in developing the tool. It is imperative that AI tools are backed by continual and robust monitoring and bias checking to ensure that equity gaps are being closed, not widened.
- Responsibility to prepare people on how to use generative AI tools. As AI is such a new space, especially within education, it is important to set users up for success in leveraging AI-driven tools. It’s important to begin upfront by assessing users’ comfort with AI and couple that with training on how to use generative AI tools ranging from beginner to intermediate.
- AI offers an opportunity the sector cannot afford to miss out on. When developed responsibly (ie: considerations towards bias, etc.), this technology offers an unprecedented opportunity to scale up services to serve beneficiaries better.
Join the waitlist for Coach here: https://bit.ly/trycoach
Signatories:
Jared Chung, Executive Director, CareerVillage.org
Giovana Cury Pagliuca, Global Program Specialist Fellow, Generation.org
Scott Gullick, Head of Platform Partnerships, Opportunity@Work
Mark X. D’Acquisto, Teacher on Special Assignment, San Francisco Unified School District
Matt McCann, Senior Director of Alumni Advancement and Operations, Year Up
Heather S. Williams, Youth Programs Director, MassHire Berkshire County
Corey Mohn, President and Executive Director, CAPS Network
Marcus L. Strother, Executive Director, MENTOR California
Rajiv Puranik, Chief Technology Officer, Empower Work
Johannah Elliott, Director of Curriculum, Southwest Technical Education District of Yuma
Chris Shaw, Head of Product, UPchieve
Rebecca Bialecki, Executive Director, MassHire Franklin-Hampshire County
Larry Lieberman, Chief Executive Officer, Mouse
Scott Brewster, Ph.D., Co-Founder and Director of Partnerships, Hats & Ladders
Allison Rhodes-Baker, VP of Program Impact & Innovation, Big Brothers Big Sisters Bay Area