We’re lifting the hood on the activity development process for Coach, our AI career coaching platform.

In the rapidly evolving career development landscape, artificial intelligence offers new possibilities for providing personalized guidance at scale. At CareerVillage, we’ve been exploring this frontier with Coach, our AI-powered career coaching platform. Today, we’re lifting the hood into our activity development process, which may be valuable for organizations considering how to enhance their own career support systems.

Leveraging a Rich Knowledge Base

Our journey in career guidance began more than 13 years ago with the launch of CareerVillage.org, a platform where professionals volunteer to answer career-related questions. This initiative has connected us with 150,000+ professional volunteers, 4 million+ learners, and a massive, growing, and openly available repository of career-related questions and advice.

This wealth of real-world career insights forms the foundation of our approach to AI-driven career guidance. It’s allowed us to identify common patterns, recurring challenges, and effective strategies across various career paths and industries. (It is important to note that Coach is not trained on CareerVillage user data. Instead, we’ve trained Coach to point users to relevant Q&A threads on CareerVillage.org.)

Collaborative Expertise

While all that professional knowledge provides a strong starting point, our team also collaborates with career experts to ensure Coach’s guidance is relevant, accurate, and accessible.

During Coach’s initial development and testing phases, we worked with more than 20 career development organizations, workforce boards, and educational institutions to co-design the tool, source the best career resources, and test it with a variety of learners, educators, and counselors around the world. 

And now, we work with an advisory group of counselors, coaches, and advisors to develop and review new Coach activities.

This collaborative approach ensures that our AI-driven guidance is grounded in both data-driven insights and expert human judgment.

A Methodical Approach to Creating Career Development Activities

Creating effective career development activities for an AI platform requires a structured, thoughtful process. Here’s an overview of our methodology:

1. Idea Generation: We source ideas from user feedback, industry trends, and expert recommendations. This ensures our activities address real, current needs in the career development space.
2. Prioritization: We evaluate potential activities using two frameworks:
   • Our Career Readiness Impact Model, based on academic research
   • Our Activity Prioritization Framework, which assesses relevance to specific learner populations (e.g., community college students or job training program participants), potential quality, and technical feasibility
3. Expert Design: Career counselors and coaches in the working group create detailed activity outlines, providing a step-by-step guide and learning goals for each activity.
4. Peer Review: Other members of the advisory group thoroughly review these outlines to ensure accuracy and completeness.
5. Technical Implementation: Our technical team builds activities to exact specifications, ensuring they align with learning objectives.
6. Testing and Iteration: We conduct internal testing followed by controlled user testing, labeled “in testing!” to gather real-world feedback. Activities are then adjusted, based on feedback they receive.

This rigorous process ensures that each activity is relevant and effective and aligns with best practices in career development.

The Future of Career Development

The integration of AI into career development opens up exciting possibilities. It allows for:

• Personalized guidance at scale, delivered at a consistent high quality
• 24/7 accessibility to career support
• Data-driven insights into career development trends and needs

It’s important to note that AI is not a replacement for human expertise. Rather, it’s a powerful tool to augment and extend the reach of career development professionals. In fact, multiple studies, including a small pilot study conducted by Generation.org using Coach, have found that learners prefer a hybrid approach that includes human and AI coaching (view our whitepaper).

We hope these insights into our development process have been valuable. If you’re interested in learning more about AI-driven career development or have insights of your own to share, we’d love to hear from you. The future of career guidance is a collaborative effort, and we’re excited to be part of the movement to make career readiness a reality for all.