Students thought they were job-ready, but AI proved them wrong

A new study has found that artificial intelligence-driven career preparation platforms are significantly improving how university students understand their skills, career readiness, and gaps in employability. The research highlights a growing shift in higher education, where AI tools are increasingly used not just for learning but for guiding students through complex career decisions in a rapidly evolving job market.

The study, titled “AI-Assisted Career Preparation and Skill Gap Awareness: A Retrospective Pretest-Posttest Study,” published in Education Sciences, examines how an AI-enabled platform influences undergraduate students’ awareness of their skills and preparedness for internships and employment.

Conducted at Nanyang Technological University in Singapore, the study evaluates a structured three-month career preparation exercise involving 100 students across STEM and humanities disciplines. Using a combination of pre-exercise surveys, post-exercise retrospective assessments, and statistical analysis, the research provides detailed evidence that AI-driven tools can reshape how students perceive their capabilities and plan their career development.

AI tools expose overconfidence and improve skill awareness

Students initially overestimated their understanding of their own skills and career readiness. Before using the AI platform, a majority of participants reported confidence in their knowledge of industry-required skills, awareness of their skill gaps, and ability to plan future development.

However, once exposed to the AI-driven system, these perceptions shifted significantly. The retrospective pretest results show that many students reassessed their earlier confidence levels, with an increase in neutral and negative responses when reflecting on their pre-exercise understanding. This shift suggests that the platform prompted deeper self-reflection and revealed gaps that students had previously overlooked.

The platform works by analyzing uploaded resumes, generating personalized skill profiles, and matching them with real-time job market requirements. It then recommends career pathways, identifies missing competencies, and suggests internships aligned with each student’s profile. This structured process enables students to compare their perceived abilities against actual industry expectations.

Statistical analysis confirms that these changes were not incidental. Across all key indicators, including knowledge of required skills, awareness of skill gaps, and ability to prioritize development, the study recorded highly significant improvements, with p-values far below conventional thresholds.

The findings highlight a critical issue in career preparation: students often lack accurate self-assessment. By introducing data-driven feedback, AI tools can correct these misperceptions and provide a more realistic foundation for career planning.

Strong gains in career planning, but gaps remain in actionable guidance

The study finds that AI platforms significantly improve students’ ability to plan their career development. Participants reported stronger clarity in identifying career options, prioritizing skills, and navigating employment pathways after using the platform.

Effect size analysis indicates that the most notable improvements occurred in areas such as understanding career trajectories and building confidence in managing future career transitions. These gains suggest that AI systems are particularly effective in helping students move from abstract career ideas to more structured and informed plans.

The study also identifies important limitations. While students became more aware of their skill gaps, translating that awareness into concrete actions remained a challenge. For example, participants reported lower confidence in developing specific course plans or identifying resources needed to close those gaps.

This limitation reflects a design gap in current AI career platforms. While they excel at diagnosing problems, such as identifying missing skills, they are less effective at prescribing detailed solutions, such as linking those gaps directly to academic courses or training programs. The findings suggest that future systems must go beyond recommendation engines and evolve into integrated career ecosystems that connect skills analysis with actionable learning pathways.

Potential of AI in career services

The study also evaluates user perceptions of the platform’s usability and effectiveness, revealing consistently positive responses. Participants rated the system highly for ease of use, navigation, and overall functionality, with median scores above neutral across all metrics. The platform’s usability emerged as a key strength, with strong agreement among users that the interface was intuitive and accessible. High internal consistency scores confirm that participants shared similar positive perceptions, indicating a reliable user experience across the sample.

In terms of career impact, students reported that the platform was particularly useful for making internship decisions and exploring career options. Many indicated they would recommend the tool to peers, signaling strong perceived value in real-world applications.

However, trust in the platform’s analytical capabilities showed slightly lower scores. Some users expressed uncertainty about how accurately the AI could evaluate their skills or interpret job market requirements. This points to an emerging challenge in AI adoption: while users may appreciate functionality, they remain cautious about the underlying algorithms.

Qualitative feedback further reveals areas for improvement. Participants called for better search functionality, faster system performance, and more customizable recommendations tailored to specific academic disciplines. There were also requests for more accurate and up-to-date internship listings, suggesting that real-time data integration remains a critical requirement for such platforms.

Bridging the skills gap in a rapidly changing job market

According to projections cited in the research, millions of jobs will be displaced while new roles emerge, requiring entirely different skill sets. This dynamic environment makes traditional career preparation methods increasingly inadequate.

AI-driven platforms offer a potential solution by continuously analyzing job market trends and aligning student skills with emerging demands. By integrating machine learning and natural language processing, these systems can process large volumes of job data, identify in-demand competencies, and provide real-time guidance.

The study demonstrates that such tools can play a critical role in bridging the skills gap, particularly by enhancing students’ awareness and decision-making capabilities. However, it also underscores that technology alone is not sufficient. Effective career preparation requires collaboration between universities, employers, and students, with AI serving as a complementary tool rather than a replacement for human guidance.

Toward smarter, more integrated career platforms

The research outlines several directions for future development, emphasizing the need for more advanced and integrated AI systems. Key recommendations include improving personalization through user behavior analysis, incorporating interactive tools such as decision trees, and providing real-time updates on courses and career opportunities.

Another major opportunity lies in predictive analytics. By forecasting future skill demands based on industry trends, AI platforms could help students prepare proactively rather than reactively. This would mark a shift from static career guidance to dynamic, forward-looking planning.

The study also calls for longitudinal research to assess the long-term impact of these tools on career outcomes. While the current findings demonstrate improvements in perception and planning, further evidence is needed to determine whether these translate into better employment results.

Importantly, the research notes that AI platforms should be embedded within broader institutional support systems. Career counselors and educators remain essential in helping students interpret AI-generated insights and translate them into meaningful actions.

AI enhances perception, but real-world impact still emerging

The study also acknowledges its limitations. The findings are based on self-reported data, meaning they reflect how students perceive their skills rather than objective measures of competence.

The sample is also limited to a single university, which may affect the generalizability of the results. Cultural, institutional, and labor market differences could influence how students interact with similar platforms in other contexts.