Research involving 5,172 support staff showed that AI assistants increased productivity by 15% on average. While less experienced workers saw gains in both speed and quality, experts experienced a slight decline in quality. Another experiment with 453 professionals demonstrated that LLMs reduced task completion time by 40% and improved quality by 18%.
Cognitive psychology defines expertise as the accumulation of mental schemes stored in long-term memory. Experts recognize meaningful patterns, classify tasks by their underlying structure, and identify the first step of a solution rapidly. Methods based on recognizing the first step of a solution have correlated with knowledge tests up to 0.92 and reduced evaluation time by 2–4.9 times.
Current hiring processes often rely on superficial metrics, such as keyword matching in resumes via ATS, non-expert screening, and technical interviews focused on typical tasks. These methods evaluate areas where LLM-assisted novices perform similarly to experts. Consequently, hiring processes fail to measure the critical skills of recognition and verification required for complex tasks.
To improve hiring, organizations should shift toward structured interviews with predefined scoring scales and work samples that mirror real-world tasks. Experts should define job roles based on specific task lists rather than generic keywords. These methods are proven to predict job performance more accurately than traditional, unstructured interviews.
New evaluation methods, such as testing a candidate's ability to identify errors in LLM-generated output or solving problems beyond AI capabilities, are proposed. While these approaches require significant time from experts to develop and assess, they are necessary to distinguish true expertise from AI-assisted performance and prevent long-term skill gaps.
