AI performance

Automation is not a new phenomenon, but AI and robotics are increasingly changing the type of tasks performed in most jobs, thereby increasing the speed and depth of change and the propensity of certain jobs to be automated.

The OECD launched the Artificial Intelligence and the Future of Skills project to develop a programme that could assess the capabilities of AI and robotics and their impact on education and work. Considering both expected future development of AI and existing research, AI and the Future of Skills project aims to develop a new set of measures to serve as a foundation for research and policy on how AI and robotics will transform skill demand and educational requirements in the decades ahead. It addresses the following concrete questions:

• What human capabilities will be too difficult for AI and robotics to reproduce over the next few decades?
• What education and training will be needed to allow most people to develop some work-related capabilities that are beyond the capabilities of AI and robotics?

To anticipate and adapt to developments in AI, it is crucial for policy makers to have a solid understanding of what AI can do and how its capabilities and limitations compare to those of people. Understanding this comparison is also important for AI researchers as they work to improve their technologies.

The project builds upon a successful pilot project that offered a novel way to measure AI capabilities and connect them to skills needed for work and education. Using the OECD’s Survey of Adult Skills (PIAAC), this study showed that 62% of workers in OECD countries use literacy, numeracy and problem-solving skills on a daily basis at work at a level AI systems were able to perform in 2016 (Elliott, 2017).

These results serve as strong motivator for the project to build valid measures that translate AI and robotics capabilities into terms that are meaningful for policy and illustrate the implications for education and training.

The project is relying on an iterative strategy involving valid, reliable and meaningful measures of AI capabilities:

The first volume from the AI and the Future of Skills project, Capabilities and Assessments, lays the foundation for a robust methodology to assess the capabilities of AI and compare them with human capabilities. Drawing from computer science and many areas in psychology, the report describes key difficulties in assessing AI and various taxonomies of skills and types of assessments that might be used to describe and measure AI capabilities. The contributing experts each formulated recommendations for the project, providing crucial elements for building a comprehensive approach to the project’s goals.

Forthcoming reports:

• A 5-year update of the Pilot Study using PIAAC
• Methodology report: Volume 2

For more information about this work, please consult the AIFS Project Brief or contact the team via futureofskills@oecd.org.