HAI Institute’s latest AI Index Report covers 3 times as many data sets

We’re excited to release the AI Index 2019 Report, one of the most comprehensive studies about AI to date. Because AI touches so many aspects of society, the Index takes an interdisciplinary approach by designing, analyzing and distilling patterns about AI’s broad global impact on everything from national economies to job growth, research and public perception. The purpose of the project is to ensure that the discussion on AI takes into account data, serving practitioners, industry leaders, policymakers and funders, the general public and the media that inform it. 

The report is an independent initiative within Stanford University’s Human-Centered Artificial Intelligence Institute and in its third year. It is the result of a collaborative effort led by the AI Index Steering Committee, an interdisciplinary group of experts from across academia and industry, in collaboration with more than 35 sponsoring partners and data contributors.

The first two reports established the Index as a preeminent source of data about AI. The 2019 edition reinforces this position and dramatically expands its coverage, tracking three times as many data sets as the 2018 edition. It includes new measures and updated versions of previous ones. They cover all aspects of AI including technical performance, the economy, societal issues, and more.

A dataviz tool

To facilite access to the data, we’ve created the Global AI Vibrancy Tool, an interactive tool that compares countries’ global activities, including both a cross-country perspective, as well as a country-specific drill down. We find this tool more useful than an arbitrary single ranking of countries. 

A search tool for the latest data

We’re also releasing the AI Index arXiv Monitor, a tool to search this preprint repository for the most up to date data on technical progress in AI. All the raw data used for the report is publicly available on Google Drive. The Graphics folder provides hi-res images for all the charts. 

Highlights from the 2019 Index

This year’s AI Index is broken into nine chapters spanning important dimensions in artificial intelligence. Here is a small sample of insights from the report:

• Between 1998 and 2018, the volume of peer-reviewed AI papers has grown by more than 300%, accounting for 3% of all peer-reviewed journal publications and 9% of published conference papers.
• China now publishes as many AI journals and conference papers per year as Europe, having passed the USA in 2006. The Field-Weighted Citation Impact of USA publications is still about 50% higher than China’s.  
• Attendance at AI conferences continues to increase significantly. In 2019, NeurIPS expects 13,500 attendees, up 41% over 2018 and over 800% relative to 2012.   Even conferences such as AAAI and CVPR are seeing annual attendance growth around 30%.
• The WiML workshop has eight times more participants than it had in 2014 and AI4ALL has 20 times more alumni than it had in 2015. These increases reflect a continued effort to include women and underrepresented groups in the AI field.  
• In a year-and-a-half, the time required to train a large image classification system on cloud infrastructure has fallen from about three hours in October 2017 to about 88 seconds in July 2019. During the same period, the cost to train such a system has fallen similarly.
• Progress on some broad sets of natural-language processing (NLP) classification tasks, as captured in the SuperGLUE and SQuAD2.0 benchmarks, has been remarkably rapid; performance is still lower on some NLP tasks requiring reasoning, such as  the AI2 Reasoning Challenge, or human-level concept learning task, such as the Omniglot Challenge.
• In the US, the share of AI jobs grew from 0.3% in 2012 to 0.8% of total jobs in 2019. AI labor demand is growing, especially in high-tech services and the manufacturing sector.
• Globally, investment in AI startups continues its steady ascent. From a total of $1.3B raised in 2010 to over $40.4B in 2018 (with $37.4B in 2019 as of November 4th), funding has increased at an average annual growth rate of over 48%.
• Autonomous Vehicles (AVs) received the largest share of global investment over the last year with $7.7B (9.9% of the total), followed by Drug, Cancer and Therapy ($4.7B, 6.1%), Facial Recognition ($4.7B, 6.0%), Video Content ($3.6B, 4.5%), and Fraud Detection and Finance ($3.1B, 3.9%).
• 58% of large companies surveyed report adopting AI in at least one function or business unit in 2019, up from 47% in 2018.
• At the graduate level, AI has rapidly become the most popular specialization among computer science PhD students in North America, with over twice as many students as the second most popular specialization (security/information assurance). In 2018, over 21% of graduating Computer Science PhDs specialized in Artificial Intelligence/Machine Learning. 
• Diversifying AI faculty along gender lines has not shown great progress, with women comprising less than 20% of the new faculty hires in 2018.  Similarly, the share of female AI PhD recipients has remained virtually constant at 20% since 2010 in the US.
• There is a significant increase in AI-related legislation in congressional records, committee reports, and legislative transcripts around the world. 
• Fairness, Interpretability and Explainability are identified as the most frequently mentioned ethical challenges across 59 ethical AI principle documents. 

Working with global experts to get measuring right

Given that measurement and evaluation in complex domains remain fraught with subtleties, the AI Index has worked hard to avoid bias and seek input from many communities. As part of this effort, on October 30, 2019, the Stanford HAI-AI Index Workshop: Measurement in AI Policy: Opportunities and Challenges convened over 150 industry and academic experts from a variety of disciplines related to AI. The group discussed the many pressing issues that arise from data measurement of AI. The Workshop Proceedings will be available shortly here.  

We hope this is a useful resource for the global AI community and always welcome feedback and new ideas for next year. Contact us at AI-Index-Report@stanford.edu.