Government

Shaping AI trends in work and society – an evidence based approach is crucial

AI has an enormous impact on work and society. Political action is required, which needs to be supported by a solid evidence base on AI in work and society. The German AI Observatory is building this evidence base.

Observatory for Artificial Intelligence in Work and Society

AI Observatory, The German Federal Ministry of Labour and Social Affairs

September 10, 2020 — 7 min read

AI has impacts on work and society that require action

Already today, common AI applications have positive and negatives as well as direct and indirect effects on the way people live and work. For example, text suggestions make it easier to write emails, news and music recommendations help us to find relevant and sought-for content which simultaneously influence our attention. Sorting applications eases recruiters’ work and at the same time influences the selection of candidates. This may however deepen inequalities. Navigation systems change route selections and thus shape cities and change how we travel. Warehouses are stocked more quickly, while individual processes are tracked closely, resulting in changes in the way warehouse workers live and work.

Policymakers have realised that they must act to safeguard the human-centred implementation of AI, while promoting innovation and competition. Over the last few years, many states have been developing AI strategies or started processes to do so. The European Commission has also communicated its objectives in a pan-European strategy. Both the German and the European AI strategy emphasise the importance of developing and implementing the technology in accordance with European values and fundamental rights. In addition, the German Federal Government’s artificial intelligence strategy stresses the need to observe the impact of AI on work and society to ensure everyone benefits from this technology. The German Observatory for Artificial Intelligence in Work and Society, a Policy Lab Digital, Work & Society project of the Federal Ministry of Labour and Social Affairs has been set up to build the evidence base in this area. The project commenced in March 2020.

AI indicators support evidence-based policymaking

At present, many of the impacts of AI on work and society remain unclear. However, tracking and understanding new developments and trends of AI in this area are crucial for policy development. Good decisions require a reliable evidence base. And new challenges need to be detected at an early stage, to inform policy decisions meaningfully. An important part of building this evidence base is the development of indicators that map the main trends of AI in work and society as accurately and timely as possible. Indicators may also help to benchmark the German AI landscape at an international level. At the same time, these indicators can also help to identify phenomena that require in depth investigation.

The evidence base on AI in work and society needs to be developed further

There are already a variety of AI indicators in the area of science and business. One example is an indicator showing the number of economic agents in the AI ecosystem by country, which is collected by AI Watch, the European Commission’s AI research institute. [1] This figure is an indicator of the extent to which a country is integrated in economic processes with relevance for AI. The analysis shows that the United States and China have the highest number of Economic AI Agents in the AI ecosystem; Germany ranks seventh (with 829 Economic agents in the AI ecosystem).

Infographic showing the Number of Economic Agents in the AI ecosystem by Country (Top 20) — Figure 1: Number of Economic Agents in the AI ecosystem by Country (Top 20). The data in this graph refers to the years 2009–2018. The economic AI operators may come from business, science or the public sector. What they all have in common is that they are playing an active role in their respective AI ecosystems and are expected to influence economic developments in these ecosystems in the future. The graph originally comes from the AI Watch dashboard. The figure for EU28 has been added.

There are however currently few indicators for AI in work and society. One reason may be that developing well-defined indicators involves a great deal of work and there are few data sources. Moreover, so far only few actors are working on AI indicators for work and society.

The German AI Observatory is developing indicators for work and society

The German Observatory for Artificial Intelligence in Work and Society (or AI Observatory) wants to change this. As a first step, it has organised a first workshop with experts from business, science and politics, social partners and civil-society organisations. The purpose of the workshop was to discuss the potential and the challenges when developing suitable AI indicators for work and society.

The experts were unanimous that developing reliable indicators to understand the development of AI in work and society is crucial. They also provided important insights into quality criteria for the indicators. Indicators should be evaluated on an ongoing basis – not least to factor in the dynamic development of the technology. In addition, when selecting and designing AI indicators, it is crucial to ensure that they provide an objective overall picture. Moreover, the indictors should take account of quantitative and qualitative aspects.

The workshop participants also discussed specific proposals for possible AI indicators in work and society:

What tasks substituted or changed by AI in the future and how does this impact labour demand and the skills required in certain types of jobs?
What implications does AI have for training and education?
To what extend can employees participate and have a say when companies implement AI applications?
What effect does AI have on inequality, social mobility? How does this change for different groups in society and in different regions?
How trustworthy and transparent are AI applications?

Over the coming months, the German AI Observatory will be developing the inputs and ideas form the workshop in close exchange with the experts.

Existing indicators on AI in work and society

Despite the limited number of suitable indicators available, there are already a number of indicators that address parts of the above questions. Here are two examples:

The penetration of AI skills in OECD countries is an indicator that was developed in collaboration of the OECD AI Policy Observatory (OECD.AI) and LinkedIn. It shows that LinkedIn users in Germany report 1.55 times as many AI skills than the OECD average. In comparison with other OECD countries, Germany ranks third behind Japan and the United States, but is ahead of the United Kingdom and France. This shows that Germany has a reasonably strong position internationally in terms of AI skills but is still lagging behind the United States.

Infographic showing the comparison of average penetration of AI skills among OECD countries with 100,000 LinkedIn members or more. — Figure 2: comparison of average penetration of AI skills among OECD countries with 100,000 LinkedIn members or more. This indicator is based on the “skills genome” index developed by LinkedIn, which tracks skills that are “both unique to and representative of” a given profession and calculates the proportion of skills classified by LinkedIn as AI skills. The proportion of AI skills in a profession may vary from country to country. In this way, it is possible to compare different countries. The average AI skills in OECD countries are used as a benchmark. It can be assumed that LinkedIn is a relatively accurate reflection of the digital skills in developed economies. Further information can be found in the methodological note of the OECD AI Policy Observatory (OECD.AI) and in an article by LinkedIn. [2]

A different indicator, developed by the UK innovation foundation Nesta, offers an insight into diversity in AI research. Preliminary studies support the theory that, gender diversity leads to the inclusion of a greater diversity of perspectives and, consequently, more diverse and inclusive AI systems. [3] The indicator shows the relative representation of AI publications with at least one female author on the arXiv preprint document server. The negative figure for Germany indicates that women are underrepresented in AI research compared with other countries.

Infographic showing the relative representation of AI publications with at least one female author on arXiv [4] — Figure 3: relative representation of AI publications with at least one female author on arXiv [4]. The Y-axis shows whether women working in AI on arXiv are overrepresented (positive figure) or underrepresented (negative figure) (Z-test to compare the proportion of women in AI publications with the proportion of women in all publications). The study covers the period from 1991 to 2018. The colours show whether the result is statistically significant (blue) or not (pink). There is evidence that arXiv is widely used in the AI research community for sharing results. [5]

Further indicators need to be developed

The indicators described above provide important insight into skills and diversity in AI research. However, to shed a light on further issues related to AI in work and society further indicators need to be developed quickly given that political decisions are already being made, as the EU Commission’s white paper on artificial intelligence shows.

The German AI Observatory is collaborating closely with AI Watch of the European Commission’s Joint Research Centre and the OECD AI Policy Observatory (OECD.AI). Both institutions are working intensely on the development of further indicators. The joint goal is to broaden and expand the evidence base for decision making in the area of AI in work and society.

References

[1] This workshop was chaired by Stefan Heumann of Stiftung Neue Verantwortung. In this connection, Stefan also provided inspiration for parts of this text.
[2] OECD (2020), OECD.AI data from partners: methodological note, https://www.oecd.ai/assets/files/Methodology_20200219.pdf (last accessed on 7 July 2020); LinkedIn (2019), How we mapped the “skills genome” of emerging jobs, https://engineering.linkedin.com/blog/2019/how-we-mapped-the-skills-genome-of-emerging-jobs (last accessed on 24 July 2020).
[3] Stathoulopoulos, K., J. Mateos-Garcia (2019), Gender Diversity in AI Research, https://www.nesta.org.uk/report/gender-diversity-ai/ (last accessed on 24 July 2020)
[4] Open-access repository for electronic preprints with more than 1.5 million articles covering research in physics, mathematics, computer science, statistics, financial mathematics, quantitative biology, electrical engineering, systems science, and economics.
[5] Klinger, J., J. Mateos-Garcia, K. Stathoulopoulos (2018), Deep learning, deep change? Mapping the Development of the Artificial Intelligence General Purpose Technology, Computers and Society.

This post was originally published by the Denkfabrik Digitale Arbetietsgesellschaft, the German Federal Ministry of Labour and Social Affairs’ Digital, Work & Society Policy Lab

Observatory for Artificial Intelligence in Work and Society

The German Federal Ministry of Labour and Social Affairs

AI Observatory

See all posts

Disclaimer: The opinions expressed and arguments employed herein are solely those of the authors and do not necessarily reflect the official views of the OECD or its member countries. The Organisation cannot be held responsible for possible violations of copyright resulting from the posting of any written material on this website/blog.