AI Energy Score

AI Energy Score is an initiative to establish standardized energy efficiency ratings for AI models in order to help the industry make informed decisions about sustainability in AI development. The project aims to establish a standardized approach for evaluating the energy efficiency of AI model inference using controlled and comparable metrics. It focuses on benchmarking models across specific tasks and standardized hardware configurations in order to provide useful insights for researchers, developers, organizations, and policymakers.

The AI Energy Score framework evaluates the relative energy efficiency of AI models by measuring the energy consumption required to perform specific machine learning tasks. The results are translated into a simple star-rating system that represents the relative energy efficiency of models evaluated for a given task. Models that consume the least energy relative to others evaluated for the same task receive higher ratings, enabling users to easily compare models according to their energy efficiency.

To ensure comparability, the benchmarking methodology standardizes key variables that influence AI inference energy consumption. These include standardized tasks and datasets, consistent hardware configurations, and controlled benchmarking procedures. All benchmarks are conducted on NVIDIA H100 GPUs and focus on GPU energy consumption to eliminate variability introduced by different hardware environments. Models are tested using consistent configurations and batching strategies to ensure fair comparisons across systems.

The framework benchmarks models across several commonly used machine learning tasks spanning different modalities, such as text generation, reasoning, summarization, question answering, text classification, sentence similarity, image classification, object detection, speech recognition, image generation, and image captioning. Results from these evaluations are made publicly available through a leaderboard, enabling transparent comparison of AI models’ energy efficiency.

By providing a clear and transparent efficiency rating system and publicly available benchmarking results, the AI Energy Score initiative aims to encourage greater transparency in reporting energy efficiency metrics. The project seeks to support stakeholders in selecting more energy-efficient AI models and to promote sustainability considerations in the development, deployment, and procurement of AI systems.