Potentials and Limitations of Active Learning: For the Reduction of Energy Consumption During Model Training

Lade...
Vorschaubild
Datum
2024-04-08
Herausgeber:innen
Autor:innen
Nenno, Sami
Zeitschriftentitel
ISSN der Zeitschrift
Bandtitel
Verlag
Weizenbaum Institute
Zusammenfassung

This article investigates the potential and limitations of using Active Learning (AL) to reduce AI’s carbon footprint and increase the accessibility of machine learning to low-resource projects. First, this paper reviews the recent literature on sustainable AI. The core of the article concerns AL as an emissions reduction technique. Because AL reduces the required data for model training, one can hypothesize that energy consumption  and, accordingly, carbon emissions – also decreases. This paper tests this assumption. The leading questions concern whether AL is more efficient than traditional data sampling strategies and how we can optimize AL for sustainability. The experiments show that the benefit of AL strongly depends on its parameter settings and the data set size. Only in limited scenarios does the size reduction outweigh the computational costs for AL. For projects with more resources for annotations, AL is beneficial from an ecological perspective and should ideally be paired with model compression techniques. For smaller projects, however, AL can even have a negative impact on machine learning’s carbon footprint.

Beschreibung
Schlagwörter
AI & Climate \ Sustainable AI \ Machine learning \ Sustainable AI \ Machine Learning \ NLP \ Active Learning \ Data Quality
Verwandte Ressource
Verwandte Ressource
Zitierform
Nenno, S. (2024). Potentials and Limitations of Active Learning: For the Reduction of Energy Consumption During Model Training. Weizenbaum Journal of the Digital Society, 4(1). https://doi.org/10.34669/WI.WJDS/4.1.3