Artificial intelligence is an increasingly important element of science, medicine, and even the minutiae of our daily lives. Chatbots, digital assistants, and movie and music recommendations from streaming services all depend on “deep learning”—a process by which computer models are trained to recognize patterns in data.
That training requires powerful computers and lots and lots of energy—and associated carbon emissions. One of the most elaborate deep learning models, designed to produce human-like language and known as GPT-3, requires an amount of energy equivalent to the yearly consumption of 126 Danish homes and creates a carbon footprint equivalent to traveling 700,000 kilometers by car for a single training session.
That’s an extreme case. Still, the computing power used in deep learning grew 300,000-fold between 2012 and 2018, and if that pace of growth continues it’s not hard to see how artificial intelligence could have a major climate impact.
But this isn’t inevitable, say researchers at the University of Copenhagen in Denmark. As it happens, the answer to a problem with algorithms could be another algorithm: the researchers created a free, open-source program to assess and predict the carbon footprint of deep learning models.
Their program, called Carbontracker, is basically an add-on to deep learning models. It uses the Python programming language, like most such models, to make it easier to integrate. The researchers wrote their program so that it would not require a lot of computing power itself and would not interfere with the deep learning algorithms.
Carbontracker periodically collects measurements of the energy being used by different elements of the training computer, as well as information about the carbon intensity of local or regional electricity sources. It uses these data to predict the duration, energy use, and carbon footprint of a training session. To make it easier for users to grasp the results intuitively, Carbontracker reports carbon footprint predictions in terms of kilometers traveled by car.
The researchers tested Carbontracker by running it while training two different deep learning models for analyzing medical images on data sets of blood vessels in the eye, lung X-rays, and lung CT scans. The program was able to predict the energy used in these training sessions to within 4.9-19.1%, the carbon footprint to within 7.3-19.9%, and the duration of training to within 0.8-4.6%, they reported at a computer science workshop this past summer.
Once computer science researchers are aware of the carbon footprint of their work, “concrete and often simple steps can be taken to reduce the impact,” the researchers write in a paper presented at the conference. One strategy is to carry out deep learning training on computers located in regions or countries that have low-carbon energy sources. For example, the carbon emissions of training a model in Estonia may be up to 61 times that of training the same model in Sweden, the researchers estimate.
In addition, in many areas the carbon intensity of energy changes depending on the time of day. Choosing low-carbon-intensity hours to train models can cut the carbon footprint of deep learning by three-quarters in Denmark and by half in Great Britain.
Artificial intelligence researchers can also design algorithms to be as efficient as possible, minimizing the computing power (and thus carbon emissions) required for model training. And they can choose more energy-efficient hardware and calibrate its settings with energy use in mind.
Deep learning models are usually evaluated in terms of accuracy, the researchers note. “We propose that the total energy and carbon footprint of model development and training is reported alongside accuracy and similar metrics,” in order to make the field more environmentally friendly overall.
Source: Wolff Anthony L.F.. et al. “Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models.” ICML Workshop on “Challenges in Deploying and Monitoring Machine Learning Systems” 2020.
