Alright, let's continue our talk on AI. We're all hyped about the next leap – GPT-5. Whispers suggest capabilities that could blow GPT-4 out of the water(may be AGI?). From coding assistants to creative partners, these Large Language Models (LLMs) are reshaping our digital world. But beneath the dazzling surface of human-like text generation and complex problem-solving lies a rather inconvenient truth: training these AI behemoths guzzles insane amounts of energy. training these AI behemoths guzzles insane amounts of energy. insane So, GPT-5 is just around the corner, right? While everyone's excited, we should probably get real about the hidden price tag. Beyond the dollars and tech, think about the sheer energy these things guzzle and the pollution they create. Is this rapid AI progress actually good for the environment in the long run? The Scale is Getting Absurd: From Billions to Trillions (of Parameters) Remember BERT? GPT-2? They were groundbreaking, showing that bigger models trained on more data meant better performance. Then came GPT-3 with its 175 billion parameters, a landmark achievement. GPT-4 reportedly upped the ante significantly, maybe hitting around 1.8 trillion parameters. trillion Now, brace yourselves for GPT-5. The rumor mill is churning out numbers anywhere from 10 to 50 trillion parameters. 10 to 50 trillion parameters trillion Think about that: We're potentially talking about a model 5 to 25 times larger than GPT-4. Think about that: We're potentially talking about a model 5 to 25 times larger than GPT-4. Think about that: Training these models involves teaching them patterns from colossal datasets – think huge chunks of the internet. This isn't something you do on your gaming PC. It requires armies of specialized hardware. Feeding the Beast: GPUs, TPUs, and the Power Grid Training LLMs is like trying to solve a billion-piece jigsaw puzzle simultaneously. You need massively parallel computation, and that means specialized chips: GPUs (Graphics Processing Units): NVIDIA's A100s have been workhorses, but the newer H100s are the current kings. They're power-hungry beasts. TPUs (Tensor Processing Units): Google's custom silicon, optimized for machine learning. GPUs (Graphics Processing Units): NVIDIA's A100s have been workhorses, but the newer H100s are the current kings. They're power-hungry beasts. GPUs (Graphics Processing Units): NVIDIA's A100s have been workhorses, but the newer H100s are the current kings. They're power-hungry beasts. GPUs (Graphics Processing Units): TPUs (Tensor Processing Units): Google's custom silicon, optimized for machine learning. TPUs (Tensor Processing Units): Google's custom silicon, optimized for machine learning. TPUs (Tensor Processing Units): Reports suggest GPT-4 training might have used the equivalent of 20,000 NVIDIA A100 GPUs running for three months. For GPT-5? Predictions point towards needing at least 50,000 of the more powerful H100 GPUs for a similar timeframe. Data centers are already sweating bullets trying to power and cool these things. 20,000 NVIDIA A100 GPUs running for three months. at least 50,000 of the more powerful H100 GPUs more powerful It's crucial to distinguish: Training: The initial, incredibly energy-intensive phase where the model learns. This is a one-off (per major version) but massive energy sink. Inference: Using the trained model to answer your prompts. Each query uses a tiny bit of energy, but billions of queries add up fast. Even the estimated 0.001 kWh per GPT-5 query hints at a heftier underlying model. Training: The initial, incredibly energy-intensive phase where the model learns. This is a one-off (per major version) but massive energy sink. Training: Inference: Using the trained model to answer your prompts. Each query uses a tiny bit of energy, but billions of queries add up fast. Even the estimated 0.001 kWh per GPT-5 query hints at a heftier underlying model. Inference: So, What's the Energy Bill for GPT-5? (Spoiler: It's HUGE) We don't have exact figures for GPT-5 yet (OpenAI keeps this stuff close to the chest), but we can extrapolate. GPT-3 Training: Estimated consumption was around 1,287 Megawatt-hours (MWh). GPT-4 Training: Estimates are fuzzy. Some suggest 10-100 MWh (seems low), while others imply it could be 50 times more than GPT-3 due to its size. GPT-3 Training: Estimated consumption was around 1,287 Megawatt-hours (MWh). GPT-3 Training: 1,287 Megawatt-hours (MWh) GPT-4 Training: Estimates are fuzzy. Some suggest 10-100 MWh (seems low), while others imply it could be 50 times more than GPT-3 due to its size. GPT-4 Training: 50 times Given the projected leap in parameters and hardware for GPT-5, we're not talking incremental increases. It's reasonable to expect its training energy consumption to be multiples of GPT-4's, likely running into the thousands of MWh. multiples of GPT-4's thousands of MWh To put 1,000 MWh in perspective: That's roughly the annual electricity consumption of over 100 typical US homes. GPT-5's training could potentially power a small town for a while. To put 1,000 MWh in perspective: That's roughly the annual electricity consumption of over 100 typical US homes. GPT-5's training could potentially power a small town for a while. To put 1,000 MWh in perspective: The Environmental Hangover: Carbon Footprints and Water Guzzling All that energy has to come from somewhere, and unless it's 100% renewable (spoiler: it rarely is), it means carbon emissions. GPT-3 Training: Released an estimated 500-550 metric tons of CO2eq. That's like driving 110-120 gasoline cars for a year or ~300 round trips between New York and San Francisco. GPT-5 Training: With potentially much higher energy needs, expect those CO2 numbers to balloon, potentially into the thousands of tons, depending heavily on the data center's energy mix (nuclear-powered training like BLOOM's showed much lower emissions). GPT-3 Training: Released an estimated 500-550 metric tons of CO2eq. That's like driving 110-120 gasoline cars for a year or ~300 round trips between New York and San Francisco. GPT-3 Training: Released an estimated 500-550 metric tons of CO2eq. That's like driving 110-120 gasoline cars for a year or ~300 round trips between New York and San Francisco. GPT-3 Training: 500-550 metric tons of CO2eq 110-120 gasoline cars for a year ~300 round trips GPT-5 Training: With potentially much higher energy needs, expect those CO2 numbers to balloon, potentially into the thousands of tons, depending heavily on the data center's energy mix (nuclear-powered training like BLOOM's showed much lower emissions). GPT-5 Training: With potentially much higher energy needs, expect those CO2 numbers to balloon, potentially into the thousands of tons, depending heavily on the data center's energy mix (nuclear-powered training like BLOOM's showed much lower emissions). GPT-5 Training: thousands of tons And it's not just carbon. Data centers are thirsty. Cooling those thousands of GPUs requires vast amounts of water. GPT-3 Training: Might have evaporated 700,000 liters (185,000 gallons) of freshwater. GPT-5 Training: Expect that water footprint to rise significantly too, putting strain on local resources, especially in water-scarce areas. GPT-3 Training: Might have evaporated 700,000 liters (185,000 gallons) of freshwater. GPT-3 Training: 700,000 liters (185,000 gallons) GPT-5 Training: Expect that water footprint to rise significantly too, putting strain on local resources, especially in water-scarce areas. GPT-5 Training: Why So Energy-Hungry? The Key Drivers What makes training these models so power-intensive? Model Size (Parameters): More parameters = more calculations = more energy. Simple as that. The trend is clear. Dataset Size: Processing petabytes of text takes time and power. Hardware Efficiency: Newer GPUs like the H100 are faster, but also draw more watts. The sheer number needed is staggering. Training Time: Months of continuous, high-intensity computation adds up. Algorithms & Optimization: Inefficient algorithms waste energy. Hyperparameter tuning (running multiple trial trainings) adds to the bill. Model Size (Parameters): More parameters = more calculations = more energy. Simple as that. The trend is clear. Model Size (Parameters): Dataset Size: Processing petabytes of text takes time and power. Dataset Size: Hardware Efficiency: Newer GPUs like the H100 are faster, but also draw more watts. The sheer number needed is staggering. Hardware Efficiency: Training Time: Months of continuous, high-intensity computation adds up. Training Time: Algorithms & Optimization: Inefficient algorithms waste energy. Hyperparameter tuning (running multiple trial trainings) adds to the bill. Algorithms & Optimization: The Quest for Green AI: Can We Tame the Beast? The good news? The industry is waking up. "Green AI" isn't just a buzzword; it's an active field of research: Algorithmic Efficiency: Smarter training methods (e.g., transfer learning, federated learning). Model Optimization: Techniques like pruning (removing useless parameters), quantization (using less precise numbers), and knowledge distillation (training smaller models to mimic big ones). Efficient Hardware: Designing chips specifically for low-power AI (like TPUs, neuromorphic computing). Honorable Mention - DeepSeek (story for another day on this) Algorithmic Efficiency: Smarter training methods (e.g., transfer learning, federated learning). Algorithmic Efficiency: Model Optimization: Techniques like pruning (removing useless parameters), quantization (using less precise numbers), and knowledge distillation (training smaller models to mimic big ones). Model Optimization: Efficient Hardware: Designing chips specifically for low-power AI (like TPUs, neuromorphic computing). Efficient Hardware: Honorable Mention - DeepSeek (story for another day on this) Honorable Mention Honorable Mention - DeepSeek DeepSeek These approaches aim to get more bang for our energy buck, achieving similar performance with less computational overhead. The Data Center Dilemma: Powering the Future Responsibly? Ultimately, where the electricity comes from matters a lot. GPT-5 is likely trained in Microsoft Azure data centers. Their energy mix directly impacts the carbon footprint. a lot The Trend: Big tech (Google, Microsoft) are investing heavily in renewables (solar, wind) and exploring nuclear/geothermal. The Challenge: The sheer growth in demand is immense, potentially straining grids faster than renewables can be deployed. We need massive infrastructure investment. The Trend: Big tech (Google, Microsoft) are investing heavily in renewables (solar, wind) and exploring nuclear/geothermal. The Trend: are The Challenge: The sheer growth in demand is immense, potentially straining grids faster than renewables can be deployed. We need massive infrastructure investment. The Challenge: growth The Road to Greener AI The Road to Greener AI The industry isn’t blind to the problem. Here’s how we can mitigate the damage: Renewable-Powered Data Centers Google and Microsoft aim to run data centers on 100% renewables by 2030. Training GPT-5 in a solar/wind-powered facility could slash emissions by 90%. Efficient Algorithms Techniques like sparse training, quantization, and distillation reduce compute needs. For example, DeepMind’s Chinchilla model outperformed GPT-3 with 4x less data. Hardware Innovation TPU v4 and NVIDIA’s H100 GPUs offer better performance-per-watt. Specialized AI chips could cut training time and energy. Policy & Transparency Mandate carbon reporting for AI projects (e.g., France’s proposed AI emissions law). Renewable-Powered Data Centers Google and Microsoft aim to run data centers on 100% renewables by 2030. Training GPT-5 in a solar/wind-powered facility could slash emissions by 90%. Renewable-Powered Data Centers Renewable-Powered Data Centers Google and Microsoft aim to run data centers on 100% renewables by 2030. Training GPT-5 in a solar/wind-powered facility could slash emissions by 90%. Google and Microsoft aim to run data centers on 100% renewables by 2030. Training GPT-5 in a solar/wind-powered facility could slash emissions by 90%. Efficient Algorithms Techniques like sparse training, quantization, and distillation reduce compute needs. For example, DeepMind’s Chinchilla model outperformed GPT-3 with 4x less data. Efficient Algorithms Efficient Algorithms Techniques like sparse training, quantization, and distillation reduce compute needs. For example, DeepMind’s Chinchilla model outperformed GPT-3 with 4x less data. Techniques like sparse training, quantization, and distillation reduce compute needs. For example, DeepMind’s Chinchilla model outperformed GPT-3 with 4x less data. Hardware Innovation TPU v4 and NVIDIA’s H100 GPUs offer better performance-per-watt. Specialized AI chips could cut training time and energy. Hardware Innovation Hardware Innovation TPU v4 and NVIDIA’s H100 GPUs offer better performance-per-watt. Specialized AI chips could cut training time and energy. TPU v4 and NVIDIA’s H100 GPUs offer better performance-per-watt. Specialized AI chips could cut training time and energy. Policy & Transparency Mandate carbon reporting for AI projects (e.g., France’s proposed AI emissions law). Policy & Transparency Policy & Transparency Mandate carbon reporting for AI projects (e.g., France’s proposed AI emissions law). Mandate carbon reporting for AI projects (e.g., France’s proposed AI emissions law). Written with a M1 MacBook Air (because every watt counts 🫠 (may be not tokens this time)) Written with a M1 MacBook Air (because every watt counts (may be not tokens this time))
