Energy, Cooling, and Natural Resources in AI
Energy Consumption in Training
Estimates for Large Models
| Model | Company | Estimated Energy | CO₂ Equivalent |
|---|---|---|---|
| GPT-3 (175B) | OpenAI | 1,287 MWh | 552 tonnes |
| GPT-4 | OpenAI | ~50 GWh (estimated) | ~25,000 tonnes |
| Llama 3 70B | Meta | ~530 MWh (training) | N/A |
| Llama 3 405B | Meta | ~7,700 MWh | ~3,500 tonnes |
| DeepSeek-R1 | DeepSeek | ~6,000 MWh (estimated) | ~2,700 tonnes |
| Grok-3 | xAI | ~500+ MWh | N/A |
Methodology: Energy = Total_power × time × (1/efficiency)
- H100: 700W × 1000 GPUs × 30 days = 504 MWh in GPU alone
- Plus cooling, network, storage: multiply by PUE (~1.3–1.5)
Energy Consumption in Inference
Per Token Generated (estimate)
| Model | Hardware | Energy/token |
|---|---|---|
| GPT-4 | H100 cluster | ~0.001–0.003 Wh |
| Llama 3 8B | RTX 4090 | ~0.0002 Wh |
| Llama 3 70B | 2× H100 | ~0.0015 Wh |
Global ChatGPT: Estimates point to ~1 GWh/day in 2024 (all queries)
IEA — Global Energy Projection for AI
| Year | Estimated consumption | Comparison |
|---|---|---|
| 2023 | 180 TWh | All of Sweden |
| 2025 | 485 TWh | All of the United Kingdom |
| 2027 | 700 TWh | All of Germany |
| 2030 | 950 TWh | All of Japan |
Source: IEA "Electricity 2024" report; confirmed by Goldman Sachs projections (700 TWh by 2030)
PUE — Power Usage Effectiveness
Formula: PUE = Total data center energy / Server energy
- PUE = 1.0 → perfection (impossible)
- PUE = 2.0 → half the energy goes to cooling/other (bad)
- Current target: PUE < 1.3
| Company | Declared PUE (2024) |
|---|---|
| Google (cloud avg) | 1.10 |
| Microsoft Azure | 1.12 |
| Meta (AI DCs) | 1.08 |
| AWS | 1.15 |
| Typical data centers | 1.5–1.8 |
Cooling Systems
Air Cooling
- Standard until 2023: 1U/2U servers with heatsinks and fans
- Limit: ~400–500W per GPU (RTX 4090, A100 PCIe)
- Cost: Cheaper; no risk of leaks
- Inadequate for: H100 SXM (700W), B200 (1,000W), GB200 rack (120 kW)
Direct Liquid Cooling (DLC)
- Mechanism: Tubes with water/glycol pass through the chip's heatsink
- Capacity: Dissipates up to 1,000W per chip
- Implementation: H100 SXM uses DLC; GB200 uses mandatory DLC
- CDU (Coolant Distribution Unit): Module in the rack that distributes the liquid
- Target temperature: Coolant inlet < 35°C, outlet < 50°C
Immersion Cooling
Single-phase: Servers submerged in an inert dielectric fluid (3M Novec or similar)
- The fluid does not conduct electricity; the GPU is submerged directly
- Dissipation: 100 kWm² vs 30 kWm² in air
- Cost: High upfront; complex maintenance
Two-phase: The fluid evaporates on contact with the chip → condenses on the tank's ceiling
- Extreme efficiency; PUE < 1.05 possible
- Use: Research; high-density clusters
Rear-Door Heat Exchangers (RDHx)
- Heat exchanger on the rack's rear door
- Removes heat from the hot air before it exits into the room
- An increment over air cooling; no changes to the server
- Effective for racks up to 40 kW
Water Usage
WUE — Water Usage Effectiveness
WUE = Volume of water evaporated / Server energy
- Air cooling with a cooling tower evaporates water
- A 100 MW data center can evaporate 2–5 million liters of water per day
| Company | WUE (2024) | Method |
|---|---|---|
| 0.96 L/kWh | Reuse in cold regions | |
| Microsoft | 0.3 L/kWh | Closed-loop DLC in new DCs |
| Amazon | 1.8 L/kWh | Evaporation in towers |
Local impact: Concerns in regions with water scarcity (Arizona, Nevada, Netherlands)
Energy Sources
| Company | Renewable Energy Commitment |
|---|---|
| 100% renewable since 2017 (credit purchases) | |
| Microsoft | 100% renewable by 2025 (PPA contracts) |
| Meta | 100% renewable; carbon negative by 2030 |
| Amazon/AWS | 85% renewable in 2024; 100% target 2025 |
| xAI (Grok) | Memphis, TN — coal/gas mix; controversial |
Problem: "100% renewable" often means carbon credits (RECs), not literal 24/7 renewable energy.
Carbon Intensity by Region:
- France: 56 gCO₂/kWh (nuclear)
- Iceland: 0 gCO₂kWh (geothermalhydro)
- USA (average): 386 gCO₂/kWh
- Poland: 713 gCO₂/kWh (coal)
Natural Resources — Critical Minerals in GPUs
Materials in an H100 GPU
| Material | Use | Main Origin | Risk |
|---|---|---|---|
| Copper | Interconnects, PCB | Chile, Peru | Low |
| Silicon | Base chip | Global quartz sand | Low |
| Tungsten | Vias (TSV) | China (85%), Russia | High |
| Cobalt | Capacitors, backup battery | Congo DRC (70%) | HIGH — conflict |
| Lithium | Batteries in the DC | Chile, Australia | Medium |
| Tantalum | Capacitors | Congo DRC, Rwanda | HIGH — conflict |
| Indium | Displays, solders | China (60%) | Medium |
| Gallium | GaAs compounds | China (95%) | CRITICAL (exports restricted since 2023) |
| Germanium | Optical fiber, chips | China (60%) | CRITICAL (exports restricted) |
| Neodymium | Magnets (cooling fans) | China (70%) | High |
| Dysprosium | High-temp magnets | China (monopoly) | CRITICAL |
| Terbium | Magnets, lasers | China (monopoly) | CRITICAL |
Rare Earth Elements (REEs)
- 17 elements (lanthanum, cerium, neodymium, etc.)
- China controls ~60% of production; ~85% of processing
- Use: Permanent magnets (fans, motors), lasers, phosphors
- China restricted exports in 2023 → impact on the supply chain
Conflict Minerals in the Congo (DRC)
- Congo: 70% of the world's cobalt, significant tantalum
- Artisanal mines (ASM): 20% of production with documented child labor
- Certifications: RMI (Responsible Minerals Initiative), EICC
- NVIDIA, AMD, Intel have due diligence policies, but traceability is difficult
- Alternatives under research: Cobalt-free batteries (LiFePO4); alternative MLCC capacitors
Electronic Waste (E-waste)
- Lifespan of a data center GPU: 3–5 years
- Global e-waste (2023): 53 million metric tonnes/year
- Actual recycling: Only ~17% is formally recycled
- Pressure: EU WEEE regulation (Waste Electrical and Electronic Equipment)
- Initiatives: Google refurbishment; Dell Asset Recovery; AWS Refurb program
Sustainability in AI Development
Efficiency Metrics
- FLOPS/Watt: Blackwell B200 ≈ 3× more efficient than H100 per inference FLOPS
- Tokens/Watt: Inference of Llama 3 8B on M2 Ultra ≈ 10× more efficient than H100 cloud
- Compute-optimal training: Chinchilla (20 tokens/parameter) avoids wasted compute
Smaller Models with the Same Quality
- Phi-4 (14B): Performance of 70B models in STEM — 5× less energy
- Qwen2.5-Coder-7B: Surpasses 13B+ models from 2023 — 2× less energy per query
- Distillation: DeepSeek-R1-7B has 70% of the 70B's capability — 10× less energy
Green AI
- Foundational paper: "Energy and Policy Considerations for Deep Learning in NLP" (Strubell et al., 2019)
- MMCE (Marginal Carbon-aware Model Evaluation): Benchmark accounting for energy cost
- Initiative: MLCommons Power Track; Energy Star for AI servers