“Says "medium" is 1,000-token. If I assume that takes 50 seconds, 40 Wh / 50 s = 2880 watts Seems a bit much? Even if I assume 1/3rd is the amortised result of the training runs and 1/6th is the cooling system, that only takes me down to the power draw of 2.4 times the H200s' 600W maximum TDP* on the mean inference run. Semi-related to main theme: why ar…”
How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference
This paper introduces an infrastructure-aware benchmarking framework for quantifying the environmental footprint of LLM inference across 30 state-of-the-art models in commercial datacenters. The framework combines public API performance data with company-specific environmental multipliers and statistical inference of hardware configurations. We additionally utilize cross-efficiency Data Envelopment Analysis (DEA) to rank models by performance relative to environmental cost and provide a dynamically updated dashboard that visualizes model-level energy, water, and carbon metrics. Results show the most energy-intensive models exceed 29 Wh per long prompt, over 65 times the most efficient systems. Even a 0.42 Wh short query, when scaled to 700M queries/day, aggregates to annual electricity comparable to 35{,}000 U.S. homes, evaporative freshwater equal to the annual drinking needs of 1.2M people, and carbon emissions requiring a Chicago-sized forest to offset. These findings highlight a growing paradox: as AI becomes cheaper and faster, global adoption drives disproportionate resource consumption. Our methodology offers a standardized, empirically grounded basis for sustainability benchmarking and accountability in AI deployment.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“One random preprint I found (https://arxiv.org/html/2505.09598v2) estimates inference is 90% of the total energy usage. From some googling around, everything seems to agree that inference dominates (or is at least comparable to) training on the large commonly-used models. I was less surprised that inference dominates training after I read that chatgpt is…”
“Page 38 has an interesting comparison saying a session of 8 "medium length" messages uses roughly 10Wh, the equivalent of ~2 phone charges. The data dashboard[1] I think is defining medium length as 1000 tokens in, 1000 tokens out. And the page before suggests GPT-5 mini (medium) used 3x the energy of GPT-4o (Aug) which I guess I assumed would be the oppo…”
- How Hungry Is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Infere ↗raybb · 2025-08-08 · 1 分