Software Lets Us Make Less GPU, But Use More Understanding how the things that we buy and use affect our environment is not a simple exercise. Unless you're foraging for food and making tools by hand, everything you consume has a complex environmental impact that's probably larger than you think. For example, that little piece of sashimi you're eating in SOHO might have been caught thousands of miles out in the Pacific and brought back to Japan (the boat burning fuel both ways), flown on an airplane (burning fuel), and delivered to the restaurant in a truck (burning fuel). Everyone involved - fisherman, airline crew, truck driver, chef, wait staff - had to get to and from work. And if you're eating an apex predator like a tuna, there's an additional set of downstream ecosystem impacts. Computer chips are no different. In fact, manufacturing at such a staggering level of microscopic precision comes with a surprisingly large investment of energy and other resources, plus other ecological impacts. And beyond fabrication, chips must be distributed, installed, and ultimately powered inside computers - all activities that impact the environment. As the juggernauts of the Metaverse, Machine Learning, Artificial Intelligence, and Gaming push forward, GPU acceleration will become even more critical and widespread - with corresponding environmental pressures mounting. The Impact of GPU Manufacturing The environmental impacts of modern chip fabrication, or "fab,” can be categorized like so : (note: I don't mean to pick on TSMC in particular, but they are the largest chip manufacturer) a full accounting of the environmental impact of a massive new factory is beyond this article, but - you can imagine the scope of the environmental impact of such a construction project. Construction of fab facilities: the projected cost of one modern TSMC fab is close to $20B Leading manufacturers like TSMC are , but the majority of their electricity still comes from traditional fossil fuel power plants. And these chip fabs use a shocking (yes, I know…) amount of power - TSMC uses about 15TWh annually, Carbon footprint of powering those facilities: making strides toward renewable energy more than the entire country of Paraguay. , Other (non-CO2) greenhouse gases: According to the EPA "Semiconductor manufacturing processes use high [global-warming-potential] fluorinated compounds including perfluorocarbons (e.g., CF4, C2F6, C3F8 and c-C4F8), hydrofluorocarbons (CHF3, CH3F and CH2F2), nitrogen trifluoride (NF3) and sulfur hexafluoride (SF6)…[up to] 80% percent of the fluorinated GHGs pass through the manufacturing tool chambers unreacted and are released into the air." TSMC uses 150,000 tons of water per day in Taiwan alone - the equivalent of 500,000 household residents. This isn't necessarily problematic in a monsoon country but could continue to be an issue . Water usage: until a drought comes in places like Arizona The fab process comprises hundreds of carefully orchestrated physical and chemical steps. Most of the chemicals used are protected as trade secrets and . The ones we do know about include a vast array of carcinogenic, mutagenic, and teratogenic metals, solvents, and polymers. In addition to the health impacts on workers inside, the resulting toxic waste ultimately leaves factories one way or another, all too often ending up in soil and groundwater. Toxic chemicals: do not even need to be disclosed The 7.8B people on Earth average of e-waste per year. Only , leaving the majority of chip manufacturing to be driven by new materials. Mining the metals required for chip production scars landscapes and leaves toxic chemicals used in the refining process in soil, groundwater, and surface water. . Mining of precious metals: 13 pounds each 20% of this is recycled Demand for metals in electronics is only rising, and these impacts are increasing Most of this impact is attributable to CPUs rather than discrete GPUs. But despite their smaller numbers, GPUs are much larger and heavier, thus carrying a higher per-unit impact - and with , the impact is sizeable. 41M discrete GPUs shipped in 2020 Other Environmental Impacts of GPUs New GPUs must, of course, be delivered around the world by ships, planes, and trucks. Distribution: Even if all you need is acceleration capacity, you can't just plug a GPU into your network - GPUs need to run inside computers. In the data center, that means " " whole servers to house the GPUs, with corresponding hardware that comes with its own environmental impact. Installation: racking and stacking GPUs draw a lot of power - and they run hot, so even more power is needed to cool them to operational temperature limits. They also consume a surprising amount of electricity when idle but "on" (ready for a workload). Overall the data centers they run in require constant cooling and other environmental controls that . Operation: consume as much power as the servers themselves do Water, power, fossil fuels, greenhouse gases, toxic chemicals - GPU manufacturing, delivery, and operation come with the whole suite of environmental impacts. And it's ironic, for example, that is also a substantial contributor to climate change. the massive computing capacity that goes into things like climate simulation Imagine instead that we could do way more with way fewer GPUs. It might sound too good to be true. Make Less, Use More GPU utilization today is woefully low, averaging less than 15%, as we've covered previously. Such dire underutilization is, without question, a colossal  waste of resources - but it's also needlessly destructive to the environment. If we can figure out a way to raise utilization to, say, 90%, we would be able to do with the GPUs we have, do the same amount of computing with 1/6 the number of GPUs, or some balance on a spectrum in between. The amount of leverage we could create would be incredible. six times the computing Let's imagine two futures five years from now where the world requires 10x the accelerated computing capacity it has today: Scenario 1: Status Quo (15% Utilization) Let's predict a ~20% annual increase in computing capacity per GPU, based on the recent slowing of Moore's Law. Over five years, this gives us 2.5x of our 10x (1.2⁵=2.49). We still need 4x on top of that 2.5x to get our 10x. So we will need 4x as many GPU cards, with a corresponding 4x in nasty environmental impacts. Let's be generous and say that innovation toward cleaner and more efficient manufacturing, mining, delivery systems, etc., reduce these impacts by 10% (a 1.1x effect). 10x compute increase / 2.5x "Moore's Law" / 1.1x cleantech innovation → 3.6x In Scenario 1, we're still left with a to get our 10x compute capacity. ~3.6x environmental impact Scenario 2: Breakthrough (90% Utilization) "Moore's Law": , as above 2.5x Innovations to reduce environmental impacts: , as above 1.1x The breakthrough innovation raises utilization from 15% to 90%: 6x That 6x can't be applied completely to all environmental impacts because raising utilization does raise the energy required for operation - but we do know that . Let's say 1/3 of the environmental impact comes from operation, so our 6x utilization drives a 2x effect in the opposite direction - most of the environmental impact of computing comes from manufacturing 0.5x 10x compute increase / 2.5x "Moore's Law" / 1.1x cleantech innovation / 6x utilization / 0.5x operational impact → 1.2x 1.2x! In Scenario 2, incredibly, we can meet this 10x accelerated computing future while . barely increasing our environmental impact Sounds great - but how? This might sound too good to be true, but we don't need to bend the laws of physics to achieve this stunning 8x result (10x compute with 1.2x environmental impact) - we just need to unleash an additional 75% utilization in our GPUs . that is already there This is not a trivial undertaking. If it were, someone would have already done it. But . the breakthrough innovation has arrived By abstracting the PCIe connector between a GPU and its application host with software, the two fundamental changes  to GPU usage - both necessary to drive the 6x increase in utilization - are now possible: Removing the physical connection between the application host and GPU enables wider and more efficient pooling of resources and lets any host dynamically attach and detach remote GPU. Accessing a GPU remotely over a network. Much more efficient than today's limited splitting options that just create smaller fixed partitions, which themselves are underutilized, general-purpose software abstraction enables a new sharing paradigm where multiple clients (consumers of GPU) can keep a single GPU working harder. Dynamically sharing a GPU. We've already , deployed our solution with customers, and . outlined how our team has done this envisioned a world where remote GPU is widely adopted As we move into an uncertain environmental future, the last thing we need is rampant waste. We hope you'll join us in our mission to bring the world to 10x compute capacity with only a marginal increase in environmental impact - driven by the simple idea of doing more with what we already have. Steve Golik is co-founder of , a startup with a vision to make computing power flow as easily as electricity. Juice Labs

Flow

YouTube

2022 - HackerNoon Contributor of the Year - Cloud Computing

Nominated for 2022 - HackerNoon Contributor of the Year - Cloud Computing

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

Tackling Environmental Issues with Software: How Remote GPU Reduces the Impact of GPUs

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Coin Mentioned

Juice Labs

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

Tackling Environmental Issues with Software: How Remote GPU Reduces the Impact of GPUs

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Coin Mentioned

Juice Labs

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES