Phil Burr, Director, Lumai

There has been a lot of discussion around data centers’ huge energy consumption and environmental impact. But the surge in power demand is also generating a significant increase in costs, which is subsequently making it harder for companies to capitalize on AI. Goldman Sachs has predicted the tech sector is on course to reach the $1 trillion mark for spend on AI data centers and hardware.

This expenditure is simultaneously accelerating innovation and producing a power strain at both data center and rack level. But without major cuts to both capital expenditure per processor and energy consumption per each AI calculation, the total cost of ownership (TCO) of data centers will increase – and they’ll not be able to improve their sustainability while matching energy demand.

However, a groundbreaking approach using 3D optics for AI acceleration is maximizing the performance and energy efficiency of data centers. Here are three ways this new technology can help reduce the TCO of data centers.

1.Improves the scalability of existing data centers

McKinsey has predicted that overall global demand for data center capacity “could rise at an annual rate of between 19 and 22 per cent from 2023 to 2030”. And AI is the driving force behind this increase. New methods are required to perform AI computation and reduce power for AI processing, meaning processing can take place within the power capabilities of these data centers. One way to do this is to integrate technology like optical processing.

Currently, data centers use silicon chip-based AI accelerators. Not only are these power hungry, but they can’t efficiently scale and generate the necessary capacity required (within reasonable power limits) for AI’s surging compute demand. An optical AI accelerator, on the other hand, can provide low power and energy efficiency for computation, similar to the benefits seen in optical communications. This produces a more scalable method which means the growing demands of AI computation can still be matched.

2. Enables more efficient AI accelerators

AI is putting the energy consumption of servers in data center racks under massive strain. The McKinsey study also revealed “average power densities have more than doubled in just two years, to 17 kilowatts (kW) per rack… and are expected to rise to as high as 30 kW by 2027 as AI workloads increase”.

Every Watt of power used requires more cooling, more energy, more infrastructure and consequently more generated emissions. All of these energy and infrastructure add-ons significantly increase the overall TCO.

Optical AI acceleration performs highly parallel computing and works by using photons instead of electrons to compute. Therefore, not only can this provide the necessary leap in performance, but crucially it is much more efficient. Optical acceleration uses only 10 percent of the power compared to GPUs currently found in data centers.

By using optical computation to create more efficient AI accelerators, data centers can increase their own lifespan while reducing the need for new buildings, substantially reducing TCO.

3. Provides an alternative to costly silicon tech

One of the core focus areas in the sector is maximising performance in current AI accelerator products. But the current strategy is to add more silicon area, power and cost – a process chasing diminishing returns. Recently, Nvidia revealed its new Blackwell GPU would cost around $30,000.

Optical processors can eliminate the need to use costly new silicon by leveraging existing optical and electronic technology in data centers instead. Therefore, if we add these cost savings to savings from using less cooling infrastructure and power, the TCO becomes a fraction of a GPU.

Reducing TCO can drive the industry ahead

The current trajectory of the TCO of data centers is far too high for both AI development and their environmental footprint. But creating a cost-efficient approach can also create a sustainable one. The use of new technology like optical AI acceleration can significantly lower the direct costs and energy consumption of AI data centers while also improving their performance, scalability and lifespan.

Above all, using such technology can reduce the TCO of data centers and thereby build a sustainable model to drive the industry ahead.