A New Photonic Computer Chip Uses Light to Slash AI Energy Costs

AI models are power hogs.

As the algorithms grow and become more complex, they’re increasingly taxing current computer chips. Multiple companies have designed chips tailored to AI to reduce power draw. But they’re all based on one fundamental rule—they use electricity.

This month, a team from Tsinghua University in China switched up the recipe. They built a neural network chip that uses light rather than electricity to run AI tasks at a fraction of the energy cost of NVIDIA’s H100, a state-of-the-art chip used to train and run AI models.

Called Taichi, the chip combines two types of light-based processing into its internal structure. Compared to previous optical chips, Taichi is far more accurate for relatively simple tasks such as recognizing hand-written numbers or other images. Unlike its predecessors, the chip can generate content too. It can make basic images in a style based on the Dutch artist Vincent van Gogh, for example, or classical musical numbers inspired by Johann Sebastian Bach.

Part of Taichi’s efficiency is due to its structure. The chip is made of multiple components called chiplets. Similar to the brain’s organization, each chiplet performs its own calculations in parallel, the results of which are then integrated with the others to reach a solution.

Faced with a challenging problem of separating images over 1,000 categories, Taichi was successful nearly 92 percent of the time, matching current chip performance, but slashing energy consumption over a thousand-fold.

For AI, “the trend of dealing with more advanced tasks [is] irreversible,” wrote the authors. “Taichi paves the way for large-scale photonic [light-based] computing,” leading to more flexible AI with lower energy costs.

Chip on the Shoulder

Today’s computer chips don’t mesh well with AI.

Part of the problem is structural. Processing and memory on traditional chips are physically separated. Shuttling data between them takes up enormous amounts of energy and time.

While efficient for solving relatively simple problems, the setup is incredibly power hungry when it comes to complex AI, like the large language models powering ChatGPT.

The main problem is how computer chips are built. Each calculation relies on transistors, which switch on or off to represent the 0s and 1s used in calculations. Engineers have dramatically shrunk transistors over the decades so they can cram ever more onto chips. But current chip technology is cruising towards a breaking point where we can’t go smaller.

Scientists have long sought to revamp current chips. One strategy inspired by the brain relies on “synapses”—the biological “dock” connecting neurons—that compute and store information at the same location. These brain-inspired, or neuromorphic, chips slash energy consumption and speed up calculations. But like current chips, they rely on electricity.

Another idea is to use a different computing mechanism altogether: light. “Photonic computing” is “attracting ever-growing attention,” wrote the authors. Rather than using electricity, it may be possible to hijack light particles to power AI at the speed of light.

Let There Be Light

Compared to electricity-based chips, light uses far less power and can simultaneously tackle multiple calculations. Tapping into these properties, scientists have built optical neural networks that use photons—particles of light—for AI chips, instead of electricity.

These chips can work two ways. In one, chips scatter light signals into engineered channels that eventually combine the rays to solve a problem. Called diffraction, these optical neural networks pack artificial neurons closely together and minimize energy costs. But they can’t be easily changed, meaning they can only work on a single, simple problem.

A different setup depends on another property of light called interference. Like ocean waves, light waves combine and cancel each other out. When inside micro-tunnels on a chip, they can collide to boost or inhibit each other—these interference patterns can be used for calculations. Chips based on interference can be easily reconfigured using a device called an interferometer. Problem is, they’re physically bulky and consume tons of energy.

Then there’s the problem of accuracy. Even in the sculpted channels often used for interference experiments, light bounces and scatters, making calculations unreliable. For a single optical neural network, the errors are tolerable. But with larger optical networks and more sophisticated problems, noise rises exponentially and becomes untenable.

This is why light-based neural networks can’t be easily scaled up. So far, they’ve only been able to solve basic tasks, such as recognizing numbers or vowels.

“Magnifying the scale of existing architectures would not proportionally improve the performances,” wrote the team.

Double Trouble

The new AI, Taichi, combined the two traits to push optical neural networks towards real-world use.

Rather than configuring a single neural network, the team used a chiplet method, which delegated different parts of a task to multiple functional blocks. Each block had its own strengths: One was set up to analyze diffraction, which could compress large amounts of data in a short period of time. Another block was embedded with interferometers to provide interference, allowing the chip to be easily reconfigured between tasks.

Compared to deep learning, Taichi took a “shallow” approach whereby the task is spread across multiple chiplets.

With standard deep learning structures, errors tend to accumulate over layers and time. This setup nips problems that come from sequential processing in the bud. When faced with a problem, Taichi distributes the workload across multiple independent clusters, making it easier to tackle larger problems with minimal errors.

The strategy paid off.

Taichi has the computational capacity of 4,256 total artificial neurons, with nearly 14 million parameters mimicking the brain connections that encode learning and memory. When sorting images into 1,000 categories, the photonic chip was nearly 92 percent accurate, comparable to “currently popular electronic neural networks,” wrote the team.

The chip also excelled in other standard AI image-recognition tests, such as identifying hand-written characters from different alphabets.

As a final test, the team challenged the photonic AI to grasp and recreate content in the style of different artists and musicians. When trained with Bach’s repertoire, the AI eventually learned the pitch and overall style of the musician. Similarly, images from van Gogh or Edvard Munch—the artist behind the famous painting, The Scream—fed into the AI allowed it to generate images in a similar style, although many looked like a toddler’s recreation.

Optical neural networks still have much further to go. But if used broadly, they could be a more energy-efficient alternative to current AI systems. Taichi is over 100 times more energy efficient than previous iterations. But the chip still requires lasers for power and data transfer units, which are hard to condense.

Next, the team is hoping to integrate readily available mini lasers and other components into a single, cohesive photonic chip. Meanwhile, they hope Taichi will “accelerate the development of more powerful optical solutions” that could eventually lead to “a new era” of powerful and energy-efficient AI.

Image Credit: spainter_vfx / Shutterstock.com