Computing is at an inflection point. Moore’s Law predicts that the number of transistors on an electronic chip will double every year, but its pace is slowing due to physical limitations in fitting more transistors onto affordable microchips. Growth in computer power is slowing as demand grows for high-performance computers capable of supporting increasingly complex artificial intelligence models. This inconvenience has prompted engineers to explore new ways to expand the machine’s computing power, but the solution remains unclear.
Photonic computing is a potential remedy for meeting the growing computational demands of machine learning models. Instead of using transistors and wires, these systems utilize photons (microscopic light particles) to perform computational operations in the analog domain. Lasers create these small beams of energy that move at the speed of light, like spaceships traveling at super speed in science fiction movies. When photonic computing cores are added to programmable accelerators such as network interface cards (NICs and their enhancements, SmartNICs), the resulting hardware can be plugged in to turbocharge standard computers.
MIT researchers have now harnessed the potential of photonics to speed up modern computing by demonstrating its power in machine learning. Their photonic-electronic reconfigurable SmartNIC, called “Lightning,” helps deep neural networks (machine learning models that mimic how the brain processes information) with reasoning tasks such as image recognition and language generation in chatbots like ChatGPT. The prototype’s novel design achieves impressive speeds, creating the first photonic computing system to serve real-time machine learning inference requests.
Despite their potential, a major challenge in realizing photonic computing devices is that they are passive, meaning that unlike electronic devices, they lack the memory or instructions to control the flow of data. Previous photonic computing systems faced this bottleneck, but Lightning removes this obstacle, ensuring smooth movement of data between electronic and photonic components.
“Photonic computing shows significant advantages in accelerating huge linear computing tasks such as matrix multiplication, while it requires electronics to handle the rest: memory access, nonlinear calculations and conditional logic. This generates huge amounts of data in photonics and electronics to complete real-world computing tasks, such as machine learning inference requests.” said Zhong Zhizhen, a postdoc in the group of Manya Ghobadi, associate professor in the MIT Department of Computer Science and Artificial Intelligence Laboratory (CSAIL). “Controlling the flow of data between photonics and electronics has been the Achilles’ heel of past state-of-the-art photonic computing efforts. Even if you have an ultrafast photonic computer, you need enough data to power it without stalling. Otherwise, your supercomputer will sit idle and not do any sensible calculations.”
Ghobadi, an associate professor in MIT’s Department of Electrical Engineering and Computer Science (EECS) and a member of CSAIL, and her colleagues were the first to identify and solve this problem. To achieve this feat, they combined the speed of photonics with the data flow control capabilities of electronic computers.
Before the Lightning Network, photonic and electronic computing schemes operated independently, using different languages. The team’s hybrid system tracks required computational operations along the data path using a reconfigurable counting action abstraction that connects photonics to the computer’s electronic components. This programming abstraction acts as a unifying language between the two, controlling access to the passing data flow. The information carried by electrons is converted into light in the form of photons, which work at the speed of light to assist in reasoning tasks. The photons are then converted back into electrons, delivering the information to the computer.
By seamlessly bridging photonics and electronics, the novel counting action abstraction enables Lightning Network’s fast real-time computing frequencies. Previous attempts have used a stop-and-go approach, meaning the data will be hampered by much slower control software that makes all decisions about its movement. “Building a photonic computing system without the counting-action programming abstraction is like trying to drive a Lamborghini without knowing how to drive,” said Ghobadi, the paper’s senior author. “What would you do? You might have a driver’s manual in one hand, then press the clutch, then check the manual, then release the brake, then check the manual, etc. It’s a stop-and-go operation because For every decision, you have to consult some higher-level entity to tell you what to do. But that’s not how we drive; The manual or driving rules behind it. Our counting action programming abstraction acts as muscle memory in lightning. It seamlessly drives electrons and photons in the system at runtime.”
Environmentally friendly solutions
Machine learning services that complete inference-based tasks, such as ChatGPT and BERT, currently require significant computing resources.Not only are they expensive – some estimates suggest ChatGPT requires US$3 million run once a month – but they’re also bad for the environment, potentially emitting more than twice the carbon dioxide of the average person.Lightning uses photons that move faster than electrons in wires, while creating less caloriesenabling it to perform calculations at a faster frequency while being more energy efficient.
To measure this, the Ghobadi team compared their device to standard graphics processing units, data processing units, SmartNICs and other accelerators using synthetic Lightning chips. The team observed that the Lightning Network was more energy efficient when completing inference requests. “Our synthesis and simulation studies show that Lightning Network reduces machine learning inference power consumption by orders of magnitude compared to state-of-the-art accelerators,” said Mingran Yang, a graduate student in Ghobadi’s lab and co-author of the paper. As a more cost-effective and faster option, Lightning Network offers a potential upgrade for data centers to reduce the carbon footprint of machine learning models while accelerating inference response times for users.
Other authors of the paper include MIT CSAIL postdoc Homa Esfahanizadeh and undergraduate student Liam Kronman, as well as MIT EECS associate professor Dirk Englund and three recent graduates of the department: Jay Lang ’22, MEng ’23; Christian · Williams ’22, MEng ’23; and Alexander Sludds ’18, MEng ’19, PhD ’23. Their research was supported in part by the DARPA FastNICs program, the ARPA-E ENLITENED program, the DAF-MIT Artificial Intelligence Accelerator, the U.S. Army Research Office through the Soldier Nanotechnology Institute, a National Science Foundation (NSF) grant, the NSF Quantum Networks Center, and Sloan scholarship.
The group will present their findings this month at the Society for Computing Machinery’s Special Interest Group on Data Communications (SIGCOMM).