What IS Physical AI, Anyway?

We’re a month out from Automate now, but one of the most striking details about the show was how far this new buzzword, “physical AI,” had spread. Nvidia only just coined the term a year and a half ago, but a quarter of the show is already selling it.

That can understandably cause a bit of whiplash. It’s a new cottage industry that has spawned incredibly quickly in a larger field that is already both extremely fluid and poorly understood. It’s grown organically with different companies individually taking up the buzzword to describe their products. It’s not a terribly intuitive term, either.

All of this, of course, means that when you try to pin down what physical AI is, no two companies will give you the same explanation. And yet, everybody from software companies to component manufacturers seem to be selling it.

Which really begs the question…

What IS Physical AI?

Physical AI is best understood as a statement of intent. It’s an umbrella term that captures numerous technologies and disciplines all dedicated to giving robots the ability to understand and interact with the physical world. It’s not new technology in and of itself — it’s a genre.

This makes physical AI’s definition a little different depending on who you ask, but most vendors give me some variation on this core three-step explanation:

1. All the equipment and sensors designed to gather sensory data the same way our five senses do. Vision systems, haptic feedback sensors, LiDAR, and so on.
2. Computing technology dedicated to translating that feedback into comprehensible, actionable data for a computer.
3. Decision-making software that takes that translated feedback and directs a machine’s response to it. The “AI” part of physical AI.

A stripped down example would look like this: You have a pick and place machine situated on a conveyor belt that red and blue blocks are going down. Step one, you have sensors capable of differentiating color. Step two, you have hardware that processes that sensory information, translates it into something a computer can understand, and software that flags which blocks are red and which are blue. Step three, you have different software that tells the robot “if the object is blue, it goes in the blue bin, and if red, it goes in the red bin.” Through these three steps, you have now taught a pick and place machine how to make simple, real-time decisions based on changing context.

Now extrapolate that concept out to less abstract applications. Use more complicated objects like food. How do you teach that pick and place machine the difference between kale and lettuce? How do you teach it to identify produce that’s going bad and remove it from the line? How do you get it to understand different packaging textures and not freak out when there’s a semi-transparent wrapper binding a six-pack of bottles together?

Apply it to inspection applications. Show a robot on a car manufacturing line what a properly built car door is supposed to look like, then use the AI’s powers of pattern recognition to compare a real car door to that control sample. Are the bolts in the same place? Are they the right size? Properly tightened? And so on.

Then apply it to manufacturing and imagine robots capable of dynamically changing tasks. Instead of programming an arm to do exactly one thing 10,000 times over the course of a day, it can be programmed with a few different jobs and recognize when it should be doing which. Accomplish that, and suddenly automation becomes far more appealing for small batch manufacturing or products that allow for customization.

Robotics is a surprisingly narrow field when you get down to it, often requiring strict and static conditions to excel, and the list of materials or situations it struggles with is long. Physical AI’s ultimate ambition is to loosen those restrictions by striking the exceptions off the list one by one. So, how are they getting there?

Glimpses of the Future

Many vision systems are tuned to work at a specific range at the cost of all other distances. You can get a sensor that is extremely accurate at 10 inches or at 10 feet, but rarely both. Then there are all kinds of other complications: dust, reflective surfaces, and semi-transparent objects can all befuddle some vision systems. If you’re trying to develop a robot capable of performing multiple tasks in an uncontrolled environment, each of these challenges need to be overcome just to make physical AI’s dream possible.

This is what SiLC Technology’s Eyeonic Edge vision system is designed to tackle. Designed specifically for automated inspection application, the Eyeonic Edge is built to be accurate from .25 meters all the way to 10 meters across multiple lighting conditions.

But its true selling point is the ability to precisely track velocity. Once you start developing software intended to make decisions based on changing context, you suddenly need hardware capable of providing that context. Being able to gauge an object’s velocity means that later steps in the process now have the data to accurately predict where that object will be and successfully track it.

This all greatly improves their flexibility in inspection applications. Think being able to analyze products for defects while they’re still moving on a manufacturing line or work with large components in industries like aerospace. Add in a few quality of life features like the ability to meaningfully study and interact with all those troublesome reflective, transparent, or dark surfaces that trip up some vision systems, and SiLC’s Eyeonic Edge unlocks entire new swaths of territory for automated inspection tools.

Another standout is Xela Robotics, who are developing “fingertips” capable of providing a robot with the gift of touch. Through a grid of haptic feedback sensors, these pads provide the sensory data to tell exactly how much pressure a robot is putting on an object. Importantly, they can also detect if the object is slipping or sliding in the robot’s grip by carefully tracking shear forces, something that previous applications of this technology have been challenged by.

As Xela was showing off at their booth, the possibilities this opened up were immediately tangible. A centerpiece display had a robotic hand successfully picking up a paper crane without causing so much as a wrinkle. Gripping such fragile objects is, traditionally, a challenge that has stumped automation because of its strict accuracy and consistency requirements. You need to hit an extremely narrow Goldilocks zone to apply enough force that the object doesn’t slip out of your hand, but not so much that the object is crushed or damaged.

Trying to program that out with traditional methods is a Sisyphean task. The slightest change would ruin the entire process. With sensors like Xela’s uSkin, however, we’re working towards a future where that Goldilocks zone can not only be found, but persistently maintained by adjusting it in real-time.

Importantly, Xela’s sensors aren’t just for humanoid robots. They’re compatible with grippers and your own custom application, too. And the company has also developed fingernails that enable machines to pick up ultra-thin objects like playing cards, further expanding the range of applications they could hypothetically work in. While we’re probably years away from catching up to Xela and perfecting robots capable of taking advantage of these sensors, they offer a promising glimpse into the future of what’s possible if we can get it right.

Building for the Speed of Thought

Sensors tooled towards these applications usually need to prioritize quality above all else. The most common mantra surrounding AI of any variety is that it is only as good as its dataset. Accuracy, clarity, consistency — the foundation that gathers all that data needs to be rock solid on all fronts for anything that comes after to work properly. Without it, you would have robots just as prone to hallucinations and errant, nonsensical decisions as the average ChatGPT output.

Physical AI’s ambitions run into another complication here, however. To provide good enough feedback for a robot to work with, these sensors need to pump a large amount of data downstream. Too much, arguably, for what most existing computing applications are prepared for. The problem gets even worse when you consider how space-constrained robotics applications are. They don’t have the luxury of carrying a datacenter on their back to handle the order of magnitude increases in data traffic.

Take SPAD (single photon avalanche diode) sensors, for example. SPAD sensors are a fundamental part of a lot of high end LiDAR devices, which in turn have become one of the most popular forms of detection across robotics, particularly the field of AMRs. They also specialize in capturing visuals in low-light conditions. When there isn’t enough light to properly expose a photo, SPAD sensors still get a clean picture by, effectively, just working harder and capturing every individual photon.

But if you think a 4k resolution makes your internet chug, try imagining how much data is getting generated once you start measuring in photons. In a robotics application where you want a constant video feed, that data becomes a consistent firehose of information that can grind an average processor to a halt. This is where a company called Ubicept comes in.

Between a pair of products, Ubicept Photon Fusion and FLARE (Flexible Light Acquisition and Representation Engine), the company has built a system for encoding and preprocessing data from vision systems. Before all that data utterly drowns some poor edge computing GPU, it has to pass through Ubicept’s software, which can clean the raw input of noise and reduce its framerate to something that GPU can handle.

Notably, Ubicept is not what one would call a “traditional” physical AI company (inasmuch as anyone could be considered one). They, in fact, stand out by directly comparing themselves against AI video denoisers and don’t really seem to be sipping the same koolaid the rest of the industry is. But there’s a reason they showed up to Automate. The magnitude of data required to accomplish physical AI’s mandate is a real stumbling block that needs to be solved for, and their work pushes us forward towards a solution.

Particularly because you also need to consider the speed at which all this data needs to be processed. A live, uncontrolled environment will not wait for a robot to calculate and make a decision. The raw volume of data is challenging enough, but all of this also needs to happen instantaneously.

To illustrate just how instantaneous we’re talking here, MemryX’s MX3 M.2 AI accelerator will put a lot in perspective. This little chip is, effectively, what’s taking all that sensor data and translating it into something the robot can interpret, and it comes with a pile of impressive specs catered to handle data-heavy applications like AI.

But when a sales rep was explaining the chip to me, the first thing he talked about was how the M.2 was effectively three processing components (CPU, GPU and accelerator) all in one chip. Normally, all three of these components might be completely separate products that have to take time to communicate with each other, but the M.2 skips that step. The benefit of this? Milliseconds shorter response time.

In computing, you may well be familiar with the idea of latency, or the time it takes for a computer to receive a command, then execute it. Outside of heavy-duty applications, this process happens so quickly that it’s basically invisible. But for robotics applications, it’s apparently too slow — noticeably enough that it’s the selling point a sales rep leads with. It turns out that building a processing system that acts as quickly as our own human reflexes (which, for the record, average around 250ms) is a tall order indeed.

Physical AI Is Just a Label, but It Matters

I can see how some might view all of this as hot air. Nvidia invented a new marketing term to sell more chips. So what?

What separates physical AI from simple marketing lingo is the way the industry is transforming around it. A noticeable portion of the automation industry is lurching in this direction, working towards a shared goal from a hundred different angles, and “physical AI” appears to be their rallying flag.

Big mission statements like these have power. Like a lot of genre labels, it is imperfect, does not capture the full nuance of all these different devices and can have a flattening effect on discussions about the topic. But it does successfully capture their aspirations and makes that discussion more comprehensible for anyone that doesn’t spend 40 hours a week having it.

And that is why I purposely choose these specific phrases — statement of intent, genre, mission statement — because it doesn’t just name a type of product, but outlines the entire strategy of a broad swath of the automation market for the next five years.

This is where automation manufacturers want to go next. This is the problem they actively wrestle with at their desks every day. And you’re going to see them talking about it more in the years to come as they try to widen the applications robotics are suitable for.