Fascinating, crazy, maybe a little scary.
I mean, it sounds like we're talking about not just
using something to help humans do their jobs,
but just replacing it entirely.
Could this be a new normal on a on a
large scale?
Hi, welcome to another episode of Cold Fusion.
This scene is from the popular animated American sitcom The
Jetsons.
This episode titled Rosie the Robot first aired on September
23rd, 1962. The main character in the scene is presented
with a few variations of a robot,
one that's responsive to commands.
We'll do housework and basically work as a personal assistant.
61 years on and we're one step closer to seeing
that reality.
This week, a tech startup called Figure launched A humanoid
robot called Figure 01.
You can have conversations with it thanks to the integration
with Open AI.
Now, at this point, you might be thinking,
we've already seen robots like this over the years,
whether it's from Boston Dynamics or Tesla.
How is this any different?
Well, first, let's take a look at the demo of
the robot.
Before we get into the weeds,
keep in mind that this demo,
as claimed by the company,
is unaltered and in real time.
But for the purposes of brevity,
I've shortened some parts. All right,
so let's watch. Great. Can I have something to eat?
Sure thing.
Great.
Can you explain why you did what you just did
while you pick up this trash
on it? So I gave you the apple
because it's the only edible item I could provide you
with from the table.
So based on the scene right now,
where do you think the dishes in front of you
go next?
The dishes on the table like that plate and cup
are likely to go into the drying rack next.
Great. Can you put them there?
Of course.
After seeing how well it did for those tasks and
the human style conversation,
some of you will be impressed,
some excited, and some even a little bit scared.
According to the company, this bot isn't being operated by
anyone.
It's able to carry out full conversations with people and
respond to external stimuli.
And that right there is one of the primary differences
between Figure 01 and the other Android robots we've seen
so far.
But let's back up for a second.
How did a start up only founded in
2022 make such headway? Who is backing Figure and what
are their intentions?
Given what NVIDIA just announced in the past week,
are we moving towards an AI driven robotics race?
And most importantly, how do these advances in robotics and
AI impact all of us?
We'll take a look at all of that and have
a roundup of all the latest robots in this episode.
So sit back, relax into your docking station as we
figure out the future of robotics.
You are watching Cold Fusion TV.
It all started in the wake of COVID.
At this time, robotic sales began surging,
hitting a record high in
2022 as the workforce retreated.
These robots were the kind that we used in warehouses
and assembly lines,
the exact sort of predictable environment that was the lowest
hanging fruit to train and deploy robots.
And
amongst this backdrop comes Figure AI.
Founded in 2022, Figure AI Inc is a
California-based tech startup with a focus on general purpose humanoid
robot.
Within just one year of its inception,
the company raised $70 million,
but now they've recently raised a whopping 675,000,000 and
now have a valuation of
2.6 billion. Officially a Unicorn,
and you know it's serious business when the likes of
Microsoft,
NVIDIA, Jeff Bezos, Intel Capital and many others have become
investors.
Figures mission, according to their website,
is to quote, expand human capabilities through advanced AI.
They aim to do so by specifically targeting the labour
force.
According to figure, the labour force is shrinking.
They mentioned that quote, there are 10 million unfilled jobs
in the United States.
7 million of those job openings are for essential roles
such as warehouse,
transportation and retail. And there are only 6 million people
available to fill these open positions.
While key warehouse suppliers predict they will run out of
people to hire by
2024 Quote As we reach the upper limits of our
production capability,
humanoids will join the workforce with the ability to think,
learn and interact with their environment safely alongside us.
Figures Founder Brett Adcock is no stranger to the tech
industry.
He's previously founded two other companies,
Veteri and Archer Aviation. Vetri is an online talent marketplace,
and in 2018 they were acquired for 100 million.
Archer Aviation is now publicly traded,
and they're working on building electric vertical take off and
landing aircraft.
So with his experience, Adcock has built a team of
industry veterans,
including professionals who've worked at Boston Dynamics,
Google DeepMind, and Tesla.
Perhaps their biggest breakthrough has come in the form of
their partnership with Open AI.
In a press release in February
2024, Figure announced their collaboration and it will help,
quote accelerate Figure's commercial timeline by enhancing the capabilities of
its robots to process and reason from natural language using
Open A IS research and large multimodal models.
In simple terms, it allows for one to have full
conversations with the robot and I suspect will also make
it better at computer vision.
So ultimately the idea is simple.
Build autonomous humanoid robots to address the global labour shortage
and to do just that.
Figures. First robot is here.
Let's start by looking at how the robot actually works.
First of all, every single action and behaviour in the
video was learned and not teleoperated.
The robot is equipped with cameras and microphones,
so when someone speaks and asks it to do something,
the transcribed texts and images from the cameras are captured
to a large multi modal model which is trained by
open AI to understand both text and images.
The model processes the full conversation including previously shared images.
It then decides which specifically learned closed loop action to
take by selecting and loading the relevant neural network configurations
onto the GPU and then executing it.
The mechanical action of figure 01 is rapid at 200
Hertz,
which basically means that after it knows what to do,
it sends back instructions at 200 times per second.
That's five milliseconds of reaction time For all those micro
adjustments.
Humans are in the order of
250 milliseconds, and fun fact,
a cat's reaction time is around 30 milliseconds.
So essentially they are using a combination of different neural
networks to sort through audio and visual inputs,
understand them, and then do the appropriate actions.
It's actually kind of amazing.
All right, so how do you think you did?
I I think I did pretty well.
The Apple found its new owner.
The trash is gone and the tableware is right where
it belongs.
Did you notice the speech hesitation there?
It's the little human like imperfections that allows us as
viewers to break the illusion that it's an inanimate robot.
We've already seen this in ChatGPT voice.
Hey ChatGPT, can you describe a nice kitchen scene?
But make it very casual and breathe and say umm
and and things that sound very natural while you're describing
sure thing.
Picture this. You're in this cozy kitchen,
right? The sun's peeking through the curtains giving that warm
glow.
You can almost smell the aroma of fresh coffee brewing.
There's this gentle hum of the fridge in the background
kind of soothing,
you know, And maybe there's a bit of chatter as
someone's flipping pancakes on the stove.
It's just this laid back vibe,
like times taking a chill
pill to see it play out in a real robot
was only the next frontier
as a side. In an episode I made back in
late
2021, I talked about how it was only a matter
of time before a company cracked the code of mixing
dexterity,
mobility, and natural language into a single robot.
And three years later, I think we're getting close.
But all that being said,
as you can see, there's still some latency between talking
and getting a response.
And I think that's from open AI side mainly,
although we don't know the intricate details of the integration,
voiced instructions probably still have to go up to a
server and come back.
I've noticed the same delay issue when playing around with
ChatGPT voice.
So there's still some work to be done if these
robots are going to be used in large scale commercial
applications.
Imagine an emergency response robot just standing there for 10
seconds while someone's having a heart attack.
Others are simply not impressed.
Months ago, figures showed a demo of the robot making
coffee.
Some people back then asked if the robots are also
learning bad habits from the training data.
The figure robot is also not the best at walking.
But we can't be too harsh.
It's still an early stage product.
As I always say, criticising is easy,
but building is hard with also such an incredible amount
of investment and very few demo videos to show.
Some have even drawn the comparison to a certain Trevor
Milton.
If you're a regular cold fusion viewer,
you'll know what that's all about,
but obviously this is way too soon for that conversation.
Broadly speaking, it's an exciting time for the robotics industry,
so the playing field for Figure AI isn't going to
be barren by any means.
Let's take a quick look at the latest in the
industry.
First up, this Digit, which is made by Agility Robotics
and backed by Amazon.
Before now, robots could only replace part of a human
job,
enough to make coffee or an omelette.
But this is digit, a robot.
It's make us say is ready to do what human
warehouse workers do.
And they can lift about 35 lbs.
You know, the the robot weighs about what a person
would weigh.
It can reach about the size of what a person
can reach.
You know, we're really going for that kind of like
OSHA regulations and requirements for the robot.
Agility Robotics is funded in part by Amazon,
which is investing a billion dollars in new industrial technologies.
The fact that it can enter places that were designed
for humans
and solve those challenges that exist without having to rebuild
the environment for automation. And it doesn't quit and it
doesn't quit.
They're about
$250,000. So they're not inexpensive,
but you know, over just a few years,
they're worth it. The Digit robots will be deployed in
Amazon warehouses,
and the company aims to produce 10,000 units per year.
Fascinating, crazy, maybe a little scary.
I mean, it sounds like we're talking about not just
using something to help humans do their jobs,
but just replacing it entirely.
Could this be a new normal on a on a
large scale?
I mean, this is the thing,
right? As a tech correspondent for years,
I've had companies tell me,
you know, well, this technology is for augmenting a human
or making a human's job a little bit easier.
In this case, it is truly right out and out
there to replace a human worker.
Amazon warehouse workers, you know,
that's one of the largest employers in America.
People really go for for that job.
And so, you know, we're really,
I think at this sort of historic turning point where
suddenly it's no longer the case of,
of, of, you know, when could technology replace a set
of jobs?
But this is truly something that's built to do that.
Next up, Apptronic is designed to move goods around a
warehouse and can work for 22 hours per day.
All you need to do is swap out the battery
every few hours.
Deliveries start in 2025. The company's also working with NASA
on a robot called Valkyrie and that aims to work
in dangerous environments and perhaps even future space walks.
Toyota and Hyundai are also currently working on robots of
their own,
but I'd say less seriously than the others
compared to other robots. As mentioned earlier,
Figure 01's key difference is the ability to directly converse
in natural language in order to get tasks done for
you.
It's not a new concept,
as about a year ago on Cold Fusion,
we covered a project from Google that did the same
thing.
At that time, we saw that the concept was still
in its infancy and the robot was quite slow.
Earlier this year, we saw Elon Musk sharing a video
of Tesla's Optimus robot folding some clothing.
At first glance, it was impressive,
but the curtains came down later when Elon added that
Optimus couldn't do it autonomously.
But since then there's been footage of the Tesla bot
carrying boxes,
watering plants, walking, crouching, handling delicate objects such as eggs,
and performing other domestic activities.
Interestingly, these abilities come from the knowledge and benefits from
Tesla's self driving efforts,
which is pretty cool. But again,
as far as we know,
it lacks that natural language aspect.
Perhaps the Atlas robot by Boston Dynamics is the most
agile robot we've ever seen.
We've seen it organise objects and even perform backflips.
But it's this and the Figure 01 robot really the
same thing.
Boston Dynamics as Atlas is definitely more focused on mobility,
whereas Figure seems to be suited for intricate tasks.
There's more of a focus on finger dexterity,
alluded to by the five fingers with three joints on
them.
And for factory deployment, this could make more sense.
In a lot of cases,
assembly line work is done by sitting down or being
stationary at a specific location for long periods of time.
And as we were working on this video,
NVIDIA announced Project Group, A general purpose foundational model for
humanoid robots.
It's meant to help humanoid robots quote,
understand natural language, and emulate movements by observing human actions,
quickly learning coordination, dexterity, and other skills in order to
navigate,
adapt, and interact with the real world.
About the same size.
The intersection of computer graphics,
physics, artificial intelligence, it all came to bear at this
moment.
Well, I think we have some special guests,
do we?
Little Jetson Robotics computers inside.
They learn to walk in.
Isaac SIM. Let's go.
Five things. Where you going?
I sit right here.
The system removes some of the legwork from training robots
in the real world via simulation.
The lessons and skills learned can be put into a
knowledge base and transferred to other robots.
Art and video wants Groot machines to be able to
understand human language,
sense and navigate the world around themselves,
and carry out arbitrary tasks.
The training part of the stack starts with video,
so the model can sort of intuitively understand physics,
then accurately modeling the robots in virtual space in order
to teach the model how to move around and interact
with terrain,
objects, or other robots. Jensen described this as a gem
here.
The model can learn basics far quicker than in the
real world.
As we've covered in another episode on the history and
size of NVIDIA,
they're already turning heads with their stocks growing at unprecedented
levels this year.
I'll leave a link to that episode in the description
as well.
The company has also mentioned that they're working with Boston
Dynamics,
Agility Robotics, Apptronic, and of course,
Figure AI.
And just thinking about this,
I can already see an industry standard forming.
Open AI provides the foundational models,
NVIDIA provides some of the processing hardware and software packages,
and finally, there's going to be a plethora of different
robot bodies by different vendors.
So why is this all happening now?
Well, the reason that a lot of this stuff didn't
work in previous decades was because the instructions for the
robots had to be laboriously hand coded.
Now, with generative AI and the latest generations of models,
robots are beginning to be able to dynamically understand their
environment and adjust seamlessly with no hard coding.
The amount of change we're going to see over the
next five years,
10 years, will dwarf everything that's happened over the last
thirty.
We went from automating pen and paper,
right, With spreadsheets to connecting PCs into local area network,
to connecting networks to get the Internet to getting the
network effect of communicating people globally.
But now we're introducing machine,
you know, artificial intelligent, machine learning,
deep learning, neural networks, etcetera.
So and what that enables is the automation of automation,
right? And so the people who were writing software,
particularly at the lower end,
unless you, you're doing these advanced things,
they're gone, right? The people that was
doing itself, right? It's just math programmer back in the
day.
So when I was writing code,
it was the algorithms were if this,
then that X or whatever it may be,
right? But you had to guess,
right? You had to use your,
your best instance. And then it got smarter and smarter
and smarter and libraries to do bigger and better things.
All that is being automated.
The progress of automation has been a funny story to
watch unfold.
If you've been following the tech world closely for long
enough,
you'd remember all of those discussions over a decade ago
about how manual labour jobs would eventually be taken by
robots.
But as we've seen since the AI boom a couple
of years ago,
it was the cognitive jobs that were the first to
be impacted.
In 2017, only one in five companies used AI,
according to McKinsey. Now, half of all companies do.
We've already seen many companies laying off thousands of employees,
in part due to technological advancements like AI.
It's completely the opposite of what most people thought would
happen.
Goldman Sachs last year reported that over 300 million jobs
could be lost or diminished by AI.
Most at risk are legal,
architecture and engineering, business and financial operations,
management, sales, healthcare, art and design.
Goldman Sachs also thinks that the price of robots and
the constitute components will drop drastically,
and they've already dropped drastically since last year.
Unlike humans, robots don't need to take a break.
They don't need to sleep,
go on holidays, strike, and they don't complain.
So as soon as these robots are feasible and cheap
enough,
companies will be chomping at the bit to install them.
So what do we do about all of this?
Well, it's your lucky day because I've already done an
episode on life after automation.
It's a huge topic and will just be way too
long to get into in this episode,
so I'll leave another link to that below.
OK, so back to Figure one.
At this point you might think,
all right, the Figure demo looks cool,
but surely it's going to be a long time until
these humanoid robots are in production.
Well, it seems like Figure didn't want to waste any
time
even before the demo. Earlier in the year,
Figure signed an agreement with BMW Manufacturing.
They're going to test the Figure 01 robot at the
Carmakers plant in Spartanburg,
SC But we're yet to get reports on how the
01 is performing and what tasks it's going to do
exactly.
But I still think that's pretty interesting.
Humanoid robots like the Figure One are a future conundrum.
Will they replace humans completely,
or just alleviate us from menial tasks,
leaving the more important stuff for humans?
But on the other hand,
of course, the military would probably want to have a
sniff at humanoid robots in the future too.
Looking at consumer products, though,
we're pretty far away from getting a highly functional robot
that's cheap that you can buy at your local store.
But the day will probably arrive.
Figure 01 is the worst robot from this company,
so there's only going to be improvement.
Aside from that, the competition is heating up.
So if it's not them,
it's going to be another company that's going to crack
the code
from the broadest perspective possible.
I'm sure opinions from all of you will be split.
For the engineering types, this is an excellent display of
human ingenuity.
For the sceptics, they'll say it's all pointless and we
should focus our efforts elsewhere.
The optimists will see this as an exciting new era
for humanity free of drudgery.
Regardless, whether that future turns into The Jetsons,
The Terminator, or Utopia will just have to let humanity
collectively figure that one out.
All I know is that I'll be watching this space
closely,
and that's where we are with Figure 1 and robotics
today.
I hope you enjoyed that or found it interesting,
and if you did, feel free to share it with
someone who will find it interesting as well.
All right, so that's about it for me.
My name is Togogo and you've been watching Coldfusion.
And if you're new, feel free to subscribe if you
like.
It's free. Cheers, guys. Have a good one,
Cold fusion. It's new thinking.