GPT-4 Gets Physical: Building a Language-Guided Service Robot

This talk demonstrates how a GPT-4o AI agent controls a robot arm to serve drinks using minimal training and language prompts, with potential for various tasks.

Overview

We gave a robot body to a GPT-4o AI agent.

Here serving a glass from a human to another🍹

With only:

• 30mn of arm demonstrations

• A language prompt to guide it

You can also teach it to clean, play, check in on you.

What would you teach it?

Links

https://x.com/ax_pey/status/1853462975216234851
GPT-4o controls a robot arm using few-shot learning and language prompts.

Tech stack