Congratulations! If you’re reading this article, that means you’re interested in contributing to the robotics revolution, which will reshape the way we work and interact with the physical world through AI robotics. More likely than not, you’re also interested in becoming (or already are!) a PrismaX Amplifier, which will let you operate a variety of autonomous robots around the world. Let’s walk through a bit of what you can do now as an Amplifier in AI robotics, why that matters for robotics data collection, and what you’ll be able to do in the future with teleoperation.

Teleoperation Capabilities Right Now in PrismaX

PrismaX Amplifier members can remotely operate a tabletop robot arm to complete simple pick-and-place type tasks in AI robotics. By engaging in teleoperation, you’re directly contributing to robotics data collection that powers autonomous robots.

Collecting Real-World Robotics Data

• Collecting data which teaches AI models how robots interact with the physical world—for example, pushing on objects makes them move, and closing the gripper sometimes grabs objects (and sometimes, they slip out).

  
  

• We’re collecting the video feed from the cameras surrounding the robot as well as the angles and positions of the robot’s joints; during training, the AI model learns to predict what happens in the videos when the robot’s joints move.

  
  

• The data we’re collecting right now will be distributed to research groups around the world to train better frontier models for robotics.

  
  

• Long-form interaction data (including mistakes!) helps autonomous robots more robustly learn how the real world works, enhancing visual data for models in AI robotics.

Building Operator Experience for AI Models

• Learning how to smoothly and accurately teleoperate the robot through the latency of the Internet and the limited number of cameras available.

  
  

• Believe it or not, building operator experience is more important right now for AI robotics. It’s the reason why the tasks are kept intentionally simple right now—if you’ve played with the teleoperation online a bit, you’ll find that it’s not trivial to operate the arm smoothly since the cameras don’t provide depth information and the network connection has latency.

  
  

• A bunch of companies are trying to build technology solutions to the problem, but we’re strong believers in the most powerful neural network in the world—the one in your head—and its amazing capabilities to predict depth and compensate for latency in teleoperation.

  
  

• PrismaX Amplifiers [link: https://gateway.prismax.ai/] play a key role in gathering visual data for models that drive autonomous robots forward.

Upcoming Teleoperation Tasks and Community Contributions

We’re already seeing a lot of experience build up among the community, so we’ll be deploying new tasks on the same tabletop arms to advance AI robotics and robotics data collection.

New Tasks for Environmental Interaction

• More complex tasks that require more precision manipulation: for example, stacking or hanging objects to improve teleoperation skills.

  
  

• Tasks that require longer, planned sequences of steps, for example, making a sandwich (out of fake food!) to enhance environmental interactions in AI robotics.

  
  

• The data collected through these tasks will enhance autonomous robots’ ability to reason about the environment, as well as perform more complex environmental interactions beyond just picking up and releasing objects.

  
  

Community-Contributed Setups

• With the help of the community [link to PrismaX Discord or About page: https://prismax.ai/about], we’ll also be rolling out more varied scenarios for the robots to play in—top community members will have the opportunity to contribute their own setups for teleoperation.

  
  

• Having community-contributed setups enriches the dataset and improves model robustness—even something as simple as lighting changes are incredibly helpful for model performance in robotics data collection.

  
  

• This approach ensures diverse visual data for models, supporting the growth of autonomous robots through collaborative AI robotics efforts.

Exciting Future Developments in AI Robotics Teleoperation

We’ve got a lot of exciting things coming up in teleoperation for AI robotics! Here are the future steps:

1. Deploy new tasks for enhanced environmental interaction in robotics data collection.

   
   

2. Roll out community setups to diversify visual data for models.

   
   

3. Expand to fleet operations and integrate humanoids for advanced autonomous robots.

Fleet Operations and Humanoids

• Faster, more powerful arms based on open-source hardware that can do more in AI robotics.

  
  

• Bimanual (two-armed) setups, which will greatly increase the tasks available for teleoperation.

  
  

• A question we’ve gotten is “when will you integrate humanoids”? The short answer is “eventually”; more precisely, humanoids introduce a lot of intricacies and failure modes that need to be scrutinized in robotics data collection.

  
  

• Another way of looking at it: arms without legs are useful, legs without arms are very questionable, so we want to solve arms first for autonomous robots.

Tele-op Tournaments and Incentives

• Fancier manipulators (hands) for the arms to be able to do things like pick up and use tools in AI robotics.

  
  

• Upcoming tele-op tournaments will offer incentives for participants, encouraging more robotics data collection through competitive teleoperation.

  
  

• These events will boost community engagement and generate high-quality visual data for models to advance autonomous robots.

Why Focus on Tabletop Manipulation for Robotics Data

Why is tabletop manipulation useful for AI robotics? Well, it turns out it is useful beyond just tabletop tasks.

Addressing Depth and Latency Challenges

• Picking up, moving, and dropping objects is a strong prior for other tasks—you can think of it as the equivalent of all the forum posts and marketing articles that get ingested into LLM training.

  
  

• The enormous quantity of tabletop data that can be collected in relatively little time helps robotics AI models pick up on the fundamental patterns that govern physical interaction, much as all that forum slop teaches LLMs how English works and how conversations work.

  
  

• Teleoperation in tabletop setups addresses depth perception and latency issues, building robust visual data for models in AI robotics.

  
  

PrismaX Vision: Scaling Teleoperation for Autonomous Robots

This is why community involvement is crucial in PrismaX's approach to AI robotics.

Community Involvement in Data Diversity

• The value of those forum posts comes from their immense diversity—with millions upon millions of people posting, nearly every topic receives some coverage.

  
  

• Similarly, the creativity of the community will allow our project to build diverse scenarios with broad coverage across tasks, objects, lighting, and environments for robotics data collection.

  
  

• Scaling teleoperation through PrismaX Amplifiers will drive the data flywheel for autonomous robots, ensuring high-quality visual data for models.