Brandon Ong retweeted
18 brilliant folks building the future of robotics, all around one dinner table.
For SF Tech Week, we brought together friends from frontier AI labs, industry, and startups, from researchers advancing RL in simulation to founders building the next generation of robotics of fascinating conversations, fresh perspectives, and plenty of laughs.
If you are building in Physical AI, drop me a message. We would love to have you at the next dinner!
@Luckyballa - Skild AI
@Stone_Tao - Physical Intelligence
@RaresAmbrus - Walden Robotics
@un1c0rnioz - 1x
@charleswongzx - Bifrost AI
@IanGlow - Zeromatter
@brandon1 - Hebbian Robotics
@immonicax - Dyna Robotics
@massiviola01 - Bedrock Robotics
@realnavida - Shiraz AI
@bhavashok - Stealth
Tiantian Li - Meta
Mikhail Balakhno - OpenAI
Yatharth Ahuja - General Motors
Wallace Lim - Physical Intelligence
Pranav Deshmane - John Deere
For SF Tech Week, we brought together friends from frontier AI labs, industry, and startups, from researchers advancing RL in simulation to founders building the next generation of robotics of fascinating conversations, fresh perspectives, and plenty of laughs.
If you are building in Physical AI, drop me a message. We would love to have you at the next dinner!
@Luckyballa - Skild AI
@Stone_Tao - Physical Intelligence
@RaresAmbrus - Walden Robotics
@un1c0rnioz - 1x
@charleswongzx - Bifrost AI
@IanGlow - Zeromatter
@brandon1 - Hebbian Robotics
@immonicax - Dyna Robotics
@massiviola01 - Bedrock Robotics
@realnavida - Shiraz AI
@bhavashok - Stealth
Tiantian Li - Meta
Mikhail Balakhno - OpenAI
Yatharth Ahuja - General Motors
Wallace Lim - Physical Intelligence
Pranav Deshmane - John Deere
7
6
74
7,177
Brandon Ong retweeted
We’ve been using agents to train models on robotics data.
In this blog post, we share what we learned about data, evaluation, and equipping agents with the right tools to do research.
We also open sourced a short example you can run to try it out for yourself.
In this blog post, we share what we learned about data, evaluation, and equipping agents with the right tools to do research.
We also open sourced a short example you can run to try it out for yourself.
4
2
14
1,150
hosting an IROS happy hour with @kstonekuan and @hbr_pbc on mon 9/28, after 6pm.
if you’re excited about open source robotics, dm us! 🦾
if you’re excited about open source robotics, dm us! 🦾
1
1
20
867
we wrapped up yc s26 last week and a friend asked what lessons are worth passing on. here is a cleaned-up version of my answer:
1. team - travel for conferences, live together, have fun together. the more cohesive you are, the better your odds of success
2. community - every their industry. i'm constantly surprised by the lessons i get from hearing how they problem-solve, despite how different our industries are
3. walk your own path - there will be noise. choose what you stand for, but be mindful of the big picture
1. team - travel for conferences, live together, have fun together. the more cohesive you are, the better your odds of success
2. community - every their industry. i'm constantly surprised by the lessons i get from hearing how they problem-solve, despite how different our industries are
3. walk your own path - there will be noise. choose what you stand for, but be mindful of the big picture
9
3
39
1,741
Brandon Ong retweeted
We're opening early access to Hebbian Robotics' custom models for robotics data quality control.
Built for the questions robotics data teams actually ask.
Are the wearer's hands in view? Is there active manipulation?
We want teams to evaluate them and shape what comes next.
Built for the questions robotics data teams actually ask.
Are the wearer's hands in view? Is there active manipulation?
We want teams to evaluate them and shape what comes next.
As more robotics data is collected at scale, catching a bug in the video early (e.g. a hair across the camera lens) and giving the operator immediate feedback become critical to ensuring the quality of the data. It also happens to cost less than running a post-upload quality available option for robot video quality control (QC) runs only after the task is completed. Record the episode, upload it, let Gemini Flash grade the file.
A strand of hair falls across the lens at second nineteen of the task, but the wearer keeps working for another five minutes. By the time the video is uploaded, the only decision left is whether to throw the episode away.
We set out to change this, training a fast fine-tuned model for robotics QC that matches Gemini 3-8 Flash's accuracy and runs on the stream as it records, today at 1 frame per second (FPS). Gemini 3-8 Flash tokenizes video at 1 FPS.
Gemini 3-8 Flash on low thinking costs $5.83 per video-hour, while our fine-tuned model costs $1.16, and less still when we catch the bug early. On an average 7.5-minute episode, that is 73 cents against 15 cents. This price differential compounds downstream.
In addition, a retake is cheap and a discard is not. That gap is what forces you to collect 1,500 hours to meet a buyer's 1,000-hour specification, and it doesn't scale as a vendor moves to 100K- and 1M-hour collections.
Dyna-2 pretrained on 1M hours of egocentric video, and the next rungs on that curve are 10M and 100M. Inspecting 1M hours for a single quality check costs $1.2M on our model and $5.8M on Flash. At 10M hours, $11.6M against $58.3M. At 100M, $116M against $583M.
In practice, a team runs ~7 checks, each tuned to their rig and their operation: overexposure, idle stretches where nothing is being manipulated, hands drifting out of frame. Every one of them is another parallel pass over the corpus. You can fold them into a single prompt, but at the expense of accuracy.
All of this is the QC bill alone, before accounting for other operational costs. Margins on ego data are already being squeezed, and we think a 10x faster, 8x cheaper inspection layer is what makes the unit economics work for even the largest data hyperscalers.
Today, there are many bottlenecks to scaling up the data robotics needs. Existing tools will not cut it, and our mission is to rebuild the ones that let robots learn from data at the scale general intelligence requires.
If your QC runs after the upload, let's chat.
7
4
32
1,737
Most people I spoke with this past week are still missing the "why now" in robotics.
In 2025, AI in robotics still looked like a research problem. In 2026 it is showing "sparks" of intelligence, similar to the moment right before ChatGPT was released.
Here is what changed. A year ago, NVIDIA was among the first to publish studies that ran on ~20K hours of robot training data and found scaling laws, similar to those in LLMs, between data scale and robot task performance. This year, Dyna Robotics (Dyna-2) took the idea further, pretrained models on 1M hours, and found that the scaling law holds at that scale.
Why scaling laws matter: robotics has conventionally been costly and integrator-heavy, because every new task and every new site was a new project. With pretraining and scaling laws, a model that adapts to a new task and a new site in hours changes how quickly robots can be put into the world to do useful work.
This is why every frontier lab is suddenly talking about "few-shot learning". First coined in the LLM space, it now means robots need only a few hours of demonstrations, sometimes as little as 2 hours, to perform useful tasks in sectors where tremendous labor shortages already exist.
The point was actually never to replace people. There is a long list of sectors that cannot hire fast enough: cleaning, data center construction and maintenance, solar. Robots let a strained workforce keep delivering the services we take for granted.
Scaling laws only pay off if the ecosystem can actually produce the next order of magnitude of data. 1M to 10M hours is orders of magnitude more challenging, and getting the industry to this next frontier of intelligence requires infrastructure that has never existed before.
One such bottleneck, and one that Hebbian actively works on, is quality control. AI models in robotics are more sensitive to noise than LLMs, which is why quality control is a core component that every AI model training pipeline relies heavily on.
Inspecting 1M hours of video for quality today costs $5.8M on Gemini 3-8 Flash. These models were never built for the demands of robotics QC, but they are the best option available to the data hyperscalers.
What ends up happening is that these data teams can only afford to run partial quality control, and buyers reluctantly accept such quality because there is no better alternative. This means the industry today is forgoing model improvement experiments for lack of high quality data, inhibiting progress in robotics.
Vendors themselves collect 1,500 hours to meet a 1,000-hour delivery. This works today. It does not survive 1M and 10M hours.
I have tremendous respect for the teams going against all odds to get us to the next frontier of robotics intelligence: @MeckaAI, @buildpbc, @ropedia_ai, Praxis, @DeepReach_AI, @DatoricAI, Luel and @joinHandshake on data.
And for the great teams at @cosmic_robotics, Proprio and Neuromorphic, pushing the frontier of what deployed robots can solve in the real world.
We think data quality tooling is the underinvested piece of infrastructure. Our mission is to accelerate the data engine for robots to learn from data at the scale general intelligence requires.
If you're excited to see more AI in robots, reach out.
In 2025, AI in robotics still looked like a research problem. In 2026 it is showing "sparks" of intelligence, similar to the moment right before ChatGPT was released.
Here is what changed. A year ago, NVIDIA was among the first to publish studies that ran on ~20K hours of robot training data and found scaling laws, similar to those in LLMs, between data scale and robot task performance. This year, Dyna Robotics (Dyna-2) took the idea further, pretrained models on 1M hours, and found that the scaling law holds at that scale.
Why scaling laws matter: robotics has conventionally been costly and integrator-heavy, because every new task and every new site was a new project. With pretraining and scaling laws, a model that adapts to a new task and a new site in hours changes how quickly robots can be put into the world to do useful work.
This is why every frontier lab is suddenly talking about "few-shot learning". First coined in the LLM space, it now means robots need only a few hours of demonstrations, sometimes as little as 2 hours, to perform useful tasks in sectors where tremendous labor shortages already exist.
The point was actually never to replace people. There is a long list of sectors that cannot hire fast enough: cleaning, data center construction and maintenance, solar. Robots let a strained workforce keep delivering the services we take for granted.
Scaling laws only pay off if the ecosystem can actually produce the next order of magnitude of data. 1M to 10M hours is orders of magnitude more challenging, and getting the industry to this next frontier of intelligence requires infrastructure that has never existed before.
One such bottleneck, and one that Hebbian actively works on, is quality control. AI models in robotics are more sensitive to noise than LLMs, which is why quality control is a core component that every AI model training pipeline relies heavily on.
Inspecting 1M hours of video for quality today costs $5.8M on Gemini 3-8 Flash. These models were never built for the demands of robotics QC, but they are the best option available to the data hyperscalers.
What ends up happening is that these data teams can only afford to run partial quality control, and buyers reluctantly accept such quality because there is no better alternative. This means the industry today is forgoing model improvement experiments for lack of high quality data, inhibiting progress in robotics.
Vendors themselves collect 1,500 hours to meet a 1,000-hour delivery. This works today. It does not survive 1M and 10M hours.
I have tremendous respect for the teams going against all odds to get us to the next frontier of robotics intelligence: @MeckaAI, @buildpbc, @ropedia_ai, Praxis, @DeepReach_AI, @DatoricAI, Luel and @joinHandshake on data.
And for the great teams at @cosmic_robotics, Proprio and Neuromorphic, pushing the frontier of what deployed robots can solve in the real world.
We think data quality tooling is the underinvested piece of infrastructure. Our mission is to accelerate the data engine for robots to learn from data at the scale general intelligence requires.
If you're excited to see more AI in robots, reach out.
2
3
25
1,525
Brandon Ong retweeted
GPT-6 Astra vs MediaPipe for 3D hand pose estimation
It looks like GPT-6 Astra is showing SOTA results in visual and spatial reasoning (mazebench) and can be used for labelling and annotating robotics datasets.
I tested this out against MediaPipe, which is a fast method for 3D egocentric data, but a known limitation is that it can’t be applied to gloves.
GPT-6 Astra was constrained to the same output schema as MediaPipe, then the results were visualized side by side.
High reasoning took 3 min per frame, while MediaPipe took only 20 ms per frame.
It looks like GPT-6 Astra is showing SOTA results in visual and spatial reasoning (mazebench) and can be used for labelling and annotating robotics datasets.
I tested this out against MediaPipe, which is a fast method for 3D egocentric data, but a known limitation is that it can’t be applied to gloves.
GPT-6 Astra was constrained to the same output schema as MediaPipe, then the results were visualized side by side.
High reasoning took 3 min per frame, while MediaPipe took only 20 ms per frame.
data labeling is dead
GPT-6 Astra easily tells Celtics players from Knicks players
whether they play at home or on the road
16
39
296
53,761
As more robotics data is collected at scale, catching a bug in the video early (e.g. a hair across the camera lens) and giving the operator immediate feedback become critical to ensuring the quality of the data. It also happens to cost less than running a post-upload quality available option for robot video quality control (QC) runs only after the task is completed. Record the episode, upload it, let Gemini Flash grade the file.
A strand of hair falls across the lens at second nineteen of the task, but the wearer keeps working for another five minutes. By the time the video is uploaded, the only decision left is whether to throw the episode away.
We set out to change this, training a fast fine-tuned model for robotics QC that matches Gemini 3-8 Flash's accuracy and runs on the stream as it records, today at 1 frame per second (FPS). Gemini 3-8 Flash tokenizes video at 1 FPS.
Gemini 3-8 Flash on low thinking costs $5.83 per video-hour, while our fine-tuned model costs $1.16, and less still when we catch the bug early. On an average 7.5-minute episode, that is 73 cents against 15 cents. This price differential compounds downstream.
In addition, a retake is cheap and a discard is not. That gap is what forces you to collect 1,500 hours to meet a buyer's 1,000-hour specification, and it doesn't scale as a vendor moves to 100K- and 1M-hour collections.
Dyna-2 pretrained on 1M hours of egocentric video, and the next rungs on that curve are 10M and 100M. Inspecting 1M hours for a single quality check costs $1.2M on our model and $5.8M on Flash. At 10M hours, $11.6M against $58.3M. At 100M, $116M against $583M.
In practice, a team runs ~7 checks, each tuned to their rig and their operation: overexposure, idle stretches where nothing is being manipulated, hands drifting out of frame. Every one of them is another parallel pass over the corpus. You can fold them into a single prompt, but at the expense of accuracy.
All of this is the QC bill alone, before accounting for other operational costs. Margins on ego data are already being squeezed, and we think a 10x faster, 8x cheaper inspection layer is what makes the unit economics work for even the largest data hyperscalers.
Today, there are many bottlenecks to scaling up the data robotics needs. Existing tools will not cut it, and our mission is to rebuild the ones that let robots learn from data at the scale general intelligence requires.
If your QC runs after the upload, let's chat.
A strand of hair falls across the lens at second nineteen of the task, but the wearer keeps working for another five minutes. By the time the video is uploaded, the only decision left is whether to throw the episode away.
We set out to change this, training a fast fine-tuned model for robotics QC that matches Gemini 3-8 Flash's accuracy and runs on the stream as it records, today at 1 frame per second (FPS). Gemini 3-8 Flash tokenizes video at 1 FPS.
Gemini 3-8 Flash on low thinking costs $5.83 per video-hour, while our fine-tuned model costs $1.16, and less still when we catch the bug early. On an average 7.5-minute episode, that is 73 cents against 15 cents. This price differential compounds downstream.
In addition, a retake is cheap and a discard is not. That gap is what forces you to collect 1,500 hours to meet a buyer's 1,000-hour specification, and it doesn't scale as a vendor moves to 100K- and 1M-hour collections.
Dyna-2 pretrained on 1M hours of egocentric video, and the next rungs on that curve are 10M and 100M. Inspecting 1M hours for a single quality check costs $1.2M on our model and $5.8M on Flash. At 10M hours, $11.6M against $58.3M. At 100M, $116M against $583M.
In practice, a team runs ~7 checks, each tuned to their rig and their operation: overexposure, idle stretches where nothing is being manipulated, hands drifting out of frame. Every one of them is another parallel pass over the corpus. You can fold them into a single prompt, but at the expense of accuracy.
All of this is the QC bill alone, before accounting for other operational costs. Margins on ego data are already being squeezed, and we think a 10x faster, 8x cheaper inspection layer is what makes the unit economics work for even the largest data hyperscalers.
Today, there are many bottlenecks to scaling up the data robotics needs. Existing tools will not cut it, and our mission is to rebuild the ones that let robots learn from data at the scale general intelligence requires.
If your QC runs after the upload, let's chat.
11
3
59
6,749
Brandon Ong retweeted
What does quality control actually look like for egocentric data used to train robotics models?
QC often needs to run automatically in the background, so it’s hard to visualize what gets flagged. Most datasets you see online have already gone through extensive post-processing, llecting (@SkildAI S1)
Grid AI shared some samples before any post-production. We ran one through HFlow’s built-in checks and visualized the results below.
Three patterns flagged:
1. Both hands out of frame → trim affected segments or improve collection guidelines.
2. No active manipulation between tasks → segment the video into useful task clips.
3. Shakiness during a camera adjustment → stabilize digitally where appropriate, or improve how the camera is secured.
These checks help teams decide what to keep, what to process, and what to improve in the next round of collection.
QC often needs to run automatically in the background, so it’s hard to visualize what gets flagged. Most datasets you see online have already gone through extensive post-processing, llecting (@SkildAI S1)
Grid AI shared some samples before any post-production. We ran one through HFlow’s built-in checks and visualized the results below.
Three patterns flagged:
1. Both hands out of frame → trim affected segments or improve collection guidelines.
2. No active manipulation between tasks → segment the video into useful task clips.
3. Shakiness during a camera adjustment → stabilize digitally where appropriate, or improve how the camera is secured.
These checks help teams decide what to keep, what to process, and what to improve in the next round of collection.
3
2
19
1,423
Brandon Ong retweeted
We just added native LeRobot Dataset v3 import support to HFlow.
HFlow uses MCAP as its canonical format and required users to handle conversion, but you can now import @LeRobotHF datasets directly without additional converters.
Any other formats we should support natively?
HFlow uses MCAP as its canonical format and required users to handle conversion, but you can now import @LeRobotHF datasets directly without additional converters.
Any other formats we should support natively?
2
3
9
527
Brandon Ong retweeted
We used HFlow to evaluate the latest open weights VLMs for processing egocentric data.
This was based on @buildpbc's Egocentric-10k evaluation, which used Gemini 2.5 Flash to measure hand visibility and active manipulation.
How much each model agreed with the reported results:
65% (baseline)
- GLM 5.3 Flash: 91.00%
- Gemma 4 26B-A4B: 90.87%
- Qwen 3.8 27B: 90.79%
- Inkling Small: 85.21%
Gemma was the standout. Its results were on par with Gemini while being 19x cheaper.
Both Gemma and Qwen models are practical to self-host, enabling private processing without data egress.
This suggests modern open weights VLMs are becoming good enough for large-scale egocentric data processing. The main differentiators are increasingly cost, throughput, output reliability, and ease of self-hosting.
This was based on @buildpbc's Egocentric-10k evaluation, which used Gemini 2.5 Flash to measure hand visibility and active manipulation.
How much each model agreed with the reported results:
65% (baseline)
- GLM 5.3 Flash: 91.00%
- Gemma 4 26B-A4B: 90.87%
- Qwen 3.8 27B: 90.79%
- Inkling Small: 85.21%
Gemma was the standout. Its results were on par with Gemini while being 19x cheaper.
Both Gemma and Qwen models are practical to self-host, enabling private processing without data egress.
This suggests modern open weights VLMs are becoming good enough for large-scale egocentric data processing. The main differentiators are increasingly cost, throughput, output reliability, and ease of self-hosting.
8
3
22
2,168
Brandon Ong retweeted
We just launched HFlow on Product Hunt, our open source SDK for scalable multimodal data pipelines in robotics and physical AI.
@brandon1 and I built HFlow because robotics teams are collecting more hours of video data than they can handle.
Each episode can contain camera streams, robot state, actions, timestamps, and metadata.
As a corpus grows, one-off scripts make it hard to answer basic questions: Did a camera freeze? Did streams drift out of sync? Which version of a check ran? Can we reproduce the dataset we used?
HFlow turns that work into a pipeline. Transforms, checks, labels, and enrichments stay yours.
HFlow handles orchestration, storage, versioning, and curation around them.
If you work with robotics or physical AI data, try the quickstart and tell us where your current data workflow loses the most context. Is it ingestion, quality control, provenance, or curation?
@hbr_pbc
@brandon1 and I built HFlow because robotics teams are collecting more hours of video data than they can handle.
Each episode can contain camera streams, robot state, actions, timestamps, and metadata.
As a corpus grows, one-off scripts make it hard to answer basic questions: Did a camera freeze? Did streams drift out of sync? Which version of a check ran? Can we reproduce the dataset we used?
HFlow turns that work into a pipeline. Transforms, checks, labels, and enrichments stay yours.
HFlow handles orchestration, storage, versioning, and curation around them.
If you work with robotics or physical AI data, try the quickstart and tell us where your current data workflow loses the most context. Is it ingestion, quality control, provenance, or curation?
@hbr_pbc
7
2
22
811
Brandon Ong retweeted
This release effectively doubles the hours of open source egocentric data on @huggingface
This was previously dominated by @buildpbc's Egocentric-100K, which was single monocular camera only
That's 25 TB vs 53 TB (Ego-100k vs EgoSuite-100k) as a size comparison
Ego-100k also only had industrial/factory data, while EgoSuite-100k has 7 different environments
Congrats @LightwheelAI team!
This was previously dominated by @buildpbc's Egocentric-100K, which was single monocular camera only
That's 25 TB vs 53 TB (Ego-100k vs EgoSuite-100k) as a size comparison
Ego-100k also only had industrial/factory data, while EgoSuite-100k has 7 different environments
Congrats @LightwheelAI team!
The largest fully annotated open egocentric human dataset.
Today we're open-sourcing EgoSuite-Open100K with @huggingface : 100,000 hours of egocentric human data, free for anyone to use.
Physical AI has a scaling law of its own now, and its input is human data. That part is still needs is a shared foundation to build on.
15,000+ tasks. 15,000+ real scenes, from factory floors to retail backrooms. Hand pose, body pose and subtask-level semantics, with wrist camera coverage on part of the set. Licensed for commercial training, not research-only.
We build the data infrastructure for Physical AI. Open100K is the first public layer of it.
The first 10,000 hours are live now, with the rest rolling out in stages.
2
6
19
2,313
