hacker-in-residence/ principal devrel engineer. data curation for physical and visual ai systems. in my physical ai era
I ship daily
Joined April 2020
- Tweets7.1K
- Following1.6K
- Followers8.2K
- Likes33.3K
Pinned Tweet
a robot can recognize every object in a room and still not know how to open a single drawer
knowing there's a cabinet is scene understanding. knowing which handle to grab, which direction to pull, and that it's a drawer not a door is affordance understanding
almost no dataset has it in 3D
SceneFun3D: 14,800+ functional elements across 710 laser-scanned indoor scenes. every handle, knob, button, and switch labeled with its affordance, motion axis, motion type, and a natural language task description
each scene's 3D point cloud with its iPad video recordings in fiftyone. the affordance annotations live in 3D and project into the video frames
check it out here huggingface.co/datasets/Voxe…
seven cameras and a LiDAR on Indian city streets. no boxes in the files.
this Minus Zero slice is 8 MCAP files, about 93 GB. cameras: front, front-left, front-right, front-tilted, rear, rear-left, rear-right. LiDAR is stored as autonome.LidarFrame protobuf, with GNSS and pose. video is H.265. recorded October 2025. CC BY-NC 4.0. a 1,469-file collection, about 3.2 TB, is the full set.
huggingface.co/datasets/gaga…
#PhysicalAI #AutonomousDriving
species labels at this count are rare in a vision set.
TreeOfLife-200M holds 233M images across 933K taxa, labeled from kingdom through species.
huggingface.co/datasets/imag…
#PhysicalAI #BioML
lidar sees a blizzard as ten thousand small objects. radar sees nothing at all
most driving datasets are shot on sunny days in california.
Boreas is UTIAS's answer: 128-beam lidar, 360 degree radar, and 5MP camera driving the same Toronto route for a full year through sun, rain, and snow.
326,180 3D boxes for cars, pedestrians, and cyclists across 7,111 labeled frames
loaded as native mcap in fiftyone so you can scrub camera, lidar, and radar on one synced timeline, and watch the 3D boxes render live on the point cloud and project onto the camera and radar images
checkout the dataset here: huggingface.co/datasets/Voxe…
or get hands-on with this hugging face space: huggingface.co/spaces/harpre…
#mcap #skills #computervision #ai #artificialintelligence #machinevision #machinelearning #physicalai
harpreet retweeted
new day, new dataset 🤗!
we now have 900+ 3D files from @NASA on the hub to explore!
- includes space shuttles, satellites, and more! 🚀
- permissively licensed, filtered
dataset link: huggingface.co/datasets/suva…
use this space to explore and look around: huggingface.co/spaces/suvadi…
closed-loop driving success on AlpaSim is up 33 percentage points.
LaPla is a VQ-VAE latent-aligned vision-language-action planner. it predicts trajectories on multi-view driving video and reports L2 trajectory error.
arxiv.org/abs/2609.04070
#PhysicalAI #AutonomousDriving
evaluating world collection usually runs into the same bottleneck:
clean lab setups rarely match messy physical environments.
World Collection captures the ground truth:
42 mathematically generated gazebo simulation worlds with programmatic python sdf generation and standardized start poses for navigation benchmarks.
github.com/mertgulerx/world-…
the contour is where a segmentation model gets sure and wrong.
RABR reads pixel-wise segmentation margin entropy and test-time augmentation variance along boundary contours to isolate overconfident failure regions.
arxiv.org/abs/2609.12892
#PhysicalAI #ComputerVision
a static avatar still has to move on command.
AvatarDynamizer turns a static 3D avatar into a controllable 4D one with generative dynamic textures in texture space, and it includes a new large multi-view set. outputs are 3D gaussians and dynamic texture maps.
arxiv.org/abs/2608.19900
#PhysicalAI #ComputerVision
the mask matches the pixel, on millions of composed images.
Synthetic Object Compositions has 2M images (FC-1M and GC-1M) and 20M synthetic segments across more than 47,000 categories, with IC-Light relighting and pixel-perfect masks. labels include panoptic masks, bounding boxes, referring expressions, and categories.
github.com/weikaih04/Synthet…
#PhysicalAI #ComputerVision
two of voxel51's finest, don't miss it
Join us Nov 9 for the "Agentic Labeling for Specialized Visual Data" workshop! Register for the Zoom - voxel51.com/events/when-prom…
Frontier multimodal models handle familiar concepts well and struggle on niche ones: sorting drill-core samples into 35 geological classes, or telling adhesion from scarring on a ceramic substrate. Voxel51's study across 11 specialized visual datasets in manufacturing inspection, geology, and medical imaging found that a few labeled examples in the prompt can close much of that gap, and it also found where that stops working.
This workshop is about applying those results to your own labeling work. We start from what the study measured, then build a few-shot labeling task live with Agentic Labeling in @Voxel51 on a real inspection dataset, and use the data to decide when to keep prompting and when to train a specialist model.
Run it on your own data! Attendees get free GPU compute from @nebiusai to try the workflow after the session.
#Voxel51 #FiftyOne #MultimodalAI #ComputerVision #MachineLearning #AIResearch
rain at night is where driving gets dangerous and where lidar and cameras quietly give up
CMHT adds the two sensors that don't care: radar and thermal. four drives from dusk/clear into night/rain, same route, so you can watch each modality degrade
i converted the raw ros2 bags into synced mcap episodes in fiftyone so you can scrub camera, thermal, lidar, radar, and GPS together frame by frame with 3D and 2D boxes playing back in sync
start here, download the full dataset: huggingface.co/datasets/Voxe…
or check out a subset in this hf space: huggingface.co/spaces/harpre…
#mcap #skills #computervision #ai #artificialintelligence #machinevision #machinelearning #physicalai
a multi-shot clip needs a camera plan before any frames exist.
ShotVerse is a plan-then-control method for multi-shot camera control in text-to-video. SIGGRAPH Asia 2026.
arxiv.org/abs/2603.11421
#PhysicalAI #ComputerVision
harpreet retweeted
If you're a rockstar DevRel in AI and in the bay area 🇺🇸, we are hiring!
DM me or apply here!
careers.nebius.com/job?gh_ji…
RT for good karma!
“Startup companies like Nebius are doing incredibly well.” - @JensenHuang, @nvidia CEO at #NVIDIAGTC Berlin keynote
hours of two-handed work, with cameras on the contact at the fingertips.
SharpaDex v1.0 is 7.69 TB: 28,993 teleoperated episodes, 32.4M frames, about 300 hours at 30 FPS, across 59 tasks, with two 7-DoF arms and two 22-DoF hands. each episode has 4 RGB views and 2 tactile cameras, 6-axis fingertip wrench sensors, 65-DoF joint state and action, instructions, and 217k subtask segments, in LeRobot parquet and mp4.
huggingface.co/datasets/Shar…
#PhysicalAI #Robotics
you feel the contact when you teleoperate a robot. the policy trained on your demo doesn't
HapTile records both: 1,699 teleoperated episodes across 38 tasks on a UR5e, with a vision-based tactile sensor on each gripper finger, two RGB-D cameras, language instructions, and the haptic feedback the operator felt
i put all 1,699 in fiftyone so you can browse by task and haptic level, and play tactile, depth and robot state on one timeline
start here, read the dataset card: huggingface.co/datasets/Voxe…
harpreet retweeted
Today at @NVIDIAGTC Berlin, @Porsche, @VW Group brands, and @cariad_tech announced that they are jointly building a shared AV data flywheel, with Porsche leading the initial implementation and ecosystem integration together with @Voxel51, @NVIDIA, and @Microsoft: link.voxel51.com/blfs3v
