Khamees Mustapha

gprisma {} How PrismaX Took π0.5 Inference From 230 ms to 87 ms PrismaX @PrismaXai encountered an unexpected performance problem while testing its π0.5 model on a different YAM-based robot setup. The goal was to see whether a model trained on one data collection rig could work reliably on another. This matters for smaller robotics teams using existing hardware. The team had investigated camera settings, lighting, training configurations, and arm calibration. Even after extensive training and more than 10,000 simulation runs, real-world performance remained below expectations. Something else was getting in the way. <> The Hidden Problem Was Not the Model The team noticed that the inference dashboard's video feed was several hundred milliseconds behind the robot's actual movement. The delay came from Python's buffered pipe, which waited for 64 KB of frame data to accumulate before passing it to the inference process. Because the frames were small, this created unnecessary waiting time. The robot was effectively making decisions using outdated visual information. After removing the buffering delay, PrismaX saw an improvement, particularly in the robot's initial approach during picking and placing actions. This revealed an important lesson. Poor performance does not always mean a model needs more training. Sometimes the surrounding system prevents it from responding promptly to the information it receives. <> Why Simulation Did Not Reveal the Full Problem PrismaX had tested latency in its simulator and found that delays of up to 280 ms had little effect on model accuracy. Yet removing a similar delay from the physical setup produced a significant improvement. The team's working explanation was that delayed visual feedback increased near misses, causing collisions, tilted containers, and objects rolling or sliding. The simulator's simpler physics model did not fully reproduce these consequences. A model could therefore perform reasonably well in simulation while behaving less reliably in a physical environment. Benchmark results alone do not always tell the whole story. <> Optimizing the Inference Engine PrismaX then looked for ways to reduce the time π0.5 needed to process information and generate actions. Several changes made a difference: * Converting model weights to bfloat16 once, rather than repeatedly, reduced latency from 230 ms to 203 ms. * Using CUDA graphs, removing unnecessary autograd recording, and moving image conversion to the GPU brought it down to 121 ms. * Reducing prompt padding and optimizing the SigLIP vision encoder lowered it further to 104 ms. * Switching the Gemma backbone to float16, while checking for numerical issues, brought final inference latency down to 87 ms. Together, these changes made inference nearly three times faster. <> What Changed for the Robot? The improvement was visible in its behaviour. With latency reduced, the team observed movements that better reflected the training demonstrations. Human operators would approach an object quickly, pause to assess the gripper's position, and make a small correction before grabbing it. This behaviour began appearing in the robot's rollout too. PrismaX also observed cross-handed placement and hand swaps in one session, behaviours included in the training data. The 87 ms result therefore meant more than a faster benchmark. It helped the robot respond more effectively to what it saw and reproduce actions closer to its demonstrations. The team also released its optimized implementation, prism-pi, on GitHub as a tested reference for developers working with π0.5. For physical AI, better models are only part of the equation. The systems running them must also respond quickly enough for real-world action. PrismaX's latency investigation shows how fixing an overlooked engineering detail can make a meaningful difference to robot behaviour. Follow the PrismaX account on X for more updates: @PrismaXai @MaxC16134