session went well, was fun to whiteboard with people again after a while. thanks @itsarnavb and @_lagrangepoint for hosting us, and everyone who joined. you’re og quick summary of what we talked about: started by sketching a crude version of how we’d integrate force as a modality ourselves. image tokenised through ViT, proprio, and text encoded into a pretrained VLM, latent flowing into an action head in the same policy. also talked about action chunking on the decoder helping with inference latency and temporal coherence within a chunk, though long chunks risk drifting from the world then started picking apart the paper. the action head is actually a separate altered transformer that ingests numerical force as input and outputs (delta_p, f), directly fed into the robot’s impedance controller which converts to motor torque and makes the arm respond to feedback. different from the usual IK-based position controller logic discussed that the improvement over pi0 might have come from the model being steered to produce a force vector as output, not the MoE in the action head. paper doesn’t show with/without MoE ablations so can’t say also briefly touched on interrupts for robots. training time we feed deliberately bad demos and recover. inference time we use DAgger and add a human intervention label and retrain. model has no internal knowledge of whether data came from human intervention or general teleop some questions that branched off from our discussion: - apart from force, what other modalities of input and feedback would make action policies more aware and accurate? - slow and fast moving policies and how they get baked together at high frequency of operation? - flow inpainting as an inference time optimisation for cloud based vlas? (under explored)