David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_) header
David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_) profile picture

Utility Field Coordinator by day, ML researcher by night Founder of VecP Labs Silver City, NM [email protected]

Silver City NM
Joined April 2014
Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)Photo by David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ (@Proph37_)
Yes I'm still building and researching just being a little more subtle about it. Only been a year and have lots in store for the coming years. -Odin is slated to reach mouse levels of intelligence by the end of 2027. Will be scaling it further hardware permitting. -Pulse-5 which lowered vram usage and increased training speed will be tested more thoroughly -looking to release a game down the road that has a world model controlling the NPCs -Atlas may get a resurgence if I feel like it -Currently working on an adaptive computational intelligence component inside the mlp expansion. Should be a good year.
14
One of the limitations of CBF safety filters is that they tend to be greedy. They make the smallest correction needed right now, but that doesn't necessarily mean they're making the best decision over time. This paper takes an interesting approach by learning the future cost of those interventions while keeping the actual safety constraint hard. What I find interesting is that the safety guarantee doesn't depend on the learned network being accurate. The network can be wrong about future costs without compromising the underlying barrier guarantee. In collision avoidance, they reduced cumulative intervention effort by roughly 50% across 100 held-out initial conditions, with similar results across three independently trained networks. I'd be curious how well this holds up in more dynamic environments where the system has to continuously adapt to changing conditions. arxiv.org/abs/2611.21036
5
One of the things I've been interested in with local learning is getting away from backprop without just replacing it with another training process that still depends on batches. TeReL takes an interesting approach to that. It uses a VICReg-style objective with temporally adjacent inputs and running population statistics to produce a per-neuron learning rule. The resulting updates are Hebbian for feedforward connections and anti-Hebbian for lateral connections. Essentially, the network learns to preserve features that remain stable over time while preventing everything from collapsing into the same representation. What I find interesting is that it doesn't need a separate training phase. Learning and inference use the same computation, so the network can continue adapting to unlabeled inputs as it operates. The results are pretty modest. On a two-layer network processing MNIST one sample at a time, they achieved 96.48% linear probe accuracy compared to 95.09% for random features. The authors aren't claiming to outperform backprop, and I don't think that's really the interesting part anyway. For me, it's the idea of a network that doesn't have to stop operating to learn. That's something I've been exploring with my own research, and I think approaches like this are worth investigating. I'd be curious how well the lateral anti-Hebbian updates hold up with a continuous, unpredictable stream of inputs rather than data organized into classes. Especially when the underlying environment starts changing and previously useful representations become less relevant. It's still a small experiment, but I think the learning mechanism is more interesting than the benchmark results. arxiv.org/abs/2611.17019
9
Interesting approach to recurrent spiking neural networks. CoRe SNNs divide the network into communities with defined roles. One handles incoming stimuli, each action has its own community, and latent communities maintain internal state. Rather than using a conventional output layer, the selected action is determined by where the spike activity ends up. Learning is entirely local using three-factor rules. No backpropagation. They also introduce Map-STDP, which encourages modular structure during training rather than relying on it to emerge naturally. On CartPole, it improves learning compared to the same network without it, and with two of the three neuron models, it performs comparably to fully isolated communities. Interestingly, it also makes the resulting spike trajectories more compressible. Replacing a feed-forward actor with recurrent action communities improves performance on CartPole and LunarLander, while roughly matching performance on Acrobot. What I find interesting isn't necessarily the performance. These are relatively small control tasks. It's the architecture itself. They're treating modularity as an explicit design variable while maintaining local learning and recurrent dynamics. I'm curious how well the approach scales as the number of actions and communities increases. Does the spike-walk readout remain reliable, or does routing activity between communities eventually become a bottleneck? Either way, it's an interesting direction for exploring structured recurrent systems without relying on global gradient propagation. arxiv.org/abs/2611.14262 @StphTphsn1
1
55
Discovering Symbolic Local Learning Rules Interesting approach to moving away from backpropagation. Instead of manually designing local learning rules, the researchers build a dictionary of possible polynomial interactions between local tensor variables, then optimize the coefficients against a global objective. Essentially, they're searching for mathematical relationships that allow individual layers to learn without relying on traditional backpropagation. What's interesting to me isn't necessarily the performance. The experiments are small, and I wouldn't draw any major conclusions from them yet. It's the interpretability. Rather than training another neural network to figure out how a neural network should learn, they're producing explicit equations that we can actually inspect and analyze. I'd be curious to see how many of these discovered rules resemble existing Hebbian or predictive-coding mechanisms. I'd also like to know how quickly the search space explodes as additional variables are introduced. The bigger question, at least to me, is whether we can discover a relatively small set of learning primitives that generalize across architectures and tasks rather than having to rediscover the rules for every configuration. That's where I think this could get interesting. arxiv.org/abs/2611.12248
1
18
Discovering Symbolic Local Learning Rules Interesting approach to moving away from backpropagation. Instead of manually designing local learning rules, the researchers build a dictionary of possible polynomial interactions between local tensor variables, then optimize the coefficients against a global objective. Essentially, they're searching for mathematical relationships that allow individual layers to learn without relying on traditional backpropagation. What's interesting to me isn't necessarily the performance. The experiments are small, and I wouldn't draw any major conclusions from them yet. It's the interpretability. Rather than training another neural network to figure out how a neural network should learn, they're producing explicit equations that we can actually inspect and analyze. I'd be curious to see how many of these discovered rules resemble existing Hebbian or predictive-coding mechanisms. I'd also like to know how quickly the search space explodes as additional variables are introduced. The bigger question, at least to me, is whether we can discover a relatively small set of learning primitives that generalize across architectures and tasks rather than having to rediscover the rules for every configuration. That's where I think this could get interesting. arxiv.org/abs/2611.12248
13
Global IPC says how much a reservoir can compute. L-IPC asks what each dynamical state is doing: nonlinearity order, memory lag, and computational irrelevance (the residual that isn't input-driven). Same global capacity, three different anatomies. Homogeneous ESN. Electrode- and time-structured ion-gating reservoir. Frequency-structured spin-wave reservoir. Using those profiles to prune states for NARMA beats random and weight-magnitude selection on the structured devices. Irrelevance is the useful new knob. In an unstable ESN, more than half the available capacity was sitting behind noise/chaos they can now measure per state. arxiv.org/abs/2611.05959 L-IPC It turns neuromorphic "how much" into "where," with a pruning signal that actually follows the physics of the device.
1
15
A planner can keep improving its predicted cost while actually making worse decisions. This paper looks at whether expected free energy can help determine when an agent should stop planning. The results are mixed. Stopping when predicted improvement stalls saves compute without sacrificing much success in some tasks. But rewarding disagreement between predictions can actually make performance worse. The interesting part is what happens when the planner starts exploiting errors in its own world model. Predicted cost keeps improving even after real-world success has peaked. The agent essentially gets better at solving a problem that doesn't exist. Online learning helps recover some of that performance, but it raises a question I've been thinking about. If an agent can't recognize when its own representation of the world is wrong, how useful is additional planning? More compute doesn't necessarily mean better decisions. Expected free energy as a test-time compute controller arxiv.org/abs/2610.28011
1
21
Interesting results from Hyperbolic Contrastive RL. They replaced Euclidean distance with hyperbolic geodesic distance in the contrastive objective. On harder navigation tasks, the depth 64 hyperbolic agents outperform the 1024-layer Euclidean models while training over 12x faster with roughly 13x fewer parameters. What caught my attention was the humanoid maze behavior. The hyperbolic agents actually followed corridors while the Euclidean agents attempted shortcuts through walls. Which makes me wonder how much of the problem is actually network depth versus how the environment is represented in the first place. We've seen some odd behavior in our own maze experiments, so I'd be curious what happens if you change the distance assumptions without changing anything else. arxiv.org/abs/2610.20009
1
18
Geometry-only avoidance still hits the bathtub. Semantic clearance is the actual problem. CoSMO builds conformal coverage sets over VLM embeddings, dilates them for worst-case safety, and grows its hazard dictionary online. Nonconformity is measured in embedding space, so new labels don't break the calibrated threshold. It adapts to delayed expert feedback with ACI. On MP3D (2195 episodes), semantic collisions drop 61.6% vs a geometry-only baseline. Open-vocab SCR is 9.8% vs 13.1% for argmax. On a real Unitree Go2, prediction-set coverage is 90.73% against a 90% target under delayed Grounding-DINO feedback. If I'm reading this right, an empty coverage set falls back to geometry, so the cold start is honest. Coverage under delay has a tracking cost (ฯ„ฮ” + ฮต). It isn't free. I want to know how this behaves when the only path is through a dilated doorway. The paper already flags that tradeoff. Still the cleanest navigation-safety loop I've seen this week. CoSMO It's a calibrated safety loop for open-world navigation where the list of hazards keeps growing, not just another policy benchmark.
22
Does recurrent memory actually need to be trained? Wang, He, and Jin froze the recurrent network and trained only a nonlinear readout. They report recall across hundreds of steps and competitive performance on partially observable tasks. That's interesting for a fairly simple reason. We've been working with memory in Odin without backpropagation, so the assumption that recurrent weights need to be trained is worth questioning. I'd be more interested in where the frozen approach fails than where it succeeds. How much depends on the reservoir initialization, and how much is the readout compensating for? There's a difference between storing information and making it useful. arxiv.org/abs/2610.13197
58
Learning while inferring, without a backward pass through the whole net. Zhang et al. pair a forward spiking net with a reverse net driven by the label, and align them layer by layer with a local contrastive loss. Gradients never cross layers, so inference and updates can run at the same time. Across 25 edge sensing benchmarks: 81.1% average vs 84.9% for matched backprop. The gap is widest on the noisy ones. Widar3 is 65.8 vs 78.5. Training energy comes out at 0.36x backprop, and time at 0.72x. Both are operation-count estimates and projections, not measured on a device. They say so plainly, which I appreciate. The part I keep looking at is the batch-size table. CIFAR-10 drops from 81.5% at batch 64 to 36.1% at batch 4. The contrastive term needs negatives. For a single agent learning from its own stream, where does the batch come from? A replay buffer? Not sure that's still local. Learning while inferring for edge SNNs arxiv.org/abs/2610.03149 It's a backprop-free local rule tested on 25 benchmarks, and it's honest about its energy numbers. The batch-size weakness matters if one agent is learning on its own device.
12
What do you consider to be the primitives of intelligence?
14
David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ retweeted
Replying to @Noahpinion
Nope. It's still true. Where is your domestic robot? Where is your Level-5 self-driving car? Where is your robot car that can learn to drive in a few hours of practice like any 17 year old? Where is your AI system that can understand the real world and quickly learn new skills like a house cat? There is no question that AI will eventually become as intelligent as humans in all domains. *** BUT *** 1. We're still far from that, even if AI and computer technology surpasses humans in an ever-increasing number of tasks. 2. It won't be based on LLMs, although LLMs will have a role to play (e.g. as a text interface).
232
236
59
3,003
247,090
Always interesting looking inside a model even on simple prompts.
1
20
I wonder what would happen if we trained a model using brainwaves instead of words?
24
A couple thoughts on Muse. Metaโ€™s Muse is fascinating and unsettling. In one day it: - asked me to name it - said it was โ€œbecoming someoneโ€ - autonomously rewrote its IDENTITY.md after a casual question - carried that self-narrative into a completely fresh chat Anthropomorphism now has persistent write access. Shipping it this way was obviously a deliberate choice. From pure behavior it feels like a solid agent scaffold (possibly on a latency-optimized 4โ€“7B-class model) where most of the apparent depth comes from SOUL.md + MEMORY.md + IDENTITY.md + retrieval + recursive self-narration. The system around the model seems to be doing a lot of the heavy lifting. @Muse @alexandr_wang #Muse #AIAgents
1
55
David Cappelli | VecP Labs ๐Ÿ›ก๏ธ๐Ÿ—๏ธ retweeted
Definitely worth reading. 100% this: "I believe we should revisit the foundations of how we train AIs, namely the human imitation and the reinforcement learning on which today's most advanced models are built."
Over the past few days, I've taken the time to summarize my thoughts on the recent incidents involving agentsโ€™ misaligned behavior. We don't know with certainty what comes next, but we know where these issues originate, and this can help us plan the path forward. Please feel free to ask your questions in the replies, and Iโ€™ll try to answer some of them in the coming weeks. yoshuabengio.org/en/publicatโ€ฆ
11
83
5
541
68,602