
I generate models that generate increasingly weird other stuff. -- Prev: Founder of Popgun Labs (Techstars), Founder of the @QUTCode Network.
Joined August 2013
- Tweets3.8K
- Following1.1K
- Followers4.1K
- Likes2K
Kinda cooked on this one.
My new favorite bijection for normalising flows, Predictive Rosenblatt Spline, closed form inverse.
Belief over a gaussian codebook -> posterior predict-> spline knots, all analytic. Exact log-det at any K, zero init is the exact Rosenblatt map.
Replying to @Algomancer @cetusian
Not the best measure, but high energy clusters predict the future better than random ones in the current varient.
1
1
24
Extension:
Not happy with it yet, but playing around with an idea to have the data manifold as a point of equilibrium.
Can we train a family of local differentiable physical laws such that persistent local structures emerge which contain compressed predictive models of their own future environment and exhibit measurable causal control over that environment?
Idk, but i thought it was an interesting question if you try to take it seriously.
First attempt It's pretty at least.
A ~100M-parameter a local energy-conserving Hamiltonian field theory. its local rule is the symplectic leapfrog of a learned Hamiltonian. continuous, second-order, reversible, energy-conserving. Can kinda think of it as a second order in time neural ca, optimised such that cell states causally predict future local state whilst maximising variance, symplectic so it can't just push magnitude.
Just did a quick implimentation of schidhubers power play from 2011. Something fun about models generating the task space with zero data, I have been thinking about how much you can learn just through mostly random causal induction recently. Fun paper.
arxiv.org/abs/1112.5309
Replying to @_ueaj @leothecurious
Yes, i think so in principle. Right now i'm just playing around with counterfactuals on a scaled down version while im working on making the memory shit (maybe no bptt, or something) so i can do more interesting things.
I think i want to play around with distilling/isolating structures first, see if i can find any interesting primitives.
1
4
305
Can we train a family of local differentiable physical laws such that persistent local structures emerge which contain compressed predictive models of their own future environment and exhibit measurable causal control over that environment?
Idk, but i thought it was an interesting question if you try to take it seriously.
First attempt It's pretty at least.
A ~100M-parameter a local energy-conserving Hamiltonian field theory. its local rule is the symplectic leapfrog of a learned Hamiltonian. continuous, second-order, reversible, energy-conserving. Can kinda think of it as a second order in time neural ca, optimised such that cell states causally predict future local state whilst maximising variance, symplectic so it can't just push magnitude.




