Ph.D. Principal Scientist Ex @YahooResearch. I do machine learning ∩ numerical methods ∩ SW dev.
Israel
Joined August 2012
- Tweets12.1K
- Following293
- Followers1.9K
- Likes2.7K
Pinned Tweet
🔥︎Delighted, thrilled, happy and please to share torchcurves - parametric curves in pure PyTorch. I believe it's mature enough to share here. Inspired by a two-year old paper from Yahoo.
code: github.com/alexshtf/torchcur…
docs: torchcurves.readthedocs.io
paper that inspired it: openreview.net/forum?id=M422…
Use:
pip install torchcurves
or my personal preference:
uv add torchcurves
A small library of vectorized, differentiable B-spline and Legendre curves with learnable coefficients. Handy for continuous embeddings of numerical features in your recommender system, shape-constrained models (e.g. monotonicity via non-decreasing control points) in your ad surplus optimization work, and KAN-style layers.
I set out to keep it small, pure PyTorch, readable, and easy to maintain, with only modest speed ambitions. It is reasonably fast for such a library, and the performance comes mainly from leaning on PyTorch’s vectorization and efficient algorithms (Cox-De Boor for the splines, Clenshaw for the polynomials). The vectorized code can evaluate several curves at a batch of points in one call.
Gradients flow to both the inputs and the coefficients via custom autograd. It’s not the fastest implementation out there, but it stays simple and transparent.
Would love to that it works for you, and what would make it better. Have fun using it!
OK, I don't know if the papers in my batch are AI slop. I have no way of reliably detecting AI. But they are slop. An incomprehensible mess. Someone just assembled a bunch of numbers, letters, lemmas and propositions, with a lot of symbols and indices, and asked me to read it.
Alex Shtoff retweeted
Disagree with "measure their impact". Impact if good science in many cases come many years after publication, and in extreme cases - even decades.
We should promote good science, and accept that we have no way of predicting which papers will turn out to be impactful.
Alex Shtoff retweeted
Looking for a postdoctoral fellow in deep learning with a solid theoretical background to work on learning from imbalanced data. Joint project with J.-M. Loubes in Toulouse, France. The sooner you can start, the better.
Write to [email protected]
So when will model-providers start publishing slop-scores in their model cards, i.e., benchmarks with multiple episodes, where a model is punished for creating complexity in first episodes that affects the resources required for success in the future episodes?
LaTeX rendering in @zeddotdev would make it truly the best environment for scientific computing and ML workflows. Please make this PR happen :)
github.com/zed-industries/ze…
Alex Shtoff retweeted
My dream had always been to prove beautiful theorems as the AI agents unloaded my dishwasher. Well, it went the wrong way... 😂
This year's ICLR I got no papers to review about LLMs or transformers. Great success! Finally - no more LLM / Transformer mumbo jumbo!
Hot take: to stop AI code slop, model providers should stop focusing on "bigger and better agentic tasks" but rather make their models produce less slop.
RL environments should simulate sequences of related tasks in a row, and give bad reward when many tokens or tool calls are spent on tasks far in the sequence. It means that code early in the sequence produced incomprehensible slop, that later in the sequence is hard to solve.
We don't need super-intelligent models that can deal with slop. We need super-intelligent models that are trained to avoid it in the first place.
Now that everyone is NeurIPS happy - a shameless plug for something unusual and outside the "transformers attention scaling alignment" box :)
preprint: arxiv.org/abs/2608.08003
code: github.com/alexshtf/spectral…
demo (see the animation): alexshtf-spectral-neuron-cla…
What if a neuron used symmetric matrices as weights - and an eigenvalue as its non-linearity?
In a new preprint "The Spectral Neuron", built by distilling latest blog-posts into a manuscript, I study models of the form:
𝑓(𝒙) = 𝛌ₖ(𝐀₀ + 𝚺ᵢ 𝑥ᵢ𝐀ᵢ).
Manuscript: arxiv.org/abs/2608.08003
Code: github.com/alexshtf/spectral…
Understanding the above-mentioned nice one-liner began as a series of blog posts, and gradually evolved into this preprint, which distills and extends the ideas into a coherent theoretical and empirical study.
How expressive does the model become as the matrices grow? What can we read directly from the learned matrices? Which shapes can be guaranteed by construction?
I develop the mathematics, give a practical initialization and training recipe, and test the model in scaling experiments on synthetic and real data.
This grew out of a question I kept returning to while working on ads at Yahoo: are there "simple" models that can be scalable, interpretable, and controllable at the same time?
AI disclaimer: manuscript written by yours truly, AI assisted in looking up canonical references and related work for literature review; code was heavily AI written and reviewed by yours truly.
