> 4. Maybe build a functioning sandbox first hmmm?!
For sticklers:
1. Good luck exfiltrating 1T+ weights at <1 byte per hour
2. Who recieved the transmitted bits on the other end?
3. Who decodes the message? (you must have encoding because noisy channel)
4. Maybe build a functioning sandbox first hmmm?!
Never go full armchair
8
509
My team in GDM Frontier AI is hiring (Mountain View). If you're a researcher interested full duplex modeling, multimodal LLMs (Gemini), modality gap, joint speech/text modeling, PGMs, streamable generative models, and representation learning for language modeling -- DM me!
Replying to @natolambert @finbarrtimbers
yea... i'm going to go out on a limb and say ecosystem tech debt is the real problem.
1
2
176
How to build *TruthGPT*? I listened to a talk by the legendary @johnschulman2. It's densely packed with lots of deep insight. Key takeaways:
- Supervised finetuning (or behavior cloning) makes the model prone to hallucination, while RL mitigates it.
- NLP is far from done!
1/🧵
5
752
Replying to @veydpz_public
Not a comment on the paper (excited to read it!) but a competitive Tacotron 2 baseline should be within the error bars with the ground truth on LJSpeech with character inputs. It's a pretty low-expressivity dataset, so it's easy to model.
The quality is close to, but below Tacotron/WaveRNN. This paper is a starting point, showing that this structure is workable for direct waveform generation. With some tuning we see no reason why we can't meet or exceed this baseline. Who wants to help? 🙂 (7/7)
Modeling K samples in parallel per decoder step yields significantly faster sampling time than a sample-autoregressive vocoder like WaveRNN, although it is slower than fully parallel vocoding as with WaveGlow/FloWaveNet. (6/7)
1












