Lu Jiang (@roadjiang) header
Lu Jiang (@roadjiang) profile picture

Research Scientist @GoogleAI #GoogleResearch. Adjunct Faculty @CarnegieMellon.

Mountain View, CA
Joined October 2010
Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)Photo by Lu Jiang (@roadjiang)
How do we generate videos on the scale of minutes, without drifting or forgetting about the historical context? We introduce Mixture of Contexts. Every minute-long video below is the direct output of our model in a single pass, with no post-processing, stitching, or editing. 1/4
21
96
25
600
173,297
Glad to share Seaweed-7B, a cost-effective foundation model for video generation. Our tech report highlights the key designs that significantly improve compute efficiency and performance given limited resources, achieving comparable quality against other industry-level models. To unleash the power of the foundation model, Seaweed-7B further enables a wide range of downstream applications including image-to-video generation, human video generation, subject-consistent video generation, video-audio joint generation, long video generation and storytelling, real-time generation, super-resolution generation, camera controlled generation. Check out our webpage and report for more details: Webpage: seaweed.video/ Paper: seaweed.video/seaweed.pdf It's a wonderful journey of the last year. Thanks to all teammates for their contributions, sincerely.
32
94
18
510
77,640
Lu Jiang retweeted
Synthetic Video Enhances Physical Fidelity in Video Synthesis A turtle swimming in a green background. + video matting illustration
4
14
4
99
16,725
We propose Long Context Tuning (LCT) for scene-level video generation to bridge the gap between current single-shot generation and real-world narrative video productions. Homepage: guoyww.github.io/projects/lo… Report: arxiv.org/abs/2503.10589
4
23
2
104
46,924
Want the deep dive? • arXiv: arxiv.org/abs/2501.06173 • Project Page: videoauteur.github.io See how VideoAuteur + CookGen are shaping long narrative video generation. Big shout out to my co-authors and advisors: @fncheng2333 @liangkegui @YuilleAlan @roadjiang
1
2
499
Lu Jiang retweeted
Seaweed APT Diffusion Adversarial Post-Training for One-Step Video Generation Existing diffusion and autoregressive generative models require repeated neural network evaluations. It is extremely slow for the high-resolution video generation task, as a few-second video can take many minutes to generate. Our work is the first to demonstrate the generation of an entire video using a single neural function evaluation (1NFE) by using our proposed adversarial post-training technique. Our model generates 2 seconds of 1280x720 24fps videos in real-time. We showcase some of the results below:
9
33
7
201
21,247
We introduce W.A.L.T, a diffusion model for photorealistic video generation. Our model is a transformer trained on image and video generation in a shared latent space. 🧵👇
48
244
70
1,224
431,418
(1/6)We propose a new metric, GEP, that utilizes fine-grained self-presentation attributes to study how gender is presented differently in text-to-image models. Paper:arxiv.org/abs/2302.03675 Website(Data & Code):salt-nlp.github.io/GEP/ (with @roadjiang, Greg Turk, @Diyi_Yang)
1
9
21
8,262
Improved Masked Image Generation with Token-Critic Outperforms recent diffusion models and GANs in terms of the trade-off between generated image quality and diversity. arxiv.org/abs/2209.04439
1
16
1
63
cleanlab 2.0 is here! cleanlab.ai/blog/cleanlab-2/ cleanlab identifies errors in datasets, tracks dataset quality, trains reliable models with noisy data, and helps curate quality datasets… often in just one line of code.
2
6
9
32
To better understand the impact of noisy labels on #MachineLearning model training, we are announcing MentorMix, a new method to mitigate the impact of noisy labels, as well as a benchmark and dataset on real-world label noise. Learn more about it at: goo.gle/2Q8K9j7
5
171
7
458
Lu Jiang retweeted
Synthetic vs Real: Deep Learning on Controlled Noise. arxiv.org/abs/1911.09781
1
1
8
Check out our work on Predicting Future Person Activities and Locations in Videos. Poster 133 today morning starting 10am, followed by Demo at the Google booth starting 3pm! #CVPR2019 More: next.cs.cmu.edu/ with Junwei Liang, Lu Jiang, Alex Hauptmann and @drfeifei
1
26
4
90