Zhenzhen Weng (@JenWeng4) profile picture

Perception@Waymo. PhD @Stanford.BSCS@CMU🚀. 💻 Ex-intern @Waymo @Adobe.

Palo Alto, CA
Joined June 2021
Photo by Zhenzhen Weng (@JenWeng4)Photo by Zhenzhen Weng (@JenWeng4)Photo by Zhenzhen Weng (@JenWeng4)
Just published in @ScienceAdvances, our work demonstrating the ability of AI and 3D computer vision to produce automated measurement of human interactions in video data from early child development research -- providing over 100x time savings compared to human annotation and enabling quantitative, big data studies. We use our method, HARMONI, to characterize longitudinal trends in infant and toddler interaction with caregivers, in over 500 hours of video data. Work led by @JenWeng4 together with co-PI @SandersMDMPH and @K_L_Humphreys, and with a great interdiscplinary team including Laura Bravo Sanchez, @bergelsonlab, @akanazawa, @StanfordCERC, and many others! science.org/doi/10.1126/scia…
2
13
53
5,899
1/3 Today, an anecdote shared by an invited speaker at #NeurIPS2024 left many Chinese scholars, myself included, feeling uncomfortable. As a community, I believe we should take a moment to reflect on why such remarks in public discourse can be offensive and harmful.
171
546
205
3,481
1,021,722
Our lab at Stanford has postdoc openings! Candidates should have expertise and interests in one or multiple of: multimodal large language models, video understanding (including video-language models), AI for science / biology, or AI for surgery. Please send inquiries by email and see marvl.stanford.edu for more information.
4
36
191
36,649
Zhenzhen Weng retweeted
🤗First benchmark on multimodal judge’s feedback for text-to-image generation!! 🏃Come and pick up your personal advice and package to choose the best judge to fine-tune your diffusion model 👉 mj-bench.github.io Paper: huggingface.co/papers/2407.0… Code: github.com/MJ-Bench/MJ-Bench
🌟NEW Paper Alert 🌟 👩‍⚖️MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? (mj-bench.github.io/) 🧐Also wonder about the best judge model to provide feedback for your diffusion models? We evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias. Key findings: 👉1. While closed-source VLM judges typically perform better, smaller CLIP-based models offer better text-image alignment and image quality feedback due to extensive pre-training on text-vision corpus. Conversely, VLMs provide more accurate feedback on safety and generation bias, thanks to their stronger reasoning capabilities. 👉2. VLM judges can provide more accurate and stable feedback in natural language (e.g. Poor, Average, Good) than numerical scales. Led by @ZRChen_AISafety, Yichao Du, Zichen Wen, @AiYiyangZ. arxiv.org/pdf/2407.04842
4
19
3,623
Zhenzhen Weng retweeted
🌟NEW Paper Alert 🌟 👩‍⚖️MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? (mj-bench.github.io/) 🧐Also wonder about the best judge model to provide feedback for your diffusion models? We evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias. Key findings: 👉1. While closed-source VLM judges typically perform better, smaller CLIP-based models offer better text-image alignment and image quality feedback due to extensive pre-training on text-vision corpus. Conversely, VLMs provide more accurate feedback on safety and generation bias, thanks to their stronger reasoning capabilities. 👉2. VLM judges can provide more accurate and stable feedback in natural language (e.g. Poor, Average, Good) than numerical scales. Led by @ZRChen_AISafety, Yichao Du, Zichen Wen, @AiYiyangZ. arxiv.org/pdf/2407.04842
3
36
4
138
20,577
If you’re in Davos, we just started giving a tutorial on Gaussian Splatting at 3DV. With @GKopanas @Snosixtytwo @antoine_guedon 3dgstutorial.github.io/ 3dvconf.github.io/2024/tutor…
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis dynamic3dgaussians.github.io… We model the world as a set of 3D Gaussians that move & rotate over time. This extends Gaussian Splatting to dynamic scenes, with accurate novel-view synthesis and dense 3D trajectories.
2
10
2
71
8,033
Thanks @_akhaliq for sharing our work! Letting LLM be an agent and long-form videos as an environment, and allowing LLM to interact with videos and decide where to look iteratively, we achieve SoTA zero-shot performance and show potential on processing extremely long videos!
VideoAgent Long-form Video Understanding with Large Language Model as Agent Long-form video understanding represents a significant challenge within computer vision, demanding a model capable of reasoning over long multi-modal sequences. Motivated by the human cognitive
1
12
36
23,810
Zhenzhen Weng retweeted
Are you hiring top AI talent? Here is a list of Ph.D. students affiliated with @StanfordAILab who are on the industry and academic job markets this year! This list showcases diverse research areas and 41% of these graduates are URMs! Check it out: ai.stanford.edu/blog/sail-gr…
4
48
214
54,718
What are differences between image datasets? (e.g. ImageNet & ImageNetv2) Errors by one model vs. another? (e.g. CLIP & ResNet) Correct vs. incorrect predictions? VisDiff can answer by describing differences in image sets w/ language. Work led by @Zhang_Yu_hui and @lisabdunlap!
[1/5] Introducing VisDiff - an #AI tool that describes differences in image sets with natural language. VisDiff can summarize model failures, compare models, find nuanced dataset differences, discover what makes an image memorable, and so much more! understanding-visual-dataset…
2
10
4,116
Zhenzhen Weng retweeted
Sneak peek in video GenAI projects we’re working on @AdobeResearch : 1/3 compositing videos in 3D with NeRF
#ProjectSceneChange You’ll be able to put yourself in any scene including your artwork. #AdobeMAX #CommunityxAdobe
4
24
7,795
Zhenzhen Weng retweeted
Check out Adobe's Project Scene Change The interesting experimental #AI tech automatically composites an actor from one shot into the environment from another without the need for #rotoscoping or #cameratracking cgchannel.com/2023/10/sneak-… #compositing #VFX #motiongraphics
8
31
5,528
Have videos of your tennis practice and wish you can put your own motion in 3D? 🎾 👟 🏋🏻 #CVPR2023 We present, NeMo, a 3D motion recovery method that is more accurate by leveraging information shared across multiple instances/repetitions! 👇🏻Resources in 🧵
2
19
1
60
9,252
Zhenzhen Weng retweeted
Check out our #CVPR2023 paper on 3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels youtu.be/vXGPW2nDHZ4 Huge shout out to @JenWeng4 who interned in our team last summer and did all the work!
1
1
4
2,466
SO much potential for this:
50
217
35
1,562
189,342
Zhenzhen Weng retweeted
(1/8) Can you diagnose and rectify a #vision model using #language? Check our work in #ICLR2023! Our analysis reveals when and how text embeddings can be used as a proxy for image embeddings to debug vision models. Paper: arxiv.org/abs/2302.04269 Code: github.com/yuhui-zh15/drml
1
17
3
102
26,533
Zhenzhen Weng retweeted
Releasing 🌟nerfstudio🌟, a plug-and-play python library to easily create your own NeRFs! @nerfstudioteam is a contributor friendly open-source repo with a realtime web viewer that makes it easy to make cool videos 📽️ github.com/nerfstudio-projec… docs.nerf.studio #nerfstudio 1/
19
427
46
2,497
Zhenzhen Weng retweeted
A quick thread on "How DALL-E 2, Imagen and Parti Architectures Differ" with breakdown into comparable modules, annotated with size 🧵 #dalle2 #imagen #parti * figures taken from corresponding papers with slight modification * parts used for training only are greyed out
19
461
28
2,046
Zhenzhen Weng retweeted
Very excited to present our paper @BMVCconf this Wednesday (day 3, session 3)! Presenting FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation, a labor of love with @HNSurgeon and @syeung10. #BMVC2021
1
5
17
Zhenzhen Weng retweeted
Excited to teach CS 523 + talk w/ amazing folks like @syeung10 & Jen Weng & @HNSurgeon, @liu_xingtong & @MathiasUnberath, Tae Soo Kim & @GregGdhager, @RyutaroT92, @meenakshik93 & @mertrory, @AdrianDalca, @dc_moyer, @NanWu__, @aktanwani, @AndreEsteva!! #MedicalAI #AcademicTwitter
Enroll in a new spring qtr course: "Research Seminar in Computer Vision and Healthcare" (CS 523) by @julia_gong on Wednesdays, 11:30am-12:30pm PST. Class starts on 3/31, learn more here: web.stanford.edu/class/cs523… Open to @Stanford & affiliates to audit. #AI #Healthcare #Stanford
1
7
26