Alberto Olmo retweeted
#LLMs can chat & spin yarns. But can they plan to realize their yarns?
The answer: they suck even at stacking blocks!
But how can this be? Come & check out our work at #NeurIPS2022 FMDM on benchmarks for LLMs on planning. Joint work with @_aolmo_, @sarath_ssreedh and @rao2z.
Alberto Olmo retweeted
🥳🎉Dr!! 🎓 Alberto Olmo Hernandez
@_aolmo_ 🎉🥳
Analyzing Failure Modes of Inscrutable Machine Learning Models (Defense Video 👉youtu.be/-NBrYnSipJE?list=PL… )
🙏 to the Committee: @liuhuan, Baoxin Li & @sailiks
Alberto Olmo retweeted
Intrigued by the profusion of 'em "#LLM's are Zero-shot <XXX>'s" papers, we set out to see how good LLMs are at planning and reasoning about change.
tldr; off-the-shelf #GPT3 is pretty bad at these..
👉arxiv.org/abs/2206.10498
(w/ @karthikv792 @sarath_ssreedh & @_aolmo_) 1/
Alberto Olmo retweeted
#Dallemini's response to the prompt
"Engineering Professor"
*Finally* #AI #Bias is working in my favor y'all! 😂
(No, didn't do anything nefarious; @_aolmo_ ran the prompt; do see his and @niharikajain_az, @maidylm & @sailiks work that presaged this 👉sciencedirect.com/science/ar…)
Alberto Olmo retweeted
ICMYI, here is Alberto's presentation👇
cc: @_aolmo_
youtube.com/watch?v=QXP54MVK…
Alberto Olmo retweeted
📢 Imperfect ImaGANation: Our comprehensive study of how #GAN-based data augmentation techniques can exacerbate biases--is now online at Artificial Intelligence Journal🍾 (w/ @niharikajain_az, @_aolmo_, @sailiks & @maidylm) #AI #AIEthics #AIbias #AIJ
sciencedirect.com/science/ar…
Alberto Olmo retweeted
Alberto @_aolmo_ spoke about the #GPT3 to Plan project at the #ICAPS2021 workshops on FinPlan (Planning for Finance) and KEPS (Knowledge Extraction for Planning and Scheduling).
Talk video 👉 youtube.com/watch?v=QFBGKp1A…
A fun project where we studied how #GPT3 compares to custom SOTA approaches for the task of extracting plans from text--a task with applications in many industries with custom workflows written for human consumption. (With @_aolmo_ and @sarath_ssreedh)
arxiv.org/abs/2106.07131
Very interesting project where we show and compare the performances of the different #GPT3 versions against SOTA action extraction models. Results show that the transformer's most powerful flavor manages to even surpass some of them! cc: @rao2z,@sarath_ssreedh
A fun project where we studied how #GPT3 compares to custom SOTA approaches for the task of extracting plans from text--a task with applications in many industries with custom workflows written for human consumption. (With @_aolmo_ and @sarath_ssreedh)
arxiv.org/abs/2106.07131
We compared #GPT3 to current state of the art techniques in action sequence extraction and could see interesting results! The transformer network proved to be quite competitive in yet another specific task.
Work with Dr. @rao2z & Mr. @sarath_ssreedh
arxiv.org/abs/2106.07131
Alberto Olmo retweeted
ICLR attendees, tune into our oral and poster presentations about GANs exacerbating facial biases at the Synthetic Data Generation Workshop! iclr.cc/virtual/2021/worksho… (cc: @_aolmo_, @sailiks, @maidylm, @rao2z)
Alberto Olmo retweeted
Looking forward to presenting our work tomorrow (9-10am PST) @ the UDL Workshop in #ICML2020 on training DNNs to make explicable mistakes to engender user trust. New results also show that this improves robustness.
Work with @_aolmo_ & @rao2z
Paper link: bit.ly/2WnYpIk
ML classifiers, trained for accuracy, often make egregious errors that can impact a user's trust or play into societal stereotypes. We aim for classifiers that align with humans’ views on both success and failure modes. 1/5
Paper: arxiv.org/abs/2006.14841
With: @rao2z @_aolmo_
Alberto Olmo retweeted
A paper from our group that looks beyond classification accuracy, and towards differential costs of failure modes..
cc: @sailiks @_aolmo_
arxiv.org/abs/2006.14841
Replying to @tdietterich @ylecun
Beyond worst-case robustness, more generally we should also consider the differential costs of failure modes rather than just the top-k accuracy (that has been the driver for far too long). A single egregious misclassification can ruin the trust in a near 99.999% accurate system
Alberto Olmo retweeted
Move over @Snapchat gender-swap selfie lens. We've trained our own CycleGAN-based STEM professor selfie lens to fix your face so you look more like a real academic.
So what if we look a little more manly in the process, right ladies?


























