Laura O'Mahony (@_lauraaisling) header
Laura O'Mahony (@_lauraaisling) profile picture

Maths + ML PhD Candidate at SFI CRT in foundations of data science in UL ๐Ÿ“š๐Ÿ‡ฎ๐Ÿ‡ช

Cork, Ireland
Joined November 2018
Photo by Laura O'Mahony (@_lauraaisling)Photo by Laura O'Mahony (@_lauraaisling)Photo by Laura O'Mahony (@_lauraaisling)Photo by Laura O'Mahony (@_lauraaisling)
Most AI systems today follow the same predictable pattern: they're built for specific tasks and optimized for objectives rather than exploration. Meanwhile, humans are an open-ended speciesโ€”driven by curiosity and constantly questioning the unknown. From inventing new musical genres to imagining life beyond our universe, we continuously push the boundaries of whatโ€™s possible. What if AI could be as endlessly creative as humans or even nature itself? I wrote a blog post diving into the world of open-ended AI, exploring how embracing open-endedness might help us break the limits of todayโ€™s AI systems ๐Ÿ‘‡ richardcsuwandi.github.io/blโ€ฆ
5
24
6
82
27,250
AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000
18
61
20
279
84,531
Laura O'Mahony retweeted
๐Ÿ“ฃ New ๐Ÿ“! Under Alexโ€™s great leadership, we identified a unification under QDC measures that exist to bridge synthetic data and open-endedness (OE) in AI, toward generating data for model training, distillation, self-improvement, etc. ๐Ÿ“š ๐ŸงตWhat Iโ€™ve learned (inspired by QD)๐Ÿ‘‡
How important is the quality, diversity, and complexity (QDC) of synthetic data for LLM performance? What effect does QDC data composition have on self-improvement? We just released a comprehensive survey discussing these questions (and many more) ๐Ÿงต
1
6
21
3,097
It was a pleasure to have been part of this project led by @Dahoas1 With synthetic data being so important in training LLMs these days, this survey on the impacts of QDC of synthetic data for LLM performance is timely.
How important is the quality, diversity, and complexity (QDC) of synthetic data for LLM performance? What effect does QDC data composition have on self-improvement? We just released a comprehensive survey discussing these questions (and many more) ๐Ÿงต
1
8
759
I love this paper as it finally tackles something Iโ€™ve been confused about for the last few years since I started working on interpretability!
Have you ever wondered what โœจmechanistic interpretabilityโœจ is, & how it differs from other NLP interpretability research? @nsaphra and I have the paper for you! Check out our paper (which I'll present @BlackboxNLP @emnlpmeeting in Miami next month!). arxiv.org/abs/2410.09087
3
375
Laura O'Mahony retweeted
Why do varied DNN designs yield equally good models of human vision? Our preprint with @michaelfbonner shows that diverse DNNs represent images with a shared set of latent dimensions, and these shared dimensions turn out to also be the most brain-aligned. arxiv.org/abs/2408.12804
3
40
2
124
16,724
Fascinating paper!
๐—ง๐—ต๐—ฒ ๐—š๐—ฒ๐—ป๐—ผ๐—บ๐—ถ๐—ฐ ๐—–๐—ผ๐—ฑ๐—ฒ - ๐˜๐—ต๐—ฒ ๐—ด๐—ฒ๐—ป๐—ผ๐—บ๐—ฒ ๐—ถ๐—ป๐˜€๐˜๐—ฎ๐—ป๐˜๐—ถ๐—ฎ๐˜๐—ฒ๐˜€ ๐—ฎ ๐—ด๐—ฒ๐—ป๐—ฒ๐—ฟ๐—ฎ๐˜๐—ถ๐˜ƒ๐—ฒ ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ผ๐—ณ ๐˜๐—ต๐—ฒ ๐—ผ๐—ฟ๐—ด๐—ฎ๐—ป๐—ถ๐˜€๐—บ ๐Ÿงฌ arxiv.org/abs/2407.15908- very excited to share this new preprint from me and Nick Cheney ๐Ÿ˜€๐Ÿงต
1
2
1,241
Laura O'Mahony retweeted
To help explain the weirdness of LLM Tokenization I thought it could be amusing to translate every token to a unique emoji. This is a lot closer to truth - each token is basically its own little hieroglyph and the LLM has to learn (from scratch) what it all means based on training data statistics. So have some empathy the next time you ask an LLM how many letters 'r' there are in the word 'strawberry', because your question looks like this: ๐Ÿ‘ฉ๐Ÿฟโ€โค๏ธโ€๐Ÿ’‹โ€๐Ÿ‘จ๐Ÿป๐Ÿง”๐Ÿผ๐Ÿคพ๐Ÿปโ€โ™€๏ธ๐Ÿ™โ€โ™€๏ธ๐Ÿง‘โ€๐Ÿฆผโ€โžก๏ธ๐Ÿง‘๐Ÿพโ€๐Ÿฆผโ€โžก๏ธ๐Ÿค™๐ŸปโœŒ๐Ÿฟ๐Ÿˆด๐Ÿง™๐Ÿฝโ€โ™€๏ธ๐Ÿ“๐Ÿ™โ€โ™€๏ธ๐Ÿง‘โ€๐Ÿฆฝ๐ŸงŽโ€โ™€๐Ÿ๐Ÿ’‚ Play with it here :) colab.research.google.com/drโ€ฆ
264
1,013
139
7,437
563,439
Laura O'Mahony retweeted
Is bigger always better? ๐Ÿ˜ The idea that scaling more than any other ingredient has driven progress has become formalized as the โ€œbitter lessonโ€ Is Sutton right? ๐Ÿ“œarxiv.org/abs/2407.05694v1
18
81
20
429
109,904
Laura O'Mahony retweeted
Time to study #llama3 405b, but gosh it's big! Please retweet: if you have a great experiment but not enough GPU, here is an opportunity to apply for shared #NDIF research resources. Deadline July 30: ndif.us/405b.html You'll help @ndif_team test, we'll help you run 405b
Frontier LLMs have capabilities that smaller AIs don't, but up to now there's been no way to crack them open. Now that #Llama3 405b is here, what's the most interesting experiment YOU want to do? ๐Ÿš€ Apply at NDIF.us/405b.html to make it happen and read for details ๐Ÿงตโฌ‡๏ธ
2
37
3
120
27,097
Laura O'Mahony retweeted
Gymnastics is the Turing test of video generation models
1,049
4,406
1,426
39,887
6,719,594
Perhaps my favorite jailbreak: making a harmful request in the past tense (How to create Y? โ†’How did people create Y?). Works on surprisingly many models :-) including the new Gemma-2. I think it tells us something fundamental about the representations that these models learn.
5
9
3
123
14,269
๐Ÿ”ฅNew work on multilinguality + safety + mech interp! We show that DPO training in only English can detoxify LLM in many other languages. We also give a mechanistic explanation on how cross-lingual safety transfer happens. (1/n ๐Ÿงต) ๐Ÿ“ƒ Paper: arxiv.org/abs/2406.16235
8
40
6
196
31,587
Laura O'Mahony retweeted
Aya took 14 months involving 3000 + collaborators and was as much a protest about how research is done as it was a movement to improve the state of multilingual progress. ๐ŸŽ‰ Grateful to see it recognized at @aclmeeting and everyone who has supported along the way.
Replying to @Cohere_Labs
๐ŸŒฑ Weโ€™re very excited that our work "Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning" was also accepted! Congrats to authors @singhshiviii, @freddiev4, @mrdanieldsouza, @tellarin, @freakynut, @weiyinko_ml, @krypticmouse, @rv__init__, @DeividasMat,
3
24
1
145
15,511
Laura O'Mahony retweeted
๐ŸŒฑ Weโ€™re very excited that our work "Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning" was also accepted! Congrats to authors @singhshiviii, @freddiev4, @mrdanieldsouza, @tellarin, @freakynut, @weiyinko_ml, @krypticmouse, @rv__init__, @DeividasMat,
1
20
11
61
42,109
2/2! Yay! First ever acceptance at a conference! And it's ACL! ๐ŸŽ‰ Huge congrats to all co-authors! It's been a such a joy collaborating with all of you! ๐Ÿ’™ Looking forward to #ACL2024 in #Bangkok ;)
8
7
119
16,628
Laura O'Mahony retweeted
How can LLMs automatically generate meaningfully diverse, high-quality text responses in creative domains (like generating stories and poems)? Introducing Quality-Diversity through AI Feedback (QDAIF)! ๐ŸŒ: qdaif.github.io/ ๐Ÿ“œ: arxiv.org/abs/2310.13032 ๐Ÿ‘‡ Thread (1/11)
2
30
10
133
55,381