@harpomaxx@sigmoid.social (@harpolabs) header
@harpomaxx@sigmoid.social (@harpolabs) profile picture

HARPO (AKA Carlos A. Catania). A Nerdo with decent social skills? #DataScience #Algorithms. English, Spanish and Français. From Mendoza Republic. South America

Argentina
Joined July 2014
Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)Photo by @harpomaxx@sigmoid.social (@harpolabs)
Since Mixture of Expert (MoE) LLMs are all the rage as of this weekend, thanks to the Mixtral-8x-7B release, here's a quick explainer. The figure below shows the architecture behind the Switch Transformer (arxiv.org/abs/2101.03961), a great intro to MoEs. The model depicted in this figure uses 1 expert per token with 4 experts in total. Mixtral-8x-7B, on the other hand, consists of 8 experts and uses 2 experts per token. Why MoEs? Combined, the 8 experts in a 7B model like Mixtral are still ~56B parameters. (Actually, it's less than 56B, because the MoE approach is only applied to the MoE layers, not the self-attention weight matrices. So, it's likely closer to 40-50B parameters.) However, since the router reroutes the tokens such that only 7B parameters (instead of all 56B) are used at a time for the forward pass, the training (and especially inference) will be much faster compared to the traditional non-MoE approach. If you read my AI and Open Source in 2023 article (magazine.sebastianraschka.co…) approx. 2 months ago, I mentioned that "It will be interesting to see if MoE approaches can lift open-source models to new heights in 2024". It looks like Mixtral started this trend early, and I am sure that this is just the beginning :).
15
251
12
1,267
179,391
Today ⁦@verovaleros⁩ gave the talk “Four Key Problems in OSINT for Cyber Threat Intelligence” in ⁦@enisa_eu⁩ CTI conf. #cti
4
13
1,012
🚨New Paper Alert! In our latest research we used LLMs as reinforcement learning red team agents: "Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments" arxiv.org/abs/2308.12086 with @ondrej_lukas @harpolabs @eldraco #AI #LLM #ML #infosec
1
15
37
4,520
Replying to @predict_addict
I'm talking on this at #PositConf2023 in Chicago (posit.co/conference/) in about 2 weeks!
2
2
16
1,776
It’s time to admit that we were all overly optimistic during COVID, thinking that organizations will finally understand the “remote work” culture. Not only is back to office, but also back to paper and physical meets. We thought we could make a change and we lost. #fail
4
13
1,033
We are happy to announce that our paper with @eldracote was accepted in ESORICS 2023: "The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning" arxiv.org/abs/2308.16562 @StratosphereIPS @ctu_cs #ai #adversarial #MachineLearning #malware #security
1
6
18
1,430
Thanks for the kits and the workshop! Help us get into the community and have fun!
Thanks all who came along to my workshop. Totally sold out and more wanted to join. I hope they enjoy their new #lora satellite stations and the @tiny__GS community. Bonus thanks to my fellow @TOG_Dublin crew for the helping hand too #cccamp23
3
12
1,504
Desde el nuevo Media Lab de la @UNCUYO acabamos de lanzar un podcast dedicado a la inteligencia artificial. Se llama "Inteligencia natural" y acá están los primeros 3 episodios. open.spotify.com/show/4oeyHh…
2
4
2
6
941
Los Harpo están conquistando el mundo @harpolabs 😂😂#LiliFiallo muy interesante arte y IA #podcasts
Desde el nuevo Media Lab de la @UNCUYO acabamos de lanzar un podcast dedicado a la inteligencia artificial. Se llama "Inteligencia natural" y acá están los primeros 3 episodios. open.spotify.com/show/4oeyHh…
2
1
199
Join us in Las Vegas for our @BlackHatEvents hands-on Advanced Malware Traffic Analysis training with @verovaleros! Learn how to detect botnets, ransomware, lateral movement, and more! Learn to detect attacks w/machine learning tools! #BlackHat #BHUSA blackhat.com/us-23/training/…
3
4
593
Let's gooo! 🔥 We're happy to introduce two new official Space Templates in collaboration with @posit_pbc: Build and share your R and Python Shiny apps directly on the Hub! 🥳
We’re so excited to announce that you can deploy both #RStats and #Python Shiny apps on @huggingface Spaces using our Docker templates. Shiny is a great front-end for AI models, and we can’t wait to see what people build with this integration. shiny.rstudio.com/blog/shiny…
35
97
49,249
Mornings are made for reading, nights are made for coding.🤓 ... Except when you have kids 🤷🏻‍♀️
1
2
123
I believe we need open-source alternatives to ChatGPT for more transparency, inclusivity, accountability and distribution of power. Excited to introduce HuggingChat, an open-source early prototype interface, powered by OpenAssistant, a model that was released a few weeks ago. – at Normandy Shores, FL
56
377
53
1,812
235,934
¡El #SAHTI2023 de las #52JAIIO @jaiio_oficial se prepara! @bmassare desde CABA, Karina Bianculli desde MdP y yo desde Cba, armando un Simposio de Historias, Tecnología e Informática que va a estar muy bueno. ¡Hasta el 1 de mayo tenés tiempo de presentar el resúmen! Te esperamos.
3
10
668
Another collaboration with visual artist Lili Fiallo. Using #dreambooth we fine tuned #stablediffusion to incorporate Lili's toy art style. incredibly simple using diffusers and gradio libraries. instagram.com/fiallolili/ 3e2bb3f1180757bdee.gradio.li…
2
213
We have planned outages in our datasets server today (20/04/23) between 9:30 and 12:00 CET. We will let you know when things are back online. ⚙️
2
1
199