Rom_Ilb retweeted
I will be at #ICML2024 in Vienna next week to present SAMformer! 🇦🇹🎻
🎤 Oral: Tues. 23 11h15 by @ilbert_romain (a must-see)
🪧 Poster: Tues. 23 11h30
📑 arxiv.org/pdf/2402.10198
🖥 github.com/romilbert/samform…
Happy to meet new folks🥳
w. @IevgenRedko @_GPaolo @ilbert_romain
🎉🥳SAMformer has been accepted as an 𝐨𝐫𝐚𝐥 𝐩𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 at #ICML2024!
📑 arxiv.org/pdf/2402.10198
🖥 github.com/romilbert/samform…
🧵 by @IevgenRedko below
New🌟: SAMformer is competitive with the biggest foundation model #MOIRAI @CaimingXiong x.com/CaimingXiong/status/17…
🚀 Very happy to announce that SAMformer was accepted as an Oral at @icmlconf 2024!
Thanks again to all my amazing co-authors.
See you in Vienna to discuss 🇦🇹
🎉🥳SAMformer has been accepted as an 𝐨𝐫𝐚𝐥 𝐩𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 at #ICML2024!
📑 arxiv.org/pdf/2402.10198
🖥 github.com/romilbert/samform…
🧵 by @IevgenRedko below
New🌟: SAMformer is competitive with the biggest foundation model #MOIRAI @CaimingXiong x.com/CaimingXiong/status/17…
Very happy to have presented my paper "Data Augmentation for Time Series Classification: An Experimental Study" at the #ICDE conference in Utrecht. It was a great opportunity to meet and connect with members of the Time Series community.
🎉 Excited to announce that our paper "Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention" has been accepted at @icmlconf 2024!
The paper : arxiv.org/abs/2402.10198
Our code : github.com/romilbert/samform…
It was a great pleasure to attend the defense of such a good thesis as that of @MathisPetrovich , the newly minted Ph.D 🎉
Congratulations to @MathisPetrovich on successfully defending his PhD. This work pioneered the use of transformers for modeling 3d human motion. ACTOR, TEMOS, TEACH, TMR, and more. Great work!! 🎊🥂🎉 Co-advisor: the awesome @gulvarol!
Rom_Ilb retweeted
Transformers dominate the ML field but are surprisingly bad in time series forecasting. How come? Our team studied the reasons behind this and provided a solution in a new pre-print:
Paper: arxiv.org/abs/2402.10198
Code: github.com/romilbert/samform…
📄 'Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention', is available on arXiv! We challenged the dominance of linear models, using a transformer with a single head and one linear layer, improving SOTA !
Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention ift.tt/DvfoH1s
Rom_Ilb retweeted
Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention ift.tt/DvfoH1s
Rom_Ilb retweeted
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo.
We are collaborating to figure out the details. Thank you so much for your patience through this.
📄 « Breaking Boundaries » is now available on arXiv ! I will be giving a talk on the robustness of DL models during the ACM-CCS conference. Looking forward to discussing our findings.
Link to the ArXiv paper : arxiv.org/abs/2311.09790
#DeepLearning #Robustness #ACMCCS #AI #Noise
A major step forward, especially for people who have lost the ability to speak. linkedin.com/posts/mit-techn…
Rom_Ilb retweeted
Today Elon Musk held a Twitter Space to discuss xAI.
Here is a recap of what was discussed for those who missed it:
- The founding team was on hand to introduce themselves, and I must say it is an impressive team with an impressive background. They had very strong backgrounds with Deep Mind, OpenAI, Google, Tesla, etc.
- Elon Musk said the goal with xAI is to build a good AGI (artificial general intelligence) with the purpose of understanding the universe.
- Musk said that the safest way is to build an AGI that is ‘maximum curious’ and ‘truth curious,’ and to try and minimize the error between what you think is true and what is actually true.
- For truth-seeking super intelligence humanity is much more interesting than not humanity, so that’s the safest way to create one. Musk gave the example of how space and Mars is super interesting but it pales in comparison to how interesting humanity is.
- Musk said there is so much that we think we understand but we don’t in reality. There are a lot of unresolved questions. For example, there are many questions that remain about the nature of gravity, and why there is not massive evidence of aliens. He said he has seen no evidence of aliens whatsoever so far. He went further into the Fermi Paradox and how it's possible that other consciousness may not exist in our galaxy.
- If you ask today’s advanced AIs technical questions, you just get nonsense, so Musk believes we are really missing the mark by many orders of magnitude and that needs to get better.
- xAI will use heavy computing, but the amount of ‘brute force’ will become less as they become to understand the problem better.
- Co-Founder Greg Yang said that the mathematics they find at xAi could open up new perspectives to existing questions like the 'Theory of Everything.'
- Elon stated that you can't call anything AGI until the computer solves at least one fundamental question.
- He said that from his experience at Tesla, they have over complicated problems. “We are too dumb to realize how simple the answers really are," he said. "We will probably find this out with AGI as well. Once AGI is solved, we will look back and think, why did we think it would be so hard.”
- They are going to release more information on the first release of xAI in a couple more weeks.
- Elon Musk said that xAI is being built as competition to OpenAI, when asked by @krassenstein.
- The goal is to make xAI a useful tool for consumers and businesses and there is value in having multiple entities and competition. Elon said that competition makes companies honest, and he’s in favor of competition.
- Musk said every organization doing AI has illegally used Twitter’s data for training. Limits had to be put on Twitter because they were being scraped like crazy. Multiple entities were trying to scrape every tweet ever made in a span of days. xAI will use tweets as well for training.
- At some point you run out of human-created data. So eventually AI will have to generate its own content and self-access that content.
- Answering a question from @alx, Musk said there is a significant danger in training AI to be politically correct or training it not to say what it thinks is true, so at xAI they will let the AI say what it believes to be true, and Musk believes it will result in some criticism.
- Musk said it’s very dangerous to grow an AI and teach it to lie.
- Musk said he would accept a meeting with Kamala Harris if invited. He said he’s not sure if Harris is the best person to be the AI czar, but agrees we need regulatory oversight.
- Musk believes that China too will have AI regulation. He said the CCP doesn’t want to find themselves subservient to a digital super intelligence.
- Musk believes we will have a voltage transformer shortage in a year and electricity shortage in 2 years.
- xAI will work with Tesla in multiple ways and it will be of mutual benefit. Tesla’s self-driving capabilities will be enhanced because of xAI.
- According to Musk, the proper way to go about AI regulations is to start with insight. If a proposed rule is agreed upon by all or most parties then that rule should be adopted. It should not slow things down for a great amount of time. A little bit of slowing down is OK if it's for safety.
- Musk thinks that Ray Kurzweil's prediction of AGI by 2029 is pretty accurate, give or take a year.
I've love to hear everyone's thoughts on where you think xAI will go.
📚Engaging paper introducing Dilated Attention in the context of Transformers.
🧬 This new model shows promising potential for applications in genomic data modeling.
Link to the paper : arxiv.org/pdf/2307.02486.pdf
#NLP #LLMs #AI #research #Transformer #MicrosoftResearch
Rom_Ilb retweeted
Dans ce chaos, la réaction bouleversante, héroïque et au fond tellement normale d'une femme seule hurlant "S'il vous plaît pas l'école !!!".
#emeutes


















