1/n AI's Next Leap: Mastering Abstraction and Reasoning Imagine a world where machines can not only understand our language but also grasp the underlying logic of our thoughts, effortlessly solving problems that demand reasoning and adaptability. This is the ambitious goal of Artificial General Intelligence (AGI), and the Abstraction and Reasoning Corpus (ARC) stands as a formidable challenge on this path. ARC tasks, with their focus on core knowledge systems and few-shot learning, have proven difficult even for the most advanced AI models. Yet, a new contender has emerged, wielding the power of language and learning in a way never seen before. Enter CodeIt, a self-improving language model that cracks the code of ARC, achieving state-of-the-art performance and offering a glimpse into the future of intelligent machines. One of the key difficulties with ARC lies in its demand for reasoning and generalization beyond simply memorizing patterns. Existing neural network approaches, such as those using GPT-4, often fall short in this regard. CodeIt addresses this by adopting a neuro-symbolic approach. Instead of directly predicting outputs, it learns to generate programs within a domain-specific language (DSL) designed for ARC tasks. This allows the model to represent and manipulate knowledge in a structured and symbolic way, enabling more effective reasoning and generalization. Furthermore, the sparse reward problem poses a significant hurdle for program synthesis methods like Expert Iteration (ExIt). Finding correct programs is challenging due to the lack of positive feedback, hindering learning progress. CodeIt ingeniously tackles this issue through hindsight relabeling. By learning from all generated programs, not just the successful ones, the model gains valuable information about the task structure and program behavior, significantly improving sample efficiency. However, hindsight relabeling introduces a new challenge: catastrophic forgetting. As the data distribution changes with each iteration, the model risks losing previously acquired knowledge. To mitigate this, CodeIt employs prioritized experience replay. By prioritizing learning from experiences that solve actual ARC tasks, the model retains and builds upon its successful strategies, ensuring continuous improvement. The experimental results demonstrate the effectiveness of CodeIt. It outperforms all existing baselines, including both neural and symbolic approaches, achieving state-of-the-art performance on the ARC evaluation set. The analysis of generated programs reveals that CodeIt learns efficient and effective solutions, often finding shorter programs and employing different primitives compared to baselines. Moreover, the model exhibits program refinement capabilities, continuously improving solutions over time. Ablation studies further confirm the importance of each component within CodeIt. Removing ExIt, hindsight relabeling, or prioritized experience replay all lead to significant performance drops, highlighting their crucial roles in addressing the challenges of ARC. Additionally, the results emphasize the value of pre-trained language models and data augmentation for effective program generation and learning. In conclusion, CodeIt represents a significant step forward in tackling complex reasoning and abstraction tasks like ARC. Its innovative combination of program synthesis, hindsight relabeling, and prioritized experience replay offers a powerful framework for developing more intelligent and adaptable AI systems.