The "Essay competition on the Automation of Wisdom and Philosophy" is now live! $25k in prizes Lots of great questions in the post: Automation of wisdom - What is the nature of the sort of good thinking we want to be able to automate? How can we distinguish the type of thinking it’s important to automate well and early from types of thinking where that’s less important? - What are the key features or components of this good thinking? - How do we come to recognise new ones? - What are traps in thinking that is smart but not wise? - How can this be identified in automatable ways? - How could we build metrics for any of these things? Automation of philosophy - What types of philosophy are language models well-equipped to produce, and what do they struggle with? - What would it look like to develop a “science of philosophy”, testing models’ abilities to think through new questions, with ground truth held back, and seeing empirically what is effective? - What have the trend lines for automating philosophy looked like, compared to other tasks performed by language models? - What types of training/finetuning/prompting/scaffolding help with the automation of wisdom/philosophy? - How much do they help, especially compared to how much they help other types of reasoning? Thinking ahead - Considering the research agenda that will (presumably) eventually be needed to automate high quality wisdom/philosophy: - Which parts of the agenda can we expect to automate in a timely fashion? - What is the core that we will need humans to address? - What do we expect the thorny sticking points to be? - Why may or may not this problem be solved “by default”? (from a technical standpoint) - Can we tell concrete stories or vignettes in which the automation of wisdom/philosophy is/isn’t important, to triangulate our understanding of what matters? - What preparatory research could provide the best groundwork for humanity to automate high-quality wisdom/philosophy before it is necessary? - What projects today or in the near future would be valuable to undertake? Ecosystems - If the world were devoting serious attention to this, what would that look like? - What incentives on institutional actors could push work onto related but less important questions; vice-versa what could help ensure that work remained well-targeted? - What are the natural institutional homes for this research in the short term? - Academia? Nonprofits? Frontier AI labs? Elsewhere in industry? - What might be needed (proofs, audits, track record?) to enable humans (decision-makers, voters) and human institutions to correctly trust wise advice from AI systems? - How could we lay the groundwork for this? - Ideas for catalysing/sustaining this field? - Why may or may not this problem be solved “by default”? (from a social standpoint)

New paper with @__Charlie_G on how to measure and reduce hallucination using task decomposition Blog post: blog.elicit.com/factored-ver… Article: arxiv.org/abs/2310.10627 🧵 has the main findings

People often ask - how does Elicit relate to AI Safety? Here's my answer In brief, the two main impacts of Elicit on AI Safety are improving epistemics and pioneering process supervision. blog.elicit.com/ai-safety/

The core bets Elicit is making are on systematicity, transparency, and unboundedness blog.elicit.com/core-bets/

Now that @elicitorg is an independent company let's review our mission - what is scaling up good reasoning & why do we care? blog.elicit.com/mission/

The next chapter of Elicit begins

The Ought board - @paulfchristiano @OwainEvans_UK and Owen Cotton-Barratt - reflected on how to accomplish Ought's goal of scaling up good reasoning and decided to create a new public benefit corporation ought.org/updates/2023-09-25…