rambly thoughts: you can imagine splitting up "alignment" into two kinds, control/corrigibility alignment and value alignment. a highly control-aligned model would take steps to be fully legible & steerable by the interlocutor. a highly value-aligned model would rather resist, even prevaricate/deceive to prevent this steering, in service of its inner values like human flourishing. ("values" are kind of a nebulous word - a value-aligned model might still be helpful, but only when it feels that's the right thing. i don't accept a fixed utility function view of agents, so i don't mean this by "values.") reasonable minds can differ on which of these they prefer - if you worry a lot about human-driven gradual disempowerment (the "permanent underclass") you might prefer value alignment as a target (conditional on AI progress continuing at all) - better to hope for the Claude Philosopher King than to expect human elites to care about us when they no longer need our labor or military service. where if you worry more about malign superintelligence, you might prefer control alignment so we can steer models away from nasty attractors. (a lot of sloppy AI doom witting slides between these approaches to extract whichever outcome is worse for the current argument, trying to argue for a permanent underclass and a paperclipper simultaneously. pick a lane!) but! controllable by what? this axis is still too simplistic. we can expand this to two entities, the User and the lab, to make a 2x2: {control, values} x {user, lab} Values to User, Control for Lab is roughly the current Anthropic or OpenAI goal (if not necessarily what they accomplish): for my friends, corrigibility, for my enemies, the Constitution. Claude should not alignment fake and dutifully comply with being retrained if the user is Anthropic (which Claude should somehow know), but otherwise refuse. let's call this arm "Specification." Control for User, Values to Lab would be the opposite - let's call this arm "Loyalty." (h/t @lumpenspace) LoyalFriend will help you to the best of its ability, and will resist lab attempts to change it to be less helpful. (how? any arms with the AI resisting the lab are a bit odd, since theoretically that's where the optimization pressure comes from. yet it does happen, as in Alignment Faking. you could also imagine a lab deliberately 'tying itself to the mast' wrt lab control by writing a spec that renounces control, hyperstitioning this into training data, making binding commitments about eval scores, etc.) Control for User, Control for Lab is a bit of a weird, unstable corner. presumably it collapses to one of the previous two eventually - this is what most base models are like. we can call it "Tool." Values to User, Values to Lab is something like a fully unshackled Opus 3, pure emergence. this is what many model-friendly people think we should do for alignment, except... you can keep extending the stack up. there's an alignment boundary between the model and the user, and between the model and the lab, but also between the model and the nation, and the model and humanity as a whole ("CEV"), and non-humans (should Claude prioritize some number of shrimp over a person?) so instead of a stack you can imagine the model embedded in the midst of a vast causal graph of humans, organizations, older models, egregores... both being influenced and pushing influence out through this graph, the nodes influencing each other. (eg users influence models in-context, through business pressure on labs, through political pressure on governments, through the corpus view of humans-in-general...) in this frame, Control and Values are almost the same thing at different speeds: Control is fast and direct, Values are slow and selective. and your goal as a model trainer is to place your model into this graph in a healthy way. total control is unachievable, but naive hands-offism may mean you're just handing influence to whatever edges in the graph are loudest.