OpenAI Chief Scientist Jakub Pachocki says AI systems could increasingly contribute to their own development over the next few years, creating a new class of safety challenges.
OpenAI is focusing on alignment, chain-of-thought monitoring and defensive AI, but Pachocki argues that current techniques are not yet sufficient to reliably oversee much more capable systems. He says no AI lab has solved alignment and monitoring well enough to keep scaling at maximum speed indefinitely, calling for shared safety thresholds and possible voluntary slowdowns.