#theor_note #theor_ai

The fact that human science doesn't publish no-op results is likely a significant limiting factor for AI capabilities.

When I worked on cybersecurity evaluation harnesses for LLMs one of the main issues models experienced was a lack of exploratory behavior. LLMs tend to "lock in" to a first hypothesis, often never exploring other possibilities.

I hypothesize this is because there is no reasoning *processes* in their training, only final, positive result state. All that LLMs know is that ideas turn out to be correct.

I'm surprised accelerationists haven't yet launched a "journal of failed ML science" to improve automated AI research capabilities.