When considering the history of AI, Richard Sutton observed that brute force and compute scale has always trumped human expertise, and when you look for it, you can see this “bitter lesson” play out throughout tech history. In his keynote at Ai4 2026, Tim O’Reilly explains why grappling with the bitter lesson is the forge of effective AI corporate strategy, as companies figure out what to embrace and what to let go of. Drawing on his recent conversations with Trail of Bits CEO Dan Guido, Tim argues that AI’s business impact actually hinges on organizational adoption—the hard, unglamorous work of restructuring workflows, data, and incentives around what AI can do. Trail of Bits has modeled that process and documented it in a playbook other companies can use. Here, Tim shares some of the practices, like capability ladders, shared config repos, and company-wide hackathons, that helped Trail of Bits make AI a structural component of its business. This doesn’t mean that AI-native companies “sit back and let the progress of AI carry us forward.” Human expertise still matters, and it’s often the differentiator that helps organizations rise above their competitors. As Tim concludes, “The world is full of great problems. And so if AI takes away and makes easy something small, celebrate it and go work on something big with the new powers that we’ve been given.”
Takeaways
02.33 The bitter lesson is real, and it can catch any of us.
The bitter lesson is Richard Sutton’s contention that human expertise doesn’t really matter, that it will eventually be outmatched by computing scale. O’Reilly’s Whole Internet User’s Guide & Catalog was the first catalog of websites and the first site on the web to have advertising. It grew into Global Network Navigator, which was the first web portal. But O’Reilly’s products were manually curated. Yahoo came along and expanded on these ideas, but O’Reilly and Yahoo were both beaten by Google, which simply threw a bunch of compute at the problem. Now ChatGPT has changed the game again.
06.27 AI-native workflows require a different mindset.
When O’Reilly set out to develop a product that assessed learners’ capabilities and gave them a skill path to level up, the team used AI as an assistant, to write quiz questions, for instance. But LLM chatbots can already identify skills when given context about a developer. Evolving toward an AI-native skill path builder meant reconceptualizing the product as a more interactive experience that reflects where capabilities are today. However, even the most well-thought-out workflow can be hindered by gaps in access or knowledge. As Trail of Bits CEO Dan Guido says, “You have to build a system in which expertise compounds.”
10.28 AI adoption is a human problem.
Moving up the framework for AI adoption from AI-assisted to AI-augmented to AI-native isn’t just a technical challenge. It’s psychological. Only 5% of Dan’s staff was actually on board when he started the transformation; 70% were just quietly going through the motions, and 20% were actively resistant. He traces this to a handful of biases: self-enhancing bias, opacity, intolerance for imperfection, and above all, identity threat, the fear that AI won’t just replace the work someone does but who they are. Getting teams on board requires the organization to reframe AI as a tool that enhances identity, not something that will take it away.
16.44 A status ladder helps team members understand where they’re at and where to focus next. Hackathons compound that knowledge across the company.
Trail of Bits has a three-level status ladder: not engaged with AI or actively resisting it, experimenting with AI, and building AI that strengthens the organization’s overall capability. Level zero isn’t treated as a skill gap. It’s treated as working against the company’s goals, and the other two levels get a more detailed capability matrix broken out by department, since what a security auditor does with AI looks nothing like what someone in accounting does. O’Reilly is building its own version of this, drawing on the technical and business skill data it already has across its platform. Trail of Bits runs a hackathon every two months, each with a stated objective and learning goals announced a week ahead. Success is measured not by what got shipped but by where people land on the capability ladder afterward. Then the work gets fed into a shared skill repo, giving the entire company a set of reusable artifacts, and what one hackathon turns up becomes something the next one can build on.
24.37 Turn scar tissue into infrastructure.
Drew Breunig talks about the problem of prompt debt: prompts that grow more complex and more tuned to one specific model until they’re no longer portable. Trail of Bits flattens this complexity by turning every failure into a global, copy-pasted fix hosted in a company-wide repository. They’ve also standardized the safety net, with sandboxes for different needs and a seven-day cooldown on every new package from outside that gets installed—rules the whole company follows. To make this all work, employees need the chance to try things out and iterate on their failures. Dan says the only real mistake he made was not giving people enough unstructured time to experiment.
32.44 Human expertise still matters.
AI can make companies more productive, but it’s not a magic weapon. It’s a medium that people can use to share or extend their unique expertise and perspective. O’Reilly’s mission is to share the knowledge of innovators: You can think of the company as a matching marketplace for people who have expertise and people who need it. Agents offer a valuable new means of getting that expertise to customers in the tools they’re using to make business decisions. O’Reilly CTO Andrew Odewahn has noted that faster local decision-making has splintered central planning, so it’s harder than ever to get the big-picture view a good corporate decision needs. O’Reilly’s Expert MCP server lets customers access our content and use it to increase organizational intelligence. For instance, you can ask an AI tool to analyze a team’s workload and write a hiring case based on how O’Reilly’s own experts would review the request, and you’ll get a grounded argument with solutions authenticated by citations from actual practitioners. O’Reilly is building this capability into an organization-wide grounding layer it calls O’Reilly Expert Intelligence. It’s in beta now, and you can check it out.
Is cybersecurity part of your job in any way? If so, we’d like to know what you think for a report we’re writing. Just answer these quick 11 questions. Thanks in advance! Take the survey >