I've been teaching computers to think since ~2013 and over the last few years I've been building almost exclusively AI powered products, from the first AI powered fraud risk engine in Government to an EdTech product used by 400k+ teachers. If I had to compress what that taught me into one line, it's the first line of my manifesto: Treat AI as a collaborator, not a vending machine.
A vending machine takes your coins and dispenses a fixed product. You press a button, something comes out, and the transaction is over. That's how a lot of people are using LLMs. They type a prompt, take the answer, and move on. The problem is the answer is only useful if you already know what you're looking for. Vending machines don't have conversations, they don't push back, they're sycophantic.
The AI powered fraud risk engine I built for Government wasn't a vending machine. It wasn't built to have questions fed into it and take whatever fell out. Instead, it screened millions of authentications and flagged what looked wrong, and then humans decided what to do with the flag. The product allowed Government to act at scale whilst still paying attention, but it didn't make a judgement call.
Since the early days of computing, having a collaborator changes how we think. Remember rubber duck sessions with colleagues on hard problems? Collaborators help you spot patterns you would have missed on your own, but it's still up to you to decide if it's a real problem or not.
I've recently spent a few months in Australia and New Zealand, and in my spare time whilst travelling I built Scroll Books - a reading app for beating doomscrolling, with AI assistance for coding. The app went from idea to the App Store and Google Play because I could use AI to move faster on the parts of the code I already understood, and to argue with me on the parts I didn't so I outputted something of value faster. The decisions that matter stayed with me, of course. A vending machine can't have good product sense.
The EdTech product taught me the same lesson from the other direction. 400k+ teachers used it, and the users who got the most from it were the ones who treated it as a tool to work with, not a vending machine to get answers from. AI that hands you certainty is a shortcut to the wrong answer. AI that shows its work and lets you push back is a tool for getting less things wrong, which compounds over time to create better outcomes.
So when someone asks me how to use AI in product work, I don't tell them to prompt better. Instead, use these models as a sparring buddy. Give it enough context to go to battle with you, show it your work, and then ask it to disagree with you. The vending machine gives you what you asked for, whilst the collaborator gives you what you actually need to hone your craft.