All that is under the Sun
Let me start with a confession that should worry you slightly. The large language model at the heart of every “AI agent” you’ve read about is, on its own, utterly helpless. It cannot remember what it did five minutes ago, press a button, read a file, run a test, or check whether its own brilliant …
In 1974, the legendary physicist Richard Feynman stood before the graduating class at Caltech and told them a story about the South Sea islanders. During World War II, these islanders had watched magnificent flying machines descend from the sky, bringing with them endless supplies of cargo. When the war ended and the planes stopped coming, …
Imagine you are the most intelligent person in the world. You have read every book, seen every movie, and studied every equation up until the year 2023. But today, you wake up with a strange condition: anterograde amnesia. You can hold a conversation, you can reason brilliantly using your vast library of knowledge, but you …
The Art of Choosing One From Many: Categorical Cross-Entropy and the AI’s Grand Decision Imagine you’re a librarian training a new assistant. This isn’t just any library; it has 50,000 different sections. You hand the assistant a book and ask, “Where does this go?” The assistant, being new, doesn’t just point to one section. Instead, …
Imagine you’re trying to teach a machine the difference between a cat and a dog. You show it a picture of a cat, and it says, with 90% confidence, “That’s a cat.” Great! Now you show it a dog, and it says, with 60% confidence, “That’s a cat.” Not so great. The second guess is …
Imagine you’re teaching an archery student. They loose an arrow, and it thuds into the target, a little off-center. You wouldn’t just say, “You missed.” To be a good teacher, you need to know how much they missed by. Was it a near-perfect shot, just a hair’s breadth from the bullseye? Or did it land …