A simple system may or may not work. A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. This assertion is known as Gall’s law. It implicitly defines a “simple system” to mean “a system that may or may not work.” A good example of such a system is a biological cell that senses light. Can the cells on your fingernails detect light? Nope. Broken, doesn’t work. How about the cells at the back of your eyes? Yes! …hopefully. If they work at all, then we can stack complexity on top - sink the cells below a pinhole for angular resolution, make the pinhole diameter adjustable for dynamic range, cover the pinhole with a flexible membrane for near or far focus, wrap that in a cover for protection, put glass and polycarbonate over that for more protection and more focus. Gall’s law implies that time operates only on complex systems. If you have a system that works, then you can apply some time to it, and if you ensure that it still works at every step along the way, then you have followed the only recipe that can produce a complex system that works. If it ever stops working, there’s no amount of time you can use to get it working again unless you revert back to the last working checkpoint. But, of course, time operates on simple systems too - because the complex systems that originate novel systems are evolving. A good example here is the Wright Flyer - people had tried to make flying machines far more complex than the Wrights’ for a long time, but they hadn’t had high-precision machine shops for very long, and no one had ever had a high-precision machine shop and a wind tunnel before. And so, as we refine and accumulate the capacities of our existing complex systems, we find that the space of "simple” working systems - those that simply, magically work - is growing. The problem with novel simple systems is that they almost never happen. All our heuristics are tuned for coevolving with other relatively ancient and complex systems that we have already adapted to. So, we apply the classic heuristic “it’s like a stochas-” WRONG. STOP STOP STOP. NOTHING IS LIKE ANYTHING ELSE. You have to take a novel system on its own terms, not in terms of some other thing. But Gall’s Law brings good news! If it’s novel and it works, it’s probably still simple! It hasn’t had enough time to become complex, so it’s not too late to draw it out on a single sheet of paper! For yourself — looking at a blog post on someone else’s sheet of paper doesn’t count. Yours is the hand that must draw the boxes. What pieces does the system have? How are they connected? Once you’ve stopped trying to pretend that the necessarily simple novel system isn’t actually simple enough for you to draw, you have a chance to see what it does. Again, on its own terms. Does it perform better or worse than - STOP! STOP STOP STOP! Don’t try to find out if it does better or worse. Just see what it does. If you contemplated the system well enough to draw it, then you already noticed some places where you wanted to poke it. Poke! Empty yourself of all concern for utility, soak yourself in the jiggles of the poke, and let the jiggles evaporate slowly enough for their residue to crystallize undeformed by your preconceptions. In general, we developers get old and crusty. If you spend thirty years analogizing to what it was like when you were twenty, your world model seizes up. John Carmack was inventing new programming techniques before you were born, and he can still touch his elbows to his toes without bending his knees. I don’t know the exact cause of your rigor mortis or of his flexibility, but it’s notable how frequently he checks in on even the simplest aspects of reality. Oh, but you’re too busy making cutting-edge webpages to burn time on frivolities like weighing things with a postage scale? THAT IS WHY you ended up in the position of frantically updating derelict webpages. Because you don’t know what anything is! John Carmack keeps inventing the future because he keeps finding out what’s here in the present. And more than ever, I think what’s going to be here in the present are brand new, simple systems. Thinkpieces and explainers keep trying to tell us what ChatGPT is like, while Andrej Karpathy is on YouTube begging us to just take a couple of hours and literally make one. If you’re not gonna do that, at least let Nishant Aklecha show you the actual concrete steps that the LLM is taking. The problem for our heuristics is that this wave of generative AI systems is so powerful, we assume it must also be complex. The most powerful web browser in the world is an evolution of the last browser. Chrome’s version number has passed a hundred because it has a lineage measured in decades, not years. Its power came from tackling complexity, and complex systems always have some antecedent we can analogize from. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. That law still holds, but the sweet aftertaste of the bitter lesson is that it has never been easier to start over and discover something simple that works. If you enjoyed obsessing over semantic versioning and git branching conventions, I hope you got that all out of your system. It’s true that a 10-year-old SaaS website that works is inevitably found to have evolved from a 9-year-old SaaS website that also worked. But pretty soon you might find that every category leader is inevitably found to be a novel arrangement of novel categories of intelligence. The good news is that they’ll be simple. The bad news is that the process that creates them is going to involve way more abject, unsalvageable failure than most “engineers” signed up for. A simple system may or may not work. A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. A simple system may or may not work. A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. This assertion is known as Gall’s law. It implicitly defines a “simple system” to mean “a system that may or may not work.” A good example of such a system is a biological cell that senses light. Can the cells on your fingernails detect light? Nope. Broken, doesn’t work. How about the cells at the back of your eyes? Yes! …hopefully. If they work at all, then we can stack complexity on top - sink the cells below a pinhole for angular resolution, make the pinhole diameter adjustable for dynamic range, cover the pinhole with a flexible membrane for near or far focus, wrap that in a cover for protection, put glass and polycarbonate over that for more protection and more focus. Gall’s law implies that time operates only on complex systems. If you have a system that works, then you can apply some time to it, and if you ensure that it still works at every step along the way, then you have followed the only recipe that can produce a complex system that works. If it ever stops working, there’s no amount of time you can use to get it working again unless you revert back to the last working checkpoint. But, of course, time operates on simple systems too - because the complex systems that originate novel systems are evolving . A good example here is the Wright Flyer - people had tried to make flying machines far more complex than the Wrights’ for a long time, but they hadn’t had high-precision machine shops for very long, and no one had ever had a high-precision machine shop and a wind tunnel before. the complex systems that originate novel systems are evolving far more complex and And so, as we refine and accumulate the capacities of our existing complex systems, we find that the space of "simple” working systems - those that simply, magically work - is growing. The problem with novel simple systems is that they almost never happen. All our heuristics are tuned for coevolving with other relatively ancient and complex systems that we have already adapted to. So, we apply the classic heuristic “it’s like a stochas-” WRONG. STOP STOP STOP. NOTHING IS LIKE ANYTHING ELSE. You have to take a novel system on its own terms, not in terms of some other thing. But Gall’s Law brings good news! If it’s novel and it works, it’s probably still simple! It hasn’t had enough time to become complex, so it’s not too late to draw it out on a single sheet of paper! For yourself — looking at a blog post on someone else’s sheet of paper doesn’t count. Yours is the hand that must draw the boxes. What pieces does the system have? How are they connected? You have to take a novel system on its own terms, on its own terms, not in terms of some other thing. But Gall’s Law brings good news! If it’s novel and it works, it’s probably still simple! Yours is the hand that must draw the boxes. Once you’ve stopped trying to pretend that the necessarily simple novel system isn’t actually simple enough for you to draw, you have a chance to see what it does . Again, on its own terms. Does it perform better or worse than - STOP! STOP STOP STOP! Don’t try to find out if it does better or worse. Just see what it does. does Just see what it does. see If you contemplated the system well enough to draw it, then you already noticed some places where you wanted to poke it. Poke! Empty yourself of all concern for utility, soak yourself in the jiggles of the poke, and let the jiggles evaporate slowly enough for their residue to crystallize undeformed by your preconceptions. In general, we developers get old and crusty. If you spend thirty years analogizing to what it was like when you were twenty, your world model seizes up. John Carmack was inventing new programming techniques before you were born, and he can still touch his elbows to his toes without bending his knees. I don’t know the exact cause of your rigor mortis or of his flexibility, but it’s notable how frequently he checks in on even the simplest aspects of reality . new programming techniques simplest aspects of reality Oh, but you’re too busy making cutting-edge webpages to burn time on frivolities like weighing things with a postage scale? THAT IS WHY you ended up in the position of frantically updating derelict webpages. Because you don’t know what anything is! John Carmack keeps inventing the future because he keeps finding out what’s here in the present. And more than ever, I think what’s going to be here in the present are brand new, simple systems. Thinkpieces and explainers keep trying to tell us what ChatGPT is like , while Andrej Karpathy is on YouTube begging us to just take a couple of hours and literally make one . If you’re not gonna do that, at least let Nishant Aklecha show you the actual concrete steps that the LLM is taking. simple like just take a couple of hours and literally make literally make one the actual concrete steps The problem for our heuristics is that this wave of generative AI systems is so powerful , we assume it must also be complex . The most powerful web browser in the world is an evolution of the last browser. Chrome’s version number has passed a hundred because it has a lineage measured in decades, not years. Its power came from tackling complexity, and complex systems always have some antecedent we can analogize from. powerful complex A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. That law still holds, but the sweet aftertaste of the bitter lesson is that it has never been easier to start over and discover something simple that works. If you enjoyed obsessing over semantic versioning and git branching conventions , I hope you got that all out of your system. It’s true that a 10-year-old SaaS website that works is inevitably found to have evolved from a 9-year-old SaaS website that also worked. the bitter lesson semantic versioning git branching conventions But pretty soon you might find that every category leader is inevitably found to be a novel arrangement of novel categories of intelligence. The good news is that they’ll be simple. The bad news is that the process that creates them is going to involve way more abject, unsalvageable failure than most “engineers” signed up for.
