What Did You Tell It to Do?

Douglas Hofstadter explains in Gödel, Escher, Bach:

There is an old saw which says, “Computers can only do what you tell them to do.” This is right in one sense, but it misses the point: you don’t know in advance the consequences of what you tell a computer to do; therefore its behavior can be as baffling and surprising and unpredictable to you as that of a person. You generally know in advance the space in which the output will fall, but you dont know details of where it will fall.

The context here is that as you move from machine language to assembly language to higher-level languages, you gain abstraction at the cost of perfect predictability. The more abstract your commands, the more room there is for gaps between what you meant and what you told the computer to do. At any level sufficiently abstract to think about the whole program, significant parts will be unpredictable, especially if you are trying to cross levels. See the Wi flag.

This is a reason why we have test servers. Large programs have too many moving pieces for anything to work as intended on the first try. Even if the coding is perfect, its aggregate effect on the system may not be what was intended. The developer did not think of what rule change X would do to ability Y when used with item Z, and now players can one-shot raid bosses. Oops. And that can happen with any moving part, which is why you want thousands of players to beat on the code, going through enough combinations to find the inevitable problems.

Back to the code level, documentation exists to reduce this. If we all programmed with textbook-perfect processes, you would be able to track any change throughout the system, and unpredictability would be lower. Unfortunately your crafting system is only half-documented, the comments are haiku in Gaelic, and the guy who wrote them quit to pursue his dream of becoming a pastry chef. Expect surprises every time you change anything. To cite a non-haiku issue along these lines, City of Heroes recently noticed that the taunt/aggro system was documented incorrectly, and no one knew that no one knew how taunt worked until someone traced back why the code was not doing what it was supposed to. It was doing what it was told to.

: Zubon

2 thoughts on “What Did You Tell It to Do?”

  1. Now there’s a blast from the past!

    I didn’t realise that anyone had actually documented the Wi effect. Thanks for posting the link Zubon, it brings back some memories from my days in AC.

    And for the number of times in many varied games where I seem to have been the victim of the Wi flag as well. ;-)

  2. Another gaming first: forget about quantum computing, we got quantum programming.

    You can either know what the failure will be, or when it will fail. Not both at the same time.

    Corollary: In quantum physics, observation changes the result. In quantum programming, the result is changed by system stress – While testing internally, everything is fine. When it’s live, it goes to hell.

Comments are closed.