One thing I’ve picked up from everyone’s favorite stable of bloggers at the George Mason economics department is that you should be willing to bet on your beliefs. If you really believe in what you are saying, it is taking easy money; if not, shut up. I think this would be useful in our world of interminable online arguments. It improves the accuracy and precision of your beliefs, and it forces others to face up to theirs.
The heart of it is nailing down just what it is that you (or others) believe. Many of your ideas exist as fuzzy generalities that have no application to the world because they are insufficiently well defined for there to be any counter-evidence. If mutually exclusive options are both consistent with what “you know,” you don’t really know anything.
The first part is exactly that testable condition: what state of affairs would prove you wrong? If you think the sun always rises in the morning, you should be willing to bet that it will rise tomorrow. If you think 2010 is the year of Aion, you should be willing to bet that it will have X subscribers at the end of the year (or some other measure that you can get more easily). This is where many people will start backing off. Yes, you believe that Hot New MMO will be the WoW-killer, but “due to marketing” you are unwilling to bet that it will have more subscribers in the first year. There is always a face-saving excuse. Accept that if you need to backtrack to a softer position while betting, your true belief is that softer position, rather than the claim you were advancing. People who want bets with terms ridiculously in their favor or are unwilling to admit that any possible state of affairs would prove them wrong are just making useless noise.
The second part is that the odds you demand show how certain you are. If you need even odds, you must think it is a 50-50 chance. If you really are 99% sure, you should be willing to bet on that basis and take easy money while giving someone at least 90-to-1 odds. This is where many people back further off. Yes, he said he was absolutely certain, but he will not even give 2-to-1 odds. If someone will not take easy money, he is just making useless noise. For your own epistemology, this is a great chance to see just how certain you are about your own beliefs. If you would not give 9-to-1 odds on a claim, you are not really 90% certain. Accept that you hold more tentative beliefs than you are willing to claim on internet forums.
The best of these are on quantitative variables: at what points would you give even odds, two-to-one odds, ten-to-one odds? This lets you determine how precise your beliefs are by seeing how much spread you need to get to 50%, 66%, or 90% certainty. Take WoW’s subscribers as an example. Would you give 100-to-1 odds that WoW has more than one million subscribers on December 31, 2010 (assuming the game still exists, to bar outrageous fortune)? Most likely. What number is your break even, no idea whether it will have more or fewer? Establishing that range makes your belief about whether WoW is growing or dying something concrete, rather than some vague generality.
I think of this as refining one’s own beliefs, but here on the internet, it would be nice to establish it as a norm just to shut some people up. “This game is dying!” Really? I just did a /who on server X at 5:10pm Eastern and got Y names; let’s take the same time and day of week 4, 8, or 16 weeks down the line and re-/who. If you’re so sure this game is dying, you’ll bet me 100 gold at 2-to-1 odds that the number will be at least 10% lower, right? Then we find out that, no, he wants even odds, and he wants some other basis of comparison (might be fair), and he wants any decrease rather than at least 10%, and really he does not want to bet because [developer] can still pull this out and save the game if it listens to his ideas. So when made concrete, “this game is dying” means “I think there is a 50-50 chance that there might be at least one person fewer logged on a few months down the line.” Wow, with a belief that strong, I can see how you think such drastic changes are necessary.
: Zubon
(An aggressive break-even point can also be a sign of strong belief. If you want even odds that WoW will have more than 7 million US/EU subscribers at the end of 2010, that means you think it is a coin flip whether it goes up or down; if you want even odds at 3.5 million or 10.5 million, that is a strong claim because you have gone all the way to a 50% increase or decrease. In that case, you are probably willing to give 10-to-1 odds on a decrease/increase from the current number.)
Update: slow blog day? We have a lot of comments. Rather than write something longer than the original post, most of the comments that are not just promoting irrationality can be answered with two points. First, I said I think of this as primarily for improving your personal beliefs, so I’m quite happy if you can nail it down to “what would be a fair bet” rather than one you will actually make. At work, I make few bets about our numbers, but I use this as an exercise to determine what I believe and how strongly/precisely. If your beliefs will never be testable, as I said, you are just making noise; if there is no foreseeable point at which your beliefs intersect with reality, they are irrelevant. Second, beyond the existing body of work on prediction markets and futarchy, there is an existing model already in place: the stock market. If your objection’s being true would imply that we could not have a stock market, consider it already refuted by reality.
Hmmm… interesting. Ever read “Earthweb,” by Marc Stiegler? It’s a science fiction novel in which a system of internet wagering is used to figure out the odds on just about everything. The output of the system is used to determine the optimal strategy for defeating an extra-terrestrial threat.
Robin Hanson has proposed running society that way. I have not read the book, but there are real proposals for ideas markets.
A tangent:
Do you prefer Fisherian or Bayesian statistics? Bayesian methods allow your prior beliefs to affect the outcome of a statistical test. For example, if you are 90% certain of something before you examine the evidence, then the evidence will have to strongly reject your hypothesis before you will abandon it.
Instinctually a Fisher approach, where all conclusions are based entirely upon the evidence at hand, seems like the only sane way to evaluate data. However, simulation studies have shown that in many settings Bayesian methods outperform Fisherian methods in terms of both statistical power (ability to detect a trend) and accuracy (rates of type one error). Despite this, I rarely use Bayesian methods. When I do, I tend to use a “flat prior” where I assume anything could be true before I run my test. Whether this makes me sagely or foolish I cannot say.
As far as I can tell, we live in a Bayesian world. Updating your beliefs is the way to go. In my work, though, frequentialist statistics is about all I need.
It’s a nice thought, but for this to take off people would first need to learn how to distinguish between wishes, beliefs, and knowledge when they write – I’ve been wading through this lately on another forum (on an unrelated topic.
To put things in MMO terms, if somebody writes “SWTOR will have full-loot open-world PvP.”
They could mean “I really hope SWTOR has…”
They could also mean “I’ve looked over all the game-play videos Bioware have released, and based on [evidence] I believe it likely that SWTOR will have…”
Or they could mean “On my recent tour of Bioware’s secret command bunker under Cheyenne Mountain I had a chance to talk with the lead designers of SWTOR, who assured me it will have…”
Any would-be bookies care to give odds on forum/blog-goers learning to make these subtle, but important, distinctions in their writing?
You (and the George Mason dudes) are assuming people want to make bets with an expected value of zero, with the goal of stating their true beliefs about the world; it seems to me that most people’s betting strategy is to try to make as much (imaginary) profit as possible off of people whose beliefs are so obviously wrong that they’re just handing out free money. And MMO players in paticular are a risk-averse bunch.
The “anyone unwilling to bet at given odds thinks the odds are worse”-thing seems a bit oversimplified. What about loss aversion and risk aversion?
I have to agree with malkav. This does not take into consideration loss aversion or risk aversion.
At my office, the phrase for culling the ideas and plans that people believe in vs. what they just throw out in a meeting is, “Would you bet your paycheck on it?”
Even though nobody is ever going to be required to pony up their net earnings on their next paycheck, just putting things in those terms seems to have quite a sobering effect on optimism. People get asked that and they suddenly want to revise that schedule one more time or nail down those requirements just a little bit more firmly.
But when somebody says they will bet their paycheck, even with caveats, we tend to believe them… and the caveats tend to point at the weaknesses in the overall plan.
If you think about it, spending money on a lottery ticket gives you odds at winning big, but at the same time there are odds in play that not spending that $1 will result in you having that $1 the next day.
Basically, not betting on your beliefs means nothing, because you are betting at the same time someone else isn’t going to disprove them. By not betting, you are actually betting.
Makes sense to me :)
No, the 2nd paragraph is the opposite of true. Re-read what you wrote there. Betting means you bet that someone else isn’t going to disprove them. At most, you could say that not betting means betting that someone else is going to disprove them, which means betting that you are wrong. Which is a weird epistemic state to be in.
Of course, it’s also possible that the cost of losing is high enough that for any non-trivial sum, a person is unwilling to take odds that aren’t a sure thing…
If you are willing to bet above break even on some prediction about MMOs, then you are already wrong. Whatever your personal opinion might be, you’re dealing with thousands of compounding variables with differing levels of effect and timing. On top of that the actions of any number of outside actors can cause sudden fluctuations on a relatively large scale.
This isn’t a simple binary, on|off, system, nor is it a spectrum, two conflicting concepts can, in fact, be completely true. You can, for instance, have the best starting zones in the industry and not get more new players. No part of it is governed by any single variable, only the combination of large cross sections. Going to one of your examples, that Blizzard would have some change in player numbers for WoW would certainly be something I’d be willing to break even on. Anything more than that would be pure guesswork. After all, I can’t predict sudden major failures of customer service or technology, nor how any of the next few releases might effect it’s particular subscriber numbers, nor the state of the economy at year’s end. Even more scary, their actual subscriber numbers could change fairly significantly without our methods of measurement being able to track that change at all due to it’s distribution across time zones, regions and servers.
There’s a lot more in one’s willingness to bet than belief that you ignore here. For example, some people are philosophically or morally opposed to wagering, so they wouldn’t bet on an outcome they believed in. Also, some prices are too steep for a bet; no matter how sure I am about an outcome, I would never bet the life of a loved one, even if I were 110% sure of an outcome.
Further, a bet doesn’t always correlate to a confidence in the outcome. Any thinking person knows that the odds are stacked against them when gambling, but last I heard Las Vegas and lotteries are still turning a tidy profit with people playing with unfavorable odds.
There’s also the problem that you can only wager the measurable in this case. Lots of people have faith in the existence of a higher power without having measurable proof. There’s also the the problem that you can’t prove a negative (not quite true, but read the link if you want to argue). There’s also the issue of taste. I might believe that WoW PvP will always suck, but that’s for my personal tastes not on an absolute scale; just because others find something acceptable doesn’t mean it’s not bad.
Reality also intrudes. What is a sizable bet to me might be chump change to another person. Likewise, my meager bet might be seen as excessively frivolous to someone in need, like the victims in Haiti. Or, perhaps someone can’t afford a bet the size needed to show their belief. I might believe that independent developers are the future of game development, so if I get a job at a large company does that mean I don’t believe in independent development? No, it would mean that I can’t afford the bet at that time.
So, I think that betting is a poor measurement of belief. Not that people don’t become a bit overly enthusiastic on the internet when presenting opinions, though.
The update covers most of this. I’m hoping you’re not calling me an idiot again with the “bet the life of a loved one” thing, because exaggerating the post past the point of stupidity is not a generous interpretation.
On proving a negative: of course we do that all the time. We have a mention of Bayesian updating above, and absence of evidence really is evidence of absence (of varying strength). If your theory predicts that X should be there, and it is not, that is evidence against your theory. Most of us are pretty confident on the question of whether werewolves exist, even without a deductive proof on the matter.
I’m hoping you’re not calling me an idiot again…
No, actually I think you’re doing the opposite: being too intellectual for your own good. You’re trying to show a justification for saying, “Put up or shut up,” and it’s falling a bit flat. As I said, some people do create more noise than is worth. These people I ignore.
As for the “life of a loved one” extreme wager, my point was that sometimes some things are too dear to bet on; I figured that was one thing that most people would agree with. Depending on my existing financial situation, $20 would be too much for me to bet on something I’m really sure about (2009 was a terrible year for contract work). Just because I was willing to wager $20 on something a few years ago but not today doesn’t mean I believe the previous thing any more strongly.
In the update, Zubon wrote:
If your beliefs will never be testable, as I said, you are just making noise….
This is the point I strongly disagree with. I think an opinion such as, “I think World of Warcraft is losing subscribers.” is interesting even if it’s not verifiable by most people. Without data that Blizzard doesn’t share, we can’t show this conclusively. Sure, we can approximate by looking at “peak times”, but you’re just adding uncertainty.
Ultimately, I think your point is that people do a terrible job of separating out fact from opinion, and I agree with that. I’ve been accused of passing off my opinion as fact before, usually by posing as an authority. But, I think you’re going a bit too far here in that you’re essentially saying opinion is useless.
You’re selling proxies short. Looking at online users at peak times reduces uncertainty. Why? If you’re not even going that far, you really are just speculating and going with “everyone I know…” which usually means “this one guy I know…” You’ve taken a step that moves you away from meaningless noise and condemned it for not moving to a Platonic ideal of knowledge. That’s bad.
Concerning your loved ones: do you let them drive, ride in cars, or walk near roads? There is a 1 in 10,000 chance per year that they will die in a crash. You are 99.99% certain that they will live through the year. You would not gamble their lives on a 110% certainty, yet you consistently gamble yours and theirs below 100%. Why? (My guess: at some point, you act on less than perfect information, as people seem to be objecting to. Sara thinks there are too many variables to talk about whether WoW will gain or lose subscribers this year, but Sara is probably willing to bet far more than $20 on a riskier question with more variables via things we call “weddings” or “moving for a new job.”)
You’re selling proxies short.
No, I just know too much and know that peak user figures are a terrible measurement of the number of people playing a game. If 5 people are on during prime time instead of early morning because an Australian guild left the game, that doesn’t mean the game has grown. Lies, damned lies, and statistics.
do you let them drive, ride in cars, or walk near roads?
I don’t “let” them do anything. They’re autonomous people who make their own decisions, which means they may take risks they choose. I take risks with my own life, true, but that’s because the opportunity costs for never leaving home home (where, really, it’s not 100% safe) are too great. Anyway, there’s a world of difference between “Hey, let’s drive to the store allowing that we might get side-swiped by an semi,” and “Hey, point this loaded gun at your head and pull the trigger. I know the chamber is empty because I just checked.” I’ll leave it as an exercise to the reader why those are different.
Your warrior spirit may have gotten away from you again.
Many of my assertions are based on “what if” theories that won’t ever be tested. There’s little point in betting on something that isn’t testable or won’t be tested for whatever reason. Even if I cared to gamble. *shrug*
the point about risk aversion is valid. I believe event X will occur at 50% probability then there is 0 profit to bet $1 on it. On average I will end up with $1 and worse case I have 0.
People will bet if they think they are making substantial profit in the bet, that compensate them for the risk. I think most people starts to be willing to bet with 1-on-1 odds when they are between 70% and 80% sure of something, meaning they make a ~50% average profit.
Also, I noticed people are easily more adverse to A-to-1 bet because they will gain little most of the time but loose a lot if things don’t go their way. Conversely, people like lotery where they are likely to loose a little but have a very little chance to win a lot. This is a documented psychological pattern (I’m too lazy to find the reference). You would say it’s not rational but that’s how people are :)
Anytime rich men make statements in confidence at me with no actual evidence I will use your “bet or change your mind” post instead of arguing in futility :)