"When you (or somebody else) finally has the good idea, you feel very stupid for not having seen it sooner." — H. M. Georgi, "Grand Unified Theories" in The New Physics
One evening two men went to a social club event. One of the men had quite a bit of alcohol, the other stayed entirely sober.
Each man drove home in his car in the darkness. The sober man got distracted and drove off the road and smashed his car into a tree, but he was not hurt.
The drunk man weaved around a bit on the road and almost hit a car, a dog, and a mail box, but kept it together and made it home without scratching his car. He even managed to stumble unhurt into bed before passing out.
The men at the social club analyzed the event. The concluded that in the interest of safety, every man would be required to drink a minimum of two alcoholic beverages at all future events.
Obviously this is a false parable, but it illustrates some very interesting aspects of reality and the human mind. In shows the difference between anecdotal evidence and statistics. Also that drinking is preferable to sobriety, until it isn't.
Statistically driving drunk is a bad bet. Take a sufficiently large sample and a trend will appear. Not everyone who drives sober drives safely every time. Not everyone who drives drunk gets in a wreck every time. But the frequency of accidents is quite a bit higher for drunk drivers than for sober drivers. The frequency of accidents also climbs as the amount of alcohol measured in the blood climbs.
I originally made up this parable to explain to my friends why one of my friends did not believe in global warming. Most of my believer friends have only the vaguest idea of how statistics work. That does not prevent them from sharing statistics that confirm their beliefs, even if it is easy to show the statistics are falsified. Math is just not a strong point with them. Let's call them artists, rather than math disabled.
On the other hand my global warming denier friend is a very capable guy. He is good at logical argument, at accounting, and at statistics. He knows more about the history of temperatures of the ancient earth than I do. So what are the chances that he is wrong and the artists are right?
We can know he is wrong by looking at the work of people, scientists, who know even more than he does. Those scientists have vast arrays of data available for analysis and know what can go wrong when data is collected.
We know he is wrong because we can step into a greenhouse during a sunny day in winter and notice it is warmer than outside. We can check (should we have the time and interest) the spectral characteristics of sunlight, of carbon dioxide, and of the radiation of heat from the earth, and see that carbon dioxide is indeed a greenhouse gas. And we can measure (in a home lab, if you have the money for equipment and the skill) the level of carbon dioxide in the air, and compare it to older measurements. And of course there are all those thermometers the scientists have set up around the world, starting in the 1700s.
But for a lot of stuff is even harder to distinguish between anecdotes, belief systems, and factual statistics. Recent studies showed many science experiments are difficult to reproduce, and that difficulty seems to be driven by mental (and ethical) problems of the scientists. In other words, Publish or Perish drives the survival of the best liars. In particular it seems like the entire profession of Psychology is run by nut cases. When specifically asked why their experimental techniques were so bad, most psychologists did not even seem to understand there was a problem. Yikes. [See How Scientists Fool Themselves]
Of course my global warming denier friend would take this information and say exactly: my global warming denier scientific minority has it right. Your ecology-warped green scientists are misinterpreting the data.
Even stay-at-home paranoid Internet mavens don't have time to check every fact. Politics and many other professions depend on lying as a basic tool. Yet we need truth or we will suffer bad consequences.
Fortunately most lies don't pass the basic smell test. My first order guess is that the scientists are right and the theologians are wrong. Scientists are not always right, and theologians are not always wrong. But it you drink the theological kool-aid, you are going to crash into reality at some point.
Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts
Monday, October 12, 2015
Monday, June 29, 2009
Insurance and Casinos
Health insurance is the big issue of the moment. All kinds of politicians made all kinds of promises to all kinds of people during the 2008 elections, and now they are pretending to act on those promises. An improved health care insurance system was promised by many to many. My own expectations are low. But I am happy to debate the issue.
It helps to look at insurance in general, as well as looking at the specifics of insuring people's health.
And to understand insurance, you need to look at it outside the box. A good way to look at something outside its box is to take a look at a similar box, then compare and contrast. The ideal other box to look at, when considering insurance, is the casino, or gambling industry. This is not just my being satirical. It has to do with statistics and probability, which were originally studied and understood by considering games of chance.
Most people know, when they walk into a casino, that the casino is not really in the business of philanthropy. Casinos make money, they don't lose money. People suspend that knowledge by overriding it with another idea. They may get lucky. An individual, going to a casino and playing games of chance for any given period of time, may gain money.
But the money that individual makes is not, except in very unusual circumstances, a concern for the casino. Because other individuals, who are also there hoping to get lucky, are losing a whole lot more money than the individuals who are winning.
Casinos are set up that way. They were set up that way in ancient times before probability and statistics were developed forms of mathematics. Each game in a casino has a return ratio. Over a series of games, the individual outcomes have predictable percentages. This is most obvious in a roulette wheel, where each of the outcomes is equally lightly. Given that, for the casino to win over time the betting odds are simply set to slightly favor the casino. I won't go into the details, but you can find some at Roulette at Wikipedia.
Other games are more complicated, but work on the same principle. In poker the fact that the casino always wins is even more obvious, since a certain amount of each pot is raked off by the casino.
Insurance is remarkably similar. In its simplest form, it is a pool of money. Some people make money (they get more out than they put in); their are expenses for managing the money, and also profits taken out by the owners. As a result, most people get paid back less than they put in. You could argue that as with casinos, if people were rational no one would patronize an insurance company.
But the world is complex, and fear is a primal human emotion. There is an argument that some forms of insurance do make sense for the people who pay for them. This is because decreasing risk has a value to individuals.
Look at auto insurance, for instance. If you get into a car wreck the expenses from it could be enough to bankrupt most people. You probably won't get into a car wreck on any given day or even year, but there is a good chance you will get into one eventually. You fear bankruptcy, so you buy automobile insurance every year. It is expensive, and you are unlikely to come out ahead unless you get into a wreck (make that an expensive collision, not a fender bender) within a year or two after you first take out insurance. Yet it is worth it to many people because it protects them from being impoverished by a random event, an accident.
One can fairly ask, how much is it worth, this diminishment of risk? The auto insurance company has administrative expense and takes out a big chunk of the cash flow as profit to distribute to its investors. So all the people who buy insurance, together, will get paid back less than they pay in. Insurance companies hire actuaries to calculate their long term pay out risks, and set rates high enough to cover that plus administrative costs plus profits. Individuals are not in a position to hire actuaries. All they can do is shop for auto insurance, hoping to find the insurance company that takes the least bite out of the pie.
One other aspect of insurance should be carefully considered: insurance as investment. We see this in some forms of life insurance. The insurance company pays out more in aggregate to the people it insures than they pay in! Even so, the insurance company covers its administrative costs and pays its investors a profit. How can this be?
When an insurance company knows it can hold its incoming revenue stream for a long time (because its actuaries told it so), it can invest its balance of money in bonds, stocks, and real estate. So from an individual insured's perspective, the insurance company resembles a sort of mutual fund. Your family gets paid when you die if you have life insurance, or you get payments from a set starting point until the time of your death if you purchase an annuity policy.
In theory Social Security works like this, as do most old-fashioned (defined benefit) pensions. You pay your social security taxes during your work life. The government invests this money in interest-bearing government bonds. The average person gets more than they pay in. Those who die early are out of luck. Those who live long get far more money than they put in. Of course in reality money put in today goes out today to those who are already drawing Social Security. But it shows the idea of insurance that not only spreads risk, but creates additional income for the entire pool of insured people.
Not that anyone in Congress is going to listen to you unless you gave at least $2000 to their election campaign, but given all this, what kind of federal health insurance would you like?
Should it provide roughly equal benefits for everyone? Should it funnel almost all the money to care for those with the most expensive medical conditions? Should the money that goes in the first year be paid out it once, or should it be accumulated, with interest, for the benefit of those who are doing the paying? Should it be subsidized by taxes, or should it generate a profit to the government that pays for non-medical budget items like the military budget?
It helps to look at insurance in general, as well as looking at the specifics of insuring people's health.
And to understand insurance, you need to look at it outside the box. A good way to look at something outside its box is to take a look at a similar box, then compare and contrast. The ideal other box to look at, when considering insurance, is the casino, or gambling industry. This is not just my being satirical. It has to do with statistics and probability, which were originally studied and understood by considering games of chance.
Most people know, when they walk into a casino, that the casino is not really in the business of philanthropy. Casinos make money, they don't lose money. People suspend that knowledge by overriding it with another idea. They may get lucky. An individual, going to a casino and playing games of chance for any given period of time, may gain money.
But the money that individual makes is not, except in very unusual circumstances, a concern for the casino. Because other individuals, who are also there hoping to get lucky, are losing a whole lot more money than the individuals who are winning.
Casinos are set up that way. They were set up that way in ancient times before probability and statistics were developed forms of mathematics. Each game in a casino has a return ratio. Over a series of games, the individual outcomes have predictable percentages. This is most obvious in a roulette wheel, where each of the outcomes is equally lightly. Given that, for the casino to win over time the betting odds are simply set to slightly favor the casino. I won't go into the details, but you can find some at Roulette at Wikipedia.
Other games are more complicated, but work on the same principle. In poker the fact that the casino always wins is even more obvious, since a certain amount of each pot is raked off by the casino.
Insurance is remarkably similar. In its simplest form, it is a pool of money. Some people make money (they get more out than they put in); their are expenses for managing the money, and also profits taken out by the owners. As a result, most people get paid back less than they put in. You could argue that as with casinos, if people were rational no one would patronize an insurance company.
But the world is complex, and fear is a primal human emotion. There is an argument that some forms of insurance do make sense for the people who pay for them. This is because decreasing risk has a value to individuals.
Look at auto insurance, for instance. If you get into a car wreck the expenses from it could be enough to bankrupt most people. You probably won't get into a car wreck on any given day or even year, but there is a good chance you will get into one eventually. You fear bankruptcy, so you buy automobile insurance every year. It is expensive, and you are unlikely to come out ahead unless you get into a wreck (make that an expensive collision, not a fender bender) within a year or two after you first take out insurance. Yet it is worth it to many people because it protects them from being impoverished by a random event, an accident.
One can fairly ask, how much is it worth, this diminishment of risk? The auto insurance company has administrative expense and takes out a big chunk of the cash flow as profit to distribute to its investors. So all the people who buy insurance, together, will get paid back less than they pay in. Insurance companies hire actuaries to calculate their long term pay out risks, and set rates high enough to cover that plus administrative costs plus profits. Individuals are not in a position to hire actuaries. All they can do is shop for auto insurance, hoping to find the insurance company that takes the least bite out of the pie.
One other aspect of insurance should be carefully considered: insurance as investment. We see this in some forms of life insurance. The insurance company pays out more in aggregate to the people it insures than they pay in! Even so, the insurance company covers its administrative costs and pays its investors a profit. How can this be?
When an insurance company knows it can hold its incoming revenue stream for a long time (because its actuaries told it so), it can invest its balance of money in bonds, stocks, and real estate. So from an individual insured's perspective, the insurance company resembles a sort of mutual fund. Your family gets paid when you die if you have life insurance, or you get payments from a set starting point until the time of your death if you purchase an annuity policy.
In theory Social Security works like this, as do most old-fashioned (defined benefit) pensions. You pay your social security taxes during your work life. The government invests this money in interest-bearing government bonds. The average person gets more than they pay in. Those who die early are out of luck. Those who live long get far more money than they put in. Of course in reality money put in today goes out today to those who are already drawing Social Security. But it shows the idea of insurance that not only spreads risk, but creates additional income for the entire pool of insured people.
Not that anyone in Congress is going to listen to you unless you gave at least $2000 to their election campaign, but given all this, what kind of federal health insurance would you like?
Should it provide roughly equal benefits for everyone? Should it funnel almost all the money to care for those with the most expensive medical conditions? Should the money that goes in the first year be paid out it once, or should it be accumulated, with interest, for the benefit of those who are doing the paying? Should it be subsidized by taxes, or should it generate a profit to the government that pays for non-medical budget items like the military budget?
Saturday, April 18, 2009
Randomness, Chance, and Two Darwins
We have the usual missionary cult groups in rural Mendocino County, California, near Point Arena: Mormons, Seventh Day Adventists, and Jehovah's Witnesses. In general they seem to be pretty nice people. I don't mind them visiting their neighbors. They'll talk if you want to talk, and go away politely if you don't want to talk. It is interesting to see their opening talk lines.
A few month's ago Paul (not his real name), an earnest young man (younger than me, anyway, no big feat these days) from one of these groups, asked me, "You don't believe all of this is due to chance, do you?" His sweeping arms indicated my bishop pines, eucalyptus, the people present, the sky and the universe.
"Yes, I do," I said. That threw him. He acted as if they forgot to put that in the recruitment play book. We talked a bit more, and they left (they always seem to come in pairs, which is good, because you can offer them a beer and they'll say No).
Yesterday I read about the difference between the randomness of classical physics and the randomness of quantum physics in Inward Bound
by Abraham Pais. It reminded me of how much trouble I have had accepting the nature of reality at the quantum level, of how most people are even more confused by randomness than I am, and the conversation with the earnest Christian.
I knew Paul was referring to the Theory of Evolution by Natural Selection. No, for things to be as snafu as they are in this world, you need a Central Planning Intelligence, a sort of cosmic CIA, God's designer intervention team.
Because even trained scientists have difficulty with issues of causation and randomness, you can bet (or lay odds) that religious recruiters are going to use people's confusion to lay out their own highly improbably, but very simple, causal story: God created things (and he speaks directly to the leaders of my religion, the only true religion).
Natural liberation involves understanding nature's randomness and causality. That means understanding their respective roles in the world. You need to remind yourself some times that words like chance, randomness, planning, and causality are used by humans to lump together a wide variety of natural phenomena.
It is said that some Roman soldiers worshipped Chance (Fortuna) as a god. Plato, on the other hand, did not like randomness, and preferred unseen causes to explain the otherwise inexplicable world. There are also the related ancient ideas of Fate and Free Will. Some people don't like causality because it seems to take away free will, but most people accept some causality in the "outside" world while reserving free will for people's decision making process.
A lot of this is illuminated by understanding the difference between the randomness of genetics and evolution, the randomness of classical physics, and the randomness of quantum physics.
Classical physical randomness is the best starting point. Think about dice or better still, a card game. The cards in poker are supposed to be dealt randomly, and for practical purposes we don't think about that when playing poker unless we think the dealer is cheating. But we know that each physical card maintains its physical identity from hand to hand; shuffling cards is a causal process that gives the card order a sort of practical randomness.
The classical physics problem that is solved with this sort of randomness is calculating the energy, specific heat or temperature of some matter. It is easiest with certain "ideal" gasses (ideal because the theory works out!). In classical physics everything is causal. The gas molecules all have locations, masses, and velocities. They bounce off each other in a causal manner similar to colliding marbles. But there are two many of them to use ordinary calculus to keep track of, much less calculate the temperature. By treating the molecules as having random velocities and directions, scientists could group them. The properties of the groups, fit into the right equations, give good answers. Even then this is complicated enough that scientists (notably Ludwig Boltzmann) did not get this essentially right until the late 1800's.
The point to be noticed is that in classical physics, there is no inherent randomness in matter or energy. Randomness is at the macro level, or the level of human perception. It arises out of microscopic events that are too difficult for us to keep track of. When we roll dice, they obey all of Newton's laws of physics, we just can't keep track. Nature, presumably, does keep track.
Charles Darwin's Theory of Natural Selection is covered under this classical randomness theory. Darwin did not know about chromosomes or DNA. We now know that the DNA of two parents is randomly mixed and selected for each child. This is an example of classical physical randomness: causal laws are at work on the cell level, but we can't follow them, we can only see the results. Survival until reproduction of the next generation involves both events that we classify as causal (quicker, and so escaped a predator) and random (standing in wrong place, wiped out by a flood).
I knew I was technically wrong to say, to Paul that I believe "all this" was a result of chance. There is chance involved, but evolution is really best described as causal. Its randomness is classical, a matter of humans using statistics to describe complex large scale events that are causal at a microscopic letter.
A second scientist Darwin (or fourth, if you want to include Erasmus Darwin and George Howard Darwin), Charles Galton Darwin, a grandson of Charles, was one of the many scientists who contributed to the creation of quantum physics. Coincidence, or caused by his high level of intelligence, education, and a family tradition of breaking new ground?
In quantum physics, very early on, it appeared there were truly random processes in nature. Radioactivity was the first example. Atoms of radioactive elements appear to decay randomly. So of course all the bright guys of the era set out to show that there were underlying causes for the decays. That would have set their classical physics world to right. But they could not find any underlying, classical mechanism for the randomness. In the 1920's some younger physicists started accepting this quantum randomness as just inherent in the universe. Today there is still no causal theory that explains quantum randomness. It is an simply a human observation about nature at the quantum level.
The philosophic and religious interpretations of this facet of natural reality tend to simply be used to reinforce prior prejudices. Some say quantum randomness proves there is free will. Some say it means there must be a multiverse. Many project the randomness into scales well beyond quantum physics. In fact, randomness averages out very quickly, even quantum randomness. Put a few atoms together in a molecule, and quantum randomness tones way down. By the time you have anything worth worrying about, say a few hundred molecules of gold in a lump, quantum randomness has no effect on causality. Classical physics works better.
Scientists and technicians have produce clever ways to bring quantum randomness into the world of objects that humans can see, but this is an engineering trick. Electronics parts may use quantum randomness (as in quantum tunneling), but they are themselves very predictable.
It takes a lot of study to gain even a basic understanding of quantum physics. Even the Bible is an easier read than an introduction to quantum physics text book, not to mention the mathematical prerequisites required to comprehend the equations in it. Pop quantum physics is about as useful as Bible quotes taken out of context.
Nevertheless, here's a recipe:
Two Darwins Quantum Piety
"Then said Jesus, Father, forgive them; for they know not what they do. And they parted his raiment, and cast lots [for it]." The Bible, Luke 23:34 [See also similar Matthew 27:35]
Thus is was that lots were cast at the most sacred event of the Christian religion, the Crucifixion [if it is not just a myth]. It was recorded because Fortuna wanted people to know that She does roll dice, both in the classical and quantum sense.
A few month's ago Paul (not his real name), an earnest young man (younger than me, anyway, no big feat these days) from one of these groups, asked me, "You don't believe all of this is due to chance, do you?" His sweeping arms indicated my bishop pines, eucalyptus, the people present, the sky and the universe.
"Yes, I do," I said. That threw him. He acted as if they forgot to put that in the recruitment play book. We talked a bit more, and they left (they always seem to come in pairs, which is good, because you can offer them a beer and they'll say No).
Yesterday I read about the difference between the randomness of classical physics and the randomness of quantum physics in Inward Bound
I knew Paul was referring to the Theory of Evolution by Natural Selection. No, for things to be as snafu as they are in this world, you need a Central Planning Intelligence, a sort of cosmic CIA, God's designer intervention team.
Because even trained scientists have difficulty with issues of causation and randomness, you can bet (or lay odds) that religious recruiters are going to use people's confusion to lay out their own highly improbably, but very simple, causal story: God created things (and he speaks directly to the leaders of my religion, the only true religion).
Natural liberation involves understanding nature's randomness and causality. That means understanding their respective roles in the world. You need to remind yourself some times that words like chance, randomness, planning, and causality are used by humans to lump together a wide variety of natural phenomena.
It is said that some Roman soldiers worshipped Chance (Fortuna) as a god. Plato, on the other hand, did not like randomness, and preferred unseen causes to explain the otherwise inexplicable world. There are also the related ancient ideas of Fate and Free Will. Some people don't like causality because it seems to take away free will, but most people accept some causality in the "outside" world while reserving free will for people's decision making process.
A lot of this is illuminated by understanding the difference between the randomness of genetics and evolution, the randomness of classical physics, and the randomness of quantum physics.
Classical physical randomness is the best starting point. Think about dice or better still, a card game. The cards in poker are supposed to be dealt randomly, and for practical purposes we don't think about that when playing poker unless we think the dealer is cheating. But we know that each physical card maintains its physical identity from hand to hand; shuffling cards is a causal process that gives the card order a sort of practical randomness.
The classical physics problem that is solved with this sort of randomness is calculating the energy, specific heat or temperature of some matter. It is easiest with certain "ideal" gasses (ideal because the theory works out!). In classical physics everything is causal. The gas molecules all have locations, masses, and velocities. They bounce off each other in a causal manner similar to colliding marbles. But there are two many of them to use ordinary calculus to keep track of, much less calculate the temperature. By treating the molecules as having random velocities and directions, scientists could group them. The properties of the groups, fit into the right equations, give good answers. Even then this is complicated enough that scientists (notably Ludwig Boltzmann) did not get this essentially right until the late 1800's.
The point to be noticed is that in classical physics, there is no inherent randomness in matter or energy. Randomness is at the macro level, or the level of human perception. It arises out of microscopic events that are too difficult for us to keep track of. When we roll dice, they obey all of Newton's laws of physics, we just can't keep track. Nature, presumably, does keep track.
Charles Darwin's Theory of Natural Selection is covered under this classical randomness theory. Darwin did not know about chromosomes or DNA. We now know that the DNA of two parents is randomly mixed and selected for each child. This is an example of classical physical randomness: causal laws are at work on the cell level, but we can't follow them, we can only see the results. Survival until reproduction of the next generation involves both events that we classify as causal (quicker, and so escaped a predator) and random (standing in wrong place, wiped out by a flood).
I knew I was technically wrong to say, to Paul that I believe "all this" was a result of chance. There is chance involved, but evolution is really best described as causal. Its randomness is classical, a matter of humans using statistics to describe complex large scale events that are causal at a microscopic letter.
A second scientist Darwin (or fourth, if you want to include Erasmus Darwin and George Howard Darwin), Charles Galton Darwin, a grandson of Charles, was one of the many scientists who contributed to the creation of quantum physics. Coincidence, or caused by his high level of intelligence, education, and a family tradition of breaking new ground?
In quantum physics, very early on, it appeared there were truly random processes in nature. Radioactivity was the first example. Atoms of radioactive elements appear to decay randomly. So of course all the bright guys of the era set out to show that there were underlying causes for the decays. That would have set their classical physics world to right. But they could not find any underlying, classical mechanism for the randomness. In the 1920's some younger physicists started accepting this quantum randomness as just inherent in the universe. Today there is still no causal theory that explains quantum randomness. It is an simply a human observation about nature at the quantum level.
The philosophic and religious interpretations of this facet of natural reality tend to simply be used to reinforce prior prejudices. Some say quantum randomness proves there is free will. Some say it means there must be a multiverse. Many project the randomness into scales well beyond quantum physics. In fact, randomness averages out very quickly, even quantum randomness. Put a few atoms together in a molecule, and quantum randomness tones way down. By the time you have anything worth worrying about, say a few hundred molecules of gold in a lump, quantum randomness has no effect on causality. Classical physics works better.
Scientists and technicians have produce clever ways to bring quantum randomness into the world of objects that humans can see, but this is an engineering trick. Electronics parts may use quantum randomness (as in quantum tunneling), but they are themselves very predictable.
It takes a lot of study to gain even a basic understanding of quantum physics. Even the Bible is an easier read than an introduction to quantum physics text book, not to mention the mathematical prerequisites required to comprehend the equations in it. Pop quantum physics is about as useful as Bible quotes taken out of context.
Nevertheless, here's a recipe:
Two Darwins Quantum Piety
"Then said Jesus, Father, forgive them; for they know not what they do. And they parted his raiment, and cast lots [for it]." The Bible, Luke 23:34 [See also similar Matthew 27:35]
Thus is was that lots were cast at the most sacred event of the Christian religion, the Crucifixion [if it is not just a myth]. It was recorded because Fortuna wanted people to know that She does roll dice, both in the classical and quantum sense.
Subscribe to:
Posts (Atom)
