Tuesday, May 15, 2007

Judith Rich Harris: No Two Alike

Judith Rich Harris's book No Two Alike is a followup to her previous work, The Nurture Assumption. In her first book, Harris explained that most of the non-genetic affects on the personalities of adults are a result of their interactions with peers rather than with their parents. She pointed out that many people want to believe (and prove) that parents are the major source of their children's personalities. According to The Nurture Assumption, the field of sociology has been confused for several decades and has been trying to distinguish nature and nurture, when they needed to be either distinguishing the effects of heredity from environment, or disentangling the environmental influences which include both parents and peers.

The best tool, according to Harris, for distinguishing the effects of heredity and environment consists of studies of twins. Comparing twins raised together and twins raised apart controls for genetic affects and allows us to see the effect of gross differences in environment. Comparing fraternal twins raised together with identical twins raised together holds the gross environment constant and makes it easier to see what differences are purely genetic.

In the new book, Harris focuses on why twins are so different, in order to isolate the causes of differences that aren't explained by other results. The existing literature says that some proportion of personality differences are due to genetics, and some proportion by each of various environmental causes: parents, wealth, neighborhood, etc. But a significant amount of variation remains that isn't apparently caused by any of these. Her focal example is that even siamese twins have different personalities, and they share all of their genes, and all of the environmental influences that anyone could hope to treat as responsible.

Harris' conclusion (skipping over most of her argument for the moment) is that there must be something driving each of us to be unique, and that means we have to find a distinction to enhance. The bottom line is that a significant part of personality (who we are) isn't determined by factors that we can examine or control. Each individual starts out with an endowment of heredity, and occupies an environment that isn't fully under their control, but the developing personality is still a negotiation between that individual and their context. If the strongest part of their innate tendencies is best suited for a niche that is already filled, they will look for a second best. When two identical individuals struggle to fill the same niche, some factor (random or not) will eventually determine a winner in each particular event, and at some point the effects of competition, if nothing else, will drive them to exploit different strengths. The different choices and different results in competition will magnify any differences, and over a reasonable lifetime, they will become recognizably different people.

Along the way, Harris spends a good deal of effort (successfully) demolishing other possible explanations (differences in environment, combination of nature and nurture, gene-environment interactions, environmental differences within the family, gene-environment correlations, and transferability of learning between situations). At the end she argues that she has demolished all the other possibilities and provided an argument for the one remaining theory (an innate drive for status), and so it must be true. But her argument for the specific mechanism is a little too weak, and it seems plausible that some variation or related description will fit the data a little better. I'm reasonably convinced that something drives us to differentiate, but it may not be purely a status drive. Two possible variations on her theory include drives for attention or to master something.

I found the style of Harris' presentation sometimes compelling, and sometimes distracting. She fit the presentation into the framework of a detective story. The presentation is salted liberally with examples from popular detective stories to show how attentive the detective has to be to details that have distracted other investigators. This worked for me when I was familiar with the detective in question (Sherlock Holmes, Kinsey Milhone), and didn't work when I hadn't read the stories (Alan Grant). I suspect that Holmes is the only one of these that is widely enough known that other readers would feel that they should get the references even when they don't.

On the whole, I think the book was successful in explaining that fundamental differences in personality are effectively the result of an innate drive that causes us to differentiate. The drive makes use of arbitrary differences in the material it has to work with (genes and environment). Parents do make a difference in the lives their children lead, but the ultimate person each child becomes isn't determined by parenting style at any gross level.

Thursday, May 10, 2007

Safe Harbor for Prediction Markets

A group of distinguished economists wrote a public letter advocating a legal safe harbor for some small-scale academic prediction markets. I can see why they limited their goals as they did, and I agree that everything they advocated should be legal, but I think they may have limited their objectives just enough to prevent any big wins.

One thing that Prediction Market maven Chris Masse constantly argues is that Prediction Markets on dry subjects need to be accompanied by entertaining questions in order to to keep the audience's attention. The economists had good reasons for shying away from recommending that sports betting should be included, but there are many other topics that diverse markets could include that give traders a reason to check back in. The range from the obvious entertainment questions (movie earnings and oscar winners) to legislative outcomes (bills passing and control of particular legislative bodies) and introduction and market success of new technologies. While these kinds of questions might be out of place on some single-topic markets modelled after the University of Iowa's markets on elections, the internal corporate markets that they also mentioned often use them to help maintain interest. The letter's recommendations that the CFTC "allow contracts that price an economically meaningful risk or uncertainty" unnecessarily limits the kinds of contracts that would be allowed.

Back on the side of supporting the letter's authors again, I'd have to admit that if the CFTC or Congress acts to implement anything resembling the recommendation it would very likely increase Prediction Market activity greatly, and eventually lead to a broader acceptance. If the initial definition is too narrow, however, questions that don't have clear economic implications (in the view of Congress and the regulators) might be stuck offshore for a long time to come.

Friday, May 04, 2007

Helping People find Me

Every once in a while, someone contacts me and says they had been trying to reach me for a while, but had an old email address. Then I find out that the address they have is one I haven't used for a decade or more. Brian Warner suggested that people who have several old, defunct addresses should publish them on the web someplace where Google might find and index them, so people who still have the old address will be able to get back in touch. I set up such a redirector about a month ago, and my old addresses page now comes up as the first result for most of my old addresses. It hasn't yet succeeded in helping someone reach me who couldn't any other way, but it might soon, since some of the easiest places to find my name don't provide working addresses any more.

Wednesday, May 02, 2007

Cato on Employment Verification

Jim Harper of the Cato Institute testified at the House's Hearing on Proposals to Improve the Electronic Employment Verification System last week, and made some very important points about privacy and government intrusiveness. Here are some quotes. There is much more valuable material in the complete text.

[I]mprovements that prevent eligible citizens from working should not be adopted. It is more important that American citizens and eligible people should be able to work than it is to exclude illegal aliens from working. There is probably no way to change the current system so that it prevents more ineligible people from working without also preventing more citizens and eligible people from working. [...]

The policy that will dissipate the need for electronic verification by fostering legality is aligning immigration law with the economic interests of the American people. [...]

Because the I-9 process and employer sanctions seek to defeat their economic interests, the system has two principle opponents: employers and workers. It relies on them for implementation, though, which is why success has been so elusive and will continue to be. [...]

Let there be no illusion that people seeking redress for a "tentative nonconfirmation" from the Social Security Administration or the Department of Homeland Security will enjoy a pleasant, speedy process. [...] People will wait in line for hours to access bureaucrats that are not terribly interested in getting them approved for employment. [...]

Electronic verification would have far greater privacy consequences than the current system — and these consequences would fall on American citizens, not on illegal immigrants. [...]

Ironically, all of this government spending and expanded bureaucracy would go toward preventing productive exchanges between employers and workers. Taxes and spending would rise to help stifle U.S. economic growth. Astounding.

Sunday, April 22, 2007

Cell Phones in the Third World

This year's Edge question: "What are you optimistic about?" generated more responses than earlier years' questions, so it has been taking me a while to get through the whole thing. The best line I've read so far (I'm on page 13 of 16) is by Sandy Pentland of MIT:
The International Telecommunications Union estimates that in the poorest countries each additional cell phone installed adds $3000 to the GDP, primarily due to the increased efficiency of business processes.

Sunday, April 01, 2007

Lee Smolin, The Trouble with Physics

Lee Smolin's book The Trouble with Physics is a very well written overview of the state of modern physics. I've been trying to keep up with String Theory, Quantum Theory, wormholes, and all the rest for quite some time, but until now I hadn't noticed that I was looking at trees and didn't know which forest any of them were in. Smolin's book gives a feeling for the lay of the land while he sets about laying waste to String Theory, which has been the great hope for a final unified theory for 25 years or so. I want to come back to the survey, but I'll cover what Smolin has to say about String Theory first, since that was his focus.

String Theory, it turns out, is a family of theories which matches what we know about the universe in some telling details, but which has too many loose ends to give specific predictions on any of the things we know we don't know. The basic idea gives an outline for what a more specific theory would look like, but there's nothing in the outline that gives a rigorous suggestion for how to fill in about thirty parameters that it leaves open. People in the field have been expecting that if someone does the right experiment or makes the right guess about one parameter, that the equations will fall into place, and the rest of the implications will be obvious. But it hasn't happened yet, and the brightest people in physics have been pursuing this path for a couple of decades.

One of Smolin's simplest demonstrations that the field has been unproductive recently is a description of the steady progress that has been made in fundamental physics for more than 200 years. From 1780 through 1980, there wasn't a quarter century that passed without important new contributions to our understanding of the basics of matter, time, energy, and motion. But most of what has been added since 1980 is String Theory, along with Dark Matter and Dark Energy. But Dark Matter and Dark Energy are questions, not answers and String Theory doesn't make any predictions that aren't made by the Standard Model, which was completed in 1973. String Theory also doesn't seem able to produce a theory that incorporates the background independence that general relativity requires.

Smolin is convinced that a major cause of the problem is that a form of groupthink has enveloped all of theoretical physics, enforced (probably unconsciously) by the oldest generation of theorists. He shows that the standard signs are all present, and shows how it has restricted researchers' choices of what to areas to explore. It's a plausible charge; I've seen compelling cases in at least two other fields that similar dynamics prevented progress for decades at a time.

Smolin ends with a plea that more research effort be funded in a variety of areas that are at odds with mainstream String Theory. He does a reasonable job of showing that they have some promise, though I'll admit that I'm way out of my depth by the time he gets here.

But I was really impressed by the first section of the book, in which Smolin presented all of fundamental physics from Copernicus, Bruno, and Galileo to the present day in a very approachable format. The focus was consistently on what pre-existing concepts were brought together in one of two ways. Sometimes the unification shows that two familiar things that are thought of as distinct are really the same thing, giving a deeper theory of both. (the Earth is one planet among several, the Sun is one star among many.) Other times, two phenomena that weren't understood well are explained as one common thing (Bacon showed that heat is a kind of motion; Newton showed that gravity explained both planetary orbits and ballistic trajectories; Maxwell showed that electricity and magnetism are the different aspects of the same phenomenon.) All of this is stuff that I have understood since high school, but viewing it this way gives me a hook to hang a few more things on. The rest of physics constitutes theories that I've been able to understand at a surface level, but which have never been integrated in any deep way. Smolin's framework makes it possible for me to hold a few more theories in my head and see how they give shape to the entire universe.

Einstein was responsible for the next three major unifications. Special relativity forces space and time to be interchangeable in order for the speed of light to be an observer-independent limit. (Mass and energy, too, must be epiphenomena of some more fundamental construct.) General relativity showed that gravity is indistinguishable from acceleration, and that space has no geometry independent of the mater and energy that it contains. These are still a little hard for me to grok completely, but I feel much more knowledgeable about them post-Smolin. Previously, I was satisfied with being familiar with the equations, even though I didn't really understand what they implied about the universe. Now I feel like I understand them as facts about the universe, though of course it's a stretch to claim I understand how they fit together.

Even with QED, QCD, supergeometry, and the two DSRs, I am much happier that I understand what they are theories about than I was before. I've read several books on the frontiers of modern physics, but I'd have been hard pressed to say, more than a month after putting each down, what they implied about the universe. With Smolin's help, I now feel like I understand what the point of most of these theories are, and what it would mean to bring them together.

Tuesday, March 27, 2007

Prometheus finalists announced

This year's Prometheus finalists have been announced. The links from the titles are to my reviews.

The finalists for Best Novel are: * Empire, by Orson Scott Card * The Ghost Brigades, by John Scalzi * Glasshouse, Charles Stross * Rainbows End, by Vernor Vinge * Harbingers, by F. Paul Wilson

And the finalists for Hall of Fame are: * A Clockwork Orange, a novel (1963) by Anthony Burgess * "As Easy as A.B.C.," short story (1912) by Rudyard Kipling * It Can't Happen Here, a novel (1936) by Sinclair Lewis * Animal Farm, a novel (1946) by George Orwell * The Lord of the Rings, a trilogy of novels (1954) by J.R.R. Tolkien * "True Names," a novella (1981) by Vernor Vinge

I haven't written reviews of any of the hall of fame nominees, though I've read them all. Perhaps that's a sign that I should read them again.

Saturday, March 24, 2007

Robert A. Heinlein and Spider Robinson: Variable Star

The author of Variable Star is listed as Robert Heinlein and Spider Robinson, but it's really a recently discovered outline by Heinlein that Robinson has turned into a novel. The story has a very anachronistic feel to it, with many heinleinian plot points (it appears to be set in Hienlein's Future History) but references to recent events and relatively modern mores.

I don't much care for stories with a reluctant hero, and Joel Johnston has to be dragged, kicking and screaming, through each of his life transitions. He's actually a fairly decent character in between, but the switch-overs are wrenching.

The story was a nominee for the Prometheus award, but wasn't chosen as a finalist. It doesn't have much to say about freedom or self-direction; it's not a cautionary tale; the governments are benign and consensual or required by circumstances (a 500 person starship has to have a captain and someone to keep the peace) and minimal, but in either case, neither something to fight against or to hold out as a model. The characters are pushed around by fate and circumstance.

The story takes so many hairpin turns that there's not much else I can say without spoiling the plot. Talking about anything after the first 50 pages would tell way too much about the most surprising development, and after that, things are pretty much pre-ordained other than the details. I'd rather read a real Heinlein juvenile, and make allowances for Heinlein's atavatistic viewpoint or a modern story by Robinson (with all the puns) than this mishmash.

Thursday, March 15, 2007

Would you Trade your Sense of Smell for a Longer Life?

Recent experiments on fruit flies undergoing Caloric Restriction seem to show that the effect of CR is reduced or eliminated if the flies are exposed to the odor of food (yeast in their case). That led to experiments in which the experimenters genetically eliminated the flies' ability to smell. They found a stronger contribution to lifespan from smelling the food than from how much the flies ate.

Would you give up your sense of smell if it meant living 50% longer? The question is only hypothetical at this point: replications in other species haven't been reported yet.

Saturday, March 10, 2007

F. Paul Wilson, Harbingers

F. Paul Wilson's Harbingers is a nominee for the Prometheus award. I understand the impulse to nominate the Repairman Jack novels for the Prometheus: Jack is a strong character who fights evil and eschews all contact with the government while living in New York City. His attitude toward self-help and antipathy to gun control also help his case. It's probably the fact that it's horror, and that both the cosmic bad guys and the cosmic okay guys are inscrutable that drives me away.

With this installment, I realized that there was another aspect that was bugging me. In the past 9 novels, little has changed about Jack's relationship to the two warring factions that are thoughtlessly interfering with Jack's life. Harbingers, on the other hand, contains several significant developments in that relationship. In fact, it reads like the second volume of a trilogy. Too bad it took Wilson 9 books (plus 6 others set in the Adversary world) to get us here.

But overall, I don't find the libertarian themes to predominate over the standard horror tropes. And inscrutable powers for whom we barely even qualify as pawns aren't much of a recommendation for a libertarian point of view.

In its defense, this book does have Jack standing up to the Ally after it attempts to take away what he most cherishes. But the negotiation ends with the Ally seeming to hint it won't hurt Jack's family for as long as he does what it wants. Not much of a recommendation.

Wednesday, March 07, 2007

Orson Scott Card: Empire

Orson Scott Card's Empire has been nominated for the Prometheus award. It stands a chance as a cautionary tale like Sinclair Lewis's It Can't Happen Here. The story covers a plot (with tentacles in the White House) to overthrow the government and bring about a left-wing coup. The action is intense, but the politics is strained. Like Lewis' novel, the point (as Card explains in an afterward) is to convince us that it wouldn't take much to turn our current heavily factionalized politics to open warfare.

The story reveals that far-left and far-right groups have been preparing plans in secret for years to take control of the government violently if it should become necessary (because the other side has gotten too much control.) The leftist radicals apparently have the advantage, since they have an extremely wealthy fanatical backer who has been funding the development of new high-tech weapons and staffing and training an army to use them. This part unfortunately reflects the fact that the backers of Card's project are game developers; the new weapons are several steps ahead of what the uniformed military has access to, but would play well on the video game consoles.

The viewpoint characters are sympathetic, well-trained special ops soldiers, and are patriotic to a fault. Their loyalty is to the nation and its legitimate leaders. The action is intense. But ultimately, the story falls short of being a cautionary tale. The motives of extremists on both side are hard to fathom; they want control of the levers of power to prevent the other sides extremists from doing something bad, but it's not clear what. After the conspirators take control of New York City and get sympathetic resolutions from a few state legislatures, they don't make any visible changes in anyone's lives.

At least Sinclair Lewis showed that the fascists who took over would have to suppress dissent brutally and ruthlessly in order to maintain control. In Card's novel, life goes on as before. At the end of the novel the viewpoint characters suspect the newly elected president of having helped orchestrate the entire coup in order to ensure popular support for his own election as a peacemaker. But they can't find any indication that he has dastardly designs to justify the conspiracy they are looking for.

The attempt at a coup and rebellion were short-lived, caused only a handful of deaths and a moderate amount of damage, and didn't have any impact on everyday life. As a cautionary tale, it's not much to worry about. If the protagonists hadn't done such a good job of protecting the nation, an extremist megalomaniac might have taken over the government, but Card's novel doesn't explain why that was something to worry about.

Tuesday, March 06, 2007

Conditional and Combinatorial Betting

After people have used Prediction Markets for a while and have gotten used to their ability to provide forecasts, they start thinking about different scenarios. Who would be the best Republican to face Clinton? How are the prospects for a market boom or crash effected by the winner of the election? How will poverty be affected by a proposed World Bank program? These kinds of questions can be posed in a number of ways using Prediction Markets. Markets can allow betting on conditional (if) or conjunctive (and) questions. Either one can be used to answer the what if questions, but they provide different choices to the bettors, and some make it easier for observers to decode the answers.

The easiest compound question to pose is a simple conjunction of two others. InTrade had separate markets in whether Bush would be reelected in 2004 ("BUSH"), and whether Osama bin Laden ("OSAMA") would be captured before the election. Justin Wolfers and Eric Zitzewitz asked InTrade to add a single combined contract that would pay off if both came true. Their paper, Experimental Political Betting Markets and the 2004 Election shows how the prices on these three contracts can be combined to show how one event would be likely to effect the other.

InTrade created three separate claims to cover combinations of the two base questions. They were "Bush wins election" (BUSH), "Osama is captured before the election" (OSAMA), and the combination: BUSH&OSAMA which would have paid out if both the others came true. Wolfers and Zitzewitz estimated the market's conditional probability by comparing the price of OSAMA with the price of BUSH&OSAMA. If the price levels were rational, the difference between the two prices had to equal the chance that Osama would be captured and Bush would not be reelected. Since the market price of BUSH&OSAMA was 91% as high as the price of Osama, they concluded that that represented the conditional probability. A weakness of this conclusion is that while investors and arbitrageurs have an incentive to ensure that the price of BUSH is correct relative to ~BUSH, (and OSAMA with respect to ~OSAMA), there's no bet that lets an arbitrageur exploit superior knowledge of the conditional probabilities.

Sometimes investors believe they know how one outcome will effect another, and want to bet directly on that linkage. If you were confident before the election that Osama's capture would raise the probability of Bush's reelection to 95% (above the level the the market prices implied), having the conjunctive bets didn't provide a bet that would have looked beneficial to you. You might think you could buy Bush&Osama (because you believe Bush's chances are improved if Osama is captured) and sell ~Bush&Osama (because this is the outcome your view says is least likely), but you'd lose both bets if Osama wasn't captured (which is an outcome your prediction doesn't specify.)

Conjunctive claims allow observers to deduce connections between claims, but since the investors aren't directly rewarded based on the conditional probabilities, they have little incentive to ensure that the implicit conditional probabilities reflect their understanding of the connections between the outcomes. In order to evaluate different proposals we have to look at what investors would spend up-front, and then compare the possible outcomes and how the investor's earnings change in each situation.

If Bush is a 60% favorite to be re-elected, and the market thinks there's only a 10% chance Osama will be captured before the election, the odds on the conjunctions might be:

 Bush reelectedBush defeated
Osama captured .09 .009
Osama free .5 .4

If you think Osama's capture would improve Bush's prospects to 95%, what should you buy or sell? Your prediction says that the ratio of Bush&Osama to ~Bush&Osama should be 19:1, but doesn't have anything to say about Bush&~Osama or ~Bush&~Osama. If you buy Bush&Osama and sell ~Bush&Osama, you can make the prices match your beliefs better, but you'll lose money if Osama isn't captured. In order to support conditional bets directly, market operators have to find ways to allow traders to buy positions without exposing themselves to risks due to the independent cases.

A contract that acts like a conditional bet directly (written as BUSH|OSAMA, pronounced as "Bush given Osama" or "Bush conditional on Osama") would pay off if Bush is elected, and return your investment if Osama bin Laden isn't captured. That gives investors the right incentive.

 Bush reelectedBush defeated
Osama capturedGain $1 Lose investment
Osama free Return investmentReturn investment

In order to support betting on conditional probabilities, the bets have to be able to return the investors' money in particular cases. I know of three detailed proposals that have this property. They are: betting on arbitrary boolean expressions, representing the complete cross-product of possible outcomes (providing a complete set of Arrow-Debreu securities), and using the independent claim as currency for purchasing the dependent claim. There are two additional suggestions that might work, but haven't been written down in sufficient detail to be sure.

Robin described and implemented Combinatorial Information Markets which represent probabilities and traders assets explicitly for all possible combinations of outcomes. Fortnow, Kilian, Pennock, and Wellman described how you might try to support bets on arbitrary boolean combinations of conditions. Their conclusion seemed to be that solving the general problem would be computationally infeasible. They didn't describe how to address the problems they found, but I think it's possible that a market that supported only binary combinations could be designed. And finally, Peter McCluskey built (and released as open source) USIFEX in 1999. It allows the user to use the coupons of the independent event as the currency. This combination allows traders to express conditionals directly. Unfortunately, that system didn't attract a user base quickly enough, and Peter stopped development soon after the initial release.

For an article on Decision Markets written in 1999, Robin Hanson suggested creating markets using assets that pay off in "units of A if B passes" (and "... if B doesn't pass."), and allow traders to trade the assets for each other. The price of A|B in terms of B (which can be built from component assets) expresses the conditional bet. Robin didn't explain how to set up a market in which people trade assets for assets and didn't describe how to let the users see how various combination bets would express the conditional claims they might have been interested in. (This is the first of the two incomplete suggestions.)

Robin's Combinatorial Information Market design uses a complex internal representation and can support arbitrary conditional bets. He built a prototype implementation that allows the user to explore these conditionals by choosing assumptions, and then adjusting probabilities in the resulting hypothetical situations. I wrote a prototype of my own in E. Neither prototype is more than a proof-of-concept that the institution works, and neither has been operated for any general market. The strength of this approach is that users can express conditional connections between arbitrary claims; this aspect has been shown to be effective in a laboratory experiment. Robin ran tests of this market after he proposed its use for PAM, and there were apparently no problems in running it with 6 traders estimating all outcome combinations for 8 events. The glaring weakness is that it doesn't scale well. It's not clear how to build a version that would work even with a market with dozens of questions and hundreds of users. I'll describe this market in more detail in a future post in this series.

Peter McCluskey built USIFEX in 1999. It works quite differently and doesn't seem to have the performance problems of the other proposals. The primary idea for supporting conditional trading is that you buy units of A|B using units of B as currency when betting on a conditional question. The effect is that when buying A|B, you end up with coupons of ~B as part of the purchase, and that's what ensures you'll be repaid if the independent event doesn't occur. USIFEX is open source, but it hasn't been maintained since it was released in 2000. The code was resurrected for use in the Swiss MarMix exchange, (AFAICT without making any use of the conditional betting features). The biggest weakness of Peter's approach, as I recall, was that it would have taken a lot of users to ensure that the conditional markets weren't extremely thin. A longer description of USIFEX is also in the works.

Todd Proebsting built an implementation of the Hanson design that works without conditionals. Dave Pennock wrote up a description of Todd's approach, focused on the Market maker. I intend to describe the implications of Todd's approach for betting on conditionals in a future post. (This is the second incomplete suggestion.) I think it might be straightforward to extend Todd's approach to support conditional betting without running into the exponential growth of Robin's solution. The drawback is that the market operator has to separately capitalize and enable every conditional question that you want the system to support, while Robin's approach enables all of them by default. It's also possible that Zocalo Open Source Prediction Market software would be compatible with this approach, where it's clear that Zocalo would require substantial modification to support the Hanson proposal.

Other Articles in this series