New Zocalo release; OpenSource JavaMail incorporated

I just published a new release of Zocalo that includes the ability for new users to register for an account. I'm writing this note to report on what I learned from integrating JavaMail into Zocalo about Sun's progress on Open Sourcing the Java libraries. More than a year ago, I started down the path to integrate email into Zocalo for new account creation. On the web today, if users are to create their own accounts without requiring intervention from an administrator, the application pretty much has to be able to send mail. (You can do it just with a captcha to ensure they're a real person, but doing a round-trip confirmation is better, since you can update forgotten passwords, and you end up with the ability to send transaction reports.)

What I discovered at the time was that Sun had announced that they were going to open source all the Java libraries, but hadn't made much progress. The mail libraries hadn't been freed up, and in order to use any of the third-party open email packages I could find, I would have had to require that people installing Zocalo manually download the base package from Sun themselves. This seemed like enough of an additional burden on the installation process that I punted. I actually implemented a rudimentary utility function that would invoke a local process to send mail. I know how to make this work on Linux machines, and it could probably have been coerced into being usable on Macintoshes, but I don't know what it would take to make it work on Windows. And then I never made any use of that functionality, since it didn't seem likely to work on most platforms.

In early July, I decided it was time to revisit the issue, and spent some time searching through the status of Sun's open source efforts. I was pleased and surprised to find that they've actually released a lot of it. (It might be all of Sun's libraries for all I know. I couldn't quickly find an overview, history, or description of the source of all this code.) It all seems to be gathered together at the GlassFish project.

I was able to find the JavaMail area reasonably easily, though figuring out what jars I would need was somewhat harder. And figuring out what the open source terms were took some reading. The license itself seems pretty impenetrable to me, but I was reassured by re-reading the Open Source Definition at Open Source.org. Since OpenSource.org thinks that Sun's CDDL meets their definition, I don't have any qualms about shipping the GlassFish JavaMail libraries with Zocalo.

I then downloaded one of the releases (GlassFish's release numbering is confusing to me; I couldn't easily figure out which version was the best stable release for basic operations), and tried adding one jar file at a time to Zocalo, to see how much I would need to get things running. The GlassFish documentation implies that you have to buy into their whole paradigm to get it to work. There's lots of J2EE stuff, and a JavaBeans Activation Framework, and I don't know what-all. All I want to do is send SMTP mail via someone else's server. The person installing Zocalo will have to specify an SMTP server, and give a password (unless there's something running on the local machine, or the SMTP server is accessed securely on a LAN, but I haven't set those up yet.) I could have included an SMTP server, but the person installing still has to make a secure connection to an external server; it isn't any easier if you're a server than if you're logging in as an SMTP client. In the end, all I needed was activation.jar and mail.jar. No extra XML configuration files, no extra VM parameters, no hassle. There are a few more configuration parameters in Zocalo's startup files, but whatever I did, you'd have to specify the SMTP server and password somewhere.

In the end I was extremely pleased with how easy it was to use Sun's recently released JavaMail package. The 2007.2 release of Zocalo Open Source Prediction Markets makes good use of it, and it wasn't too painful to incorporate.

Thomas Barnett: TED talk

Thomas Barnett (author of "The Pentagon's New Map") gave an entertaining talk for TED. I listened to it via podcast, but re-watched it later, since it was clear from the audio that it was pretty visual. The US has an unstoppable military force. But what comes after it works its magic? You can agree or disagree with his prescriptions, but at least it's a start on discussing a new strategy for the US Military. The book (and apparently his newer book) have a lot more detail and are well worth reading.)

David Warsh: Knowledge and the Wealth of Nations

0.3

Chris Hibbert

David Warsh describes the develpment of economists' models of growth and progress in his book Knowledge and the Wealth of Nations . The story revolves around the origins and consequences of a particular paper ("Endogenous Technological Change" by Paul Romer, 1990) in order to give us a view into the way economists think and the way economic theory evolves.

The book spent more time on personalities and personality clashes that I would have preferred, but Warsh apparently wanted this book to say more about how economists work than about the ideas they've developed. As a result, you have to work a little harder to keep track of how economists' models of progress evolve from Adam Smith to the present.

It shouldn't be a surprise that models of the real world (in any field) usually start simple, and accumulate details as people discover areas where the models' predictions aren't sufficiently clear to answer questions that arise. The thing that's interesting and hard to track when looking backward is remembering where the gaps were, and the order in which the problems were addressed. Warsh traces the earliest descriptions of how business works, and how progress is made to Adam Smith and his description of the division of labor in his Pin Factory example.

But the earliest economists described everything in prose. When the models were formalized, they started out simple. Until quite recently, all formal models of production in an economy were static: they assumed the means of production didn't change over time. Often the eonomists who presented these models explicitly recognized that that was an important factor that was missing from their models, but they still had to start simple in order to have models they could manage.

The path of evolution of the models next added the idea of growth, but assumed that progress was constant and outside the influence of the manufacturers in the model. This allowed the economists to model and describe more sophisticated situations, but didn't match what people could see about how developing countries advanced over time. Romer's contribution to the field was to build a model that made entrepreneurial investments change the cost of doing business and the alternatives available to actors within the model.

Romer's model assumes that the technological advances produced as a side effect of investment are general knowledge (they are non-rival goods), but that some of the benefits can be monopolized by the developer for a time (they are partially excludable). One of the major consequences of this is that the model predicts that trade barriers prevent underdeveloped countries from advancing, and that borders that are open to trade all lesser developed nations to gradually catch up with their trading partners, since they can take advantage of the greater stock of knowledge in the market.

I've read several books over the last few years that point out that the idea of progress is failry recent in human history. Romer's contribution wasn't in noticing it, it was deciding it was important, and figuring out a way of bringing it into the models that economists use. Now that invention, discovery, and the sharing and hoarding of knowledge are explicit in the models, economists' recommendations to policy makers more often point in the direction of investment in education and new technology.

Along the way, Warsh presents an interesting history, and describes other ideas Romer and his colleagues were struggling with and how they led to the particular paths chosen. If you're interested in the history of this particular idea, or how economists (or scientists in general) work, it's an engaging book.

Judith Rich Harris: No Two Alike

0.3

Chris Hibbert

Judith Rich Harris's book No Two Alike is a followup to her previous work, The Nurture Assumption. In her first book, Harris explained that most of the non-genetic affects on the personalities of adults are a result of their interactions with peers rather than with their parents. She pointed out that many people want to believe (and prove) that parents are the major source of their children's personalities. According to The Nurture Assumption, the field of sociology has been confused for several decades and has been trying to distinguish nature and nurture, when they needed to be either distinguishing the effects of heredity from environment, or disentangling the environmental influences which include both parents and peers.

The best tool, according to Harris, for distinguishing the effects of heredity and environment consists of studies of twins. Comparing twins raised together and twins raised apart controls for genetic affects and allows us to see the effect of gross differences in environment. Comparing fraternal twins raised together with identical twins raised together holds the gross environment constant and makes it easier to see what differences are purely genetic.

In the new book, Harris focuses on why twins are so different, in order to isolate the causes of differences that aren't explained by other results. The existing literature says that some proportion of personality differences are due to genetics, and some proportion by each of various environmental causes: parents, wealth, neighborhood, etc. But a significant amount of variation remains that isn't apparently caused by any of these. Her focal example is that even siamese twins have different personalities, and they share all of their genes, and all of the environmental influences that anyone could hope to treat as responsible.

Harris' conclusion (skipping over most of her argument for the moment) is that there must be something driving each of us to be unique, and that means we have to find a distinction to enhance. The bottom line is that a significant part of personality (who we are) isn't determined by factors that we can examine or control. Each individual starts out with an endowment of heredity, and occupies an environment that isn't fully under their control, but the developing personality is still a negotiation between that individual and their context. If the strongest part of their innate tendencies is best suited for a niche that is already filled, they will look for a second best. When two identical individuals struggle to fill the same niche, some factor (random or not) will eventually determine a winner in each particular event, and at some point the effects of competition, if nothing else, will drive them to exploit different strengths. The different choices and different results in competition will magnify any differences, and over a reasonable lifetime, they will become recognizably different people.

Along the way, Harris spends a good deal of effort (successfully) demolishing other possible explanations (differences in environment, combination of nature and nurture, gene-environment interactions, environmental differences within the family, gene-environment correlations, and transferability of learning between situations). At the end she argues that she has demolished all the other possibilities and provided an argument for the one remaining theory (an innate drive for status), and so it must be true. But her argument for the specific mechanism is a little too weak, and it seems plausible that some variation or related description will fit the data a little better. I'm reasonably convinced that something drives us to differentiate, but it may not be purely a status drive. Two possible variations on her theory include drives for attention or to master something.

I found the style of Harris' presentation sometimes compelling, and sometimes distracting. She fit the presentation into the framework of a detective story. The presentation is salted liberally with examples from popular detective stories to show how attentive the detective has to be to details that have distracted other investigators. This worked for me when I was familiar with the detective in question (Sherlock Holmes, Kinsey Milhone), and didn't work when I hadn't read the stories (Alan Grant). I suspect that Holmes is the only one of these that is widely enough known that other readers would feel that they should get the references even when they don't.

On the whole, I think the book was successful in explaining that fundamental differences in personality are effectively the result of an innate drive that causes us to differentiate. The drive makes use of arbitrary differences in the material it has to work with (genes and environment). Parents do make a difference in the lives their children lead, but the ultimate person each child becomes isn't determined by parenting style at any gross level.

Safe Harbor for Prediction Markets

A group of distinguished economists wrote a public letter advocating a legal safe harbor for some small-scale academic prediction markets. I can see why they limited their goals as they did, and I agree that everything they advocated should be legal, but I think they may have limited their objectives just enough to prevent any big wins.

One thing that Prediction Market maven Chris Masse constantly argues is that Prediction Markets on dry subjects need to be accompanied by entertaining questions in order to to keep the audience's attention. The economists had good reasons for shying away from recommending that sports betting should be included, but there are many other topics that diverse markets could include that give traders a reason to check back in. The range from the obvious entertainment questions (movie earnings and oscar winners) to legislative outcomes (bills passing and control of particular legislative bodies) and introduction and market success of new technologies. While these kinds of questions might be out of place on some single-topic markets modelled after the University of Iowa's markets on elections, the internal corporate markets that they also mentioned often use them to help maintain interest. The letter's recommendations that the CFTC "allow contracts that price an economically meaningful risk or uncertainty" unnecessarily limits the kinds of contracts that would be allowed.

Back on the side of supporting the letter's authors again, I'd have to admit that if the CFTC or Congress acts to implement anything resembling the recommendation it would very likely increase Prediction Market activity greatly, and eventually lead to a broader acceptance. If the initial definition is too narrow, however, questions that don't have clear economic implications (in the view of Congress and the regulators) might be stuck offshore for a long time to come.

Helping People find Me

Every once in a while, someone contacts me and says they had been trying to reach me for a while, but had an old email address. Then I find out that the address they have is one I haven't used for a decade or more. Brian Warner suggested that people who have several old, defunct addresses should publish them on the web someplace where Google might find and index them, so people who still have the old address will be able to get back in touch. I set up such a redirector about a month ago, and my old addresses page now comes up as the first result for most of my old addresses. It hasn't yet succeeded in helping someone reach me who couldn't any other way, but it might soon, since some of the easiest places to find my name don't provide working addresses any more.

Cato on Employment Verification

Jim Harper of the Cato Institute testified at the House's Hearing on Proposals to Improve the Electronic Employment Verification System last week, and made some very important points about privacy and government intrusiveness. Here are some quotes. There is much more valuable material in the complete text.

[I]mprovements that prevent eligible citizens from working should not be adopted. It is more important that American citizens and eligible people should be able to work than it is to exclude illegal aliens from working. There is probably no way to change the current system so that it prevents more ineligible people from working without also preventing more citizens and eligible people from working. [...]

The policy that will dissipate the need for electronic verification by fostering legality is aligning immigration law with the economic interests of the American people. [...]

Because the I-9 process and employer sanctions seek to defeat their economic interests, the system has two principle opponents: employers and workers. It relies on them for implementation, though, which is why success has been so elusive and will continue to be. [...]

Let there be no illusion that people seeking redress for a "tentative nonconfirmation" from the Social Security Administration or the Department of Homeland Security will enjoy a pleasant, speedy process. [...] People will wait in line for hours to access bureaucrats that are not terribly interested in getting them approved for employment. [...]

Electronic verification would have far greater privacy consequences than the current system — and these consequences would fall on American citizens, not on illegal immigrants. [...]

Ironically, all of this government spending and expanded bureaucracy would go toward preventing productive exchanges between employers and workers. Taxes and spending would rise to help stifle U.S. economic growth. Astounding.

Cell Phones in the Third World

This year's Edge question: "What are you optimistic about?" generated more responses than earlier years' questions, so it has been taking me a while to get through the whole thing. The best line I've read so far (I'm on page 13 of 16) is by Sandy Pentland of MIT:

The International Telecommunications Union estimates that in the poorest countries each additional cell phone installed adds $3000 to the GDP, primarily due to the increased efficiency of business processes.

Lee Smolin, The Trouble with Physics

0.3

Chris Hibbert

Lee Smolin's book The Trouble with Physics is a very well written overview of the state of modern physics. I've been trying to keep up with String Theory, Quantum Theory, wormholes, and all the rest for quite some time, but until now I hadn't noticed that I was looking at trees and didn't know which forest any of them were in. Smolin's book gives a feeling for the lay of the land while he sets about laying waste to String Theory, which has been the great hope for a final unified theory for 25 years or so. I want to come back to the survey, but I'll cover what Smolin has to say about String Theory first, since that was his focus.

String Theory, it turns out, is a family of theories which matches what we know about the universe in some telling details, but which has too many loose ends to give specific predictions on any of the things we know we don't know. The basic idea gives an outline for what a more specific theory would look like, but there's nothing in the outline that gives a rigorous suggestion for how to fill in about thirty parameters that it leaves open. People in the field have been expecting that if someone does the right experiment or makes the right guess about one parameter, that the equations will fall into place, and the rest of the implications will be obvious. But it hasn't happened yet, and the brightest people in physics have been pursuing this path for a couple of decades.

One of Smolin's simplest demonstrations that the field has been unproductive recently is a description of the steady progress that has been made in fundamental physics for more than 200 years. From 1780 through 1980, there wasn't a quarter century that passed without important new contributions to our understanding of the basics of matter, time, energy, and motion. But most of what has been added since 1980 is String Theory, along with Dark Matter and Dark Energy. But Dark Matter and Dark Energy are questions, not answers and String Theory doesn't make any predictions that aren't made by the Standard Model, which was completed in 1973. String Theory also doesn't seem able to produce a theory that incorporates the background independence that general relativity requires.

Smolin is convinced that a major cause of the problem is that a form of groupthink has enveloped all of theoretical physics, enforced (probably unconsciously) by the oldest generation of theorists. He shows that the standard signs are all present, and shows how it has restricted researchers' choices of what to areas to explore. It's a plausible charge; I've seen compelling cases in at least two other fields that similar dynamics prevented progress for decades at a time.

Smolin ends with a plea that more research effort be funded in a variety of areas that are at odds with mainstream String Theory. He does a reasonable job of showing that they have some promise, though I'll admit that I'm way out of my depth by the time he gets here.

But I was really impressed by the first section of the book, in which Smolin presented all of fundamental physics from Copernicus, Bruno, and Galileo to the present day in a very approachable format. The focus was consistently on what pre-existing concepts were brought together in one of two ways. Sometimes the unification shows that two familiar things that are thought of as distinct are really the same thing, giving a deeper theory of both. (the Earth is one planet among several, the Sun is one star among many.) Other times, two phenomena that weren't understood well are explained as one common thing (Bacon showed that heat is a kind of motion; Newton showed that gravity explained both planetary orbits and ballistic trajectories; Maxwell showed that electricity and magnetism are the different aspects of the same phenomenon.) All of this is stuff that I have understood since high school, but viewing it this way gives me a hook to hang a few more things on. The rest of physics constitutes theories that I've been able to understand at a surface level, but which have never been integrated in any deep way. Smolin's framework makes it possible for me to hold a few more theories in my head and see how they give shape to the entire universe.

Einstein was responsible for the next three major unifications. Special relativity forces space and time to be interchangeable in order for the speed of light to be an observer-independent limit. (Mass and energy, too, must be epiphenomena of some more fundamental construct.) General relativity showed that gravity is indistinguishable from acceleration, and that space has no geometry independent of the mater and energy that it contains. These are still a little hard for me to grok completely, but I feel much more knowledgeable about them post-Smolin. Previously, I was satisfied with being familiar with the equations, even though I didn't really understand what they implied about the universe. Now I feel like I understand them as facts about the universe, though of course it's a stretch to claim I understand how they fit together.

Even with QED, QCD, supergeometry, and the two DSRs, I am much happier that I understand what they are theories about than I was before. I've read several books on the frontiers of modern physics, but I'd have been hard pressed to say, more than a month after putting each down, what they implied about the universe. With Smolin's help, I now feel like I understand what the point of most of these theories are, and what it would mean to bring them together.

Prometheus finalists announced

This year's Prometheus finalists have been announced. The links from the titles are to my reviews.

The finalists for Best Novel are: * Empire, by Orson Scott Card * The Ghost Brigades, by John Scalzi * Glasshouse, Charles Stross * Rainbows End, by Vernor Vinge * Harbingers, by F. Paul Wilson

And the finalists for Hall of Fame are: * A Clockwork Orange, a novel (1963) by Anthony Burgess * "As Easy as A.B.C.," short story (1912) by Rudyard Kipling * It Can't Happen Here, a novel (1936) by Sinclair Lewis * Animal Farm, a novel (1946) by George Orwell * The Lord of the Rings, a trilogy of novels (1954) by J.R.R. Tolkien * "True Names," a novella (1981) by Vernor Vinge

I haven't written reviews of any of the hall of fame nominees, though I've read them all. Perhaps that's a sign that I should read them again.

Robert A. Heinlein and Spider Robinson: Variable Star

0.3

Chris Hibbert

The author of Variable Star is listed as Robert Heinlein and Spider Robinson, but it's really a recently discovered outline by Heinlein that Robinson has turned into a novel. The story has a very anachronistic feel to it, with many heinleinian plot points (it appears to be set in Hienlein's Future History) but references to recent events and relatively modern mores.

I don't much care for stories with a reluctant hero, and Joel Johnston has to be dragged, kicking and screaming, through each of his life transitions. He's actually a fairly decent character in between, but the switch-overs are wrenching.

The story was a nominee for the Prometheus award, but wasn't chosen as a finalist. It doesn't have much to say about freedom or self-direction; it's not a cautionary tale; the governments are benign and consensual or required by circumstances (a 500 person starship has to have a captain and someone to keep the peace) and minimal, but in either case, neither something to fight against or to hold out as a model. The characters are pushed around by fate and circumstance.

The story takes so many hairpin turns that there's not much else I can say without spoiling the plot. Talking about anything after the first 50 pages would tell way too much about the most surprising development, and after that, things are pretty much pre-ordained other than the details. I'd rather read a real Heinlein juvenile, and make allowances for Heinlein's atavatistic viewpoint or a modern story by Robinson (with all the puns) than this mishmash.

Would you Trade your Sense of Smell for a Longer Life?

Recent experiments on fruit flies undergoing Caloric Restriction seem to show that the effect of CR is reduced or eliminated if the flies are exposed to the odor of food (yeast in their case). That led to experiments in which the experimenters genetically eliminated the flies' ability to smell. They found a stronger contribution to lifespan from smelling the food than from how much the flies ate.

Would you give up your sense of smell if it meant living 50% longer? The question is only hypothetical at this point: replications in other species haven't been reported yet.