More on Models

Computer modeling is a phrase that seems to always draw respect.

This level of respect has always been puzzling to me. I have done a little computer modeling (in my time at a Tech University), and I learned early on that the "Computer" part is the last part of the process of Computer Modeling.

I also learned that the biggest weakness of Computer Modeling is in the model.
Earlier, I talked about an astronomer and astrologer who spent a large portion of his life working on a model for planetary orbits. Kepler, a contemporary of Galileo, was trying to figure out a shape for his model of Mars' orbit that would produce predictions lining up with the years of observations supplied by another astronomer.

Kepler did not have a computer (or even a slide rule). His results would not have been noticeably different if he did, with the exception that a computer would have compressed years of work with pen and paper into hours or minutes.

Modern scientists use other kinds of models, often with the help of computers, to study hurricanes, the structure of the Milky Way galaxy, snowfall patterns, and the population size of endangered species.

In each case, the model used is a simplified version of the real thing being modeled. Computer models for local weather will often provide good predictions for 3-5 days, and estimates for 5-10 days. Computer models for the long, slow evolution of the MIlky Way galaxy appear to be correct over million-year timespans.

Models for endangered species and their survival appear to be fairly good, as long as the biologists know and can predict all the population pressures involved. (An unexpected change in temperature, availability of food, or presence of predators can cause the population to shrink rapidly, or grow unexpectedly.)

In all these cases, the modelers should have followed this process:

1. Create model.
2. Run short test on model, using data to predict results for situations which have already been studied.
3. Compare model output to known behavior of the real system.
4. Adjust model if necessary, and repeat steps 2 and 3.
5. Once model is producing results that match reality, use it to predict future events.
6. Once one of these future events happen, compare it with the predicted result of the model. If there is a difference, repeat steps 2 and 3.
7. Repeat step 6 as necessary, and then attempt to publish results in a respected scientific journal.
   

I note that steps 3, 4, 5, and 6 are kind of hard to do for the modelers of the Milky Way galaxy. (This is why astronomers talk about Dark Matter...certain models produce results far from the observed reality, which could only be explained if there's a lot of matter out there that can't be seen.)

For models about next week's weather, next year's population-counts of California Condors or Blue Whales, where the hurricane-of-the-month will hit land, or where the astronomers should look in the sky to find Mars, these steps are fairly straightforward.

Whenever I hear a news story (aside from weather forecasts) that depends on computer modeling, I wonder how much the model has been tested. It might be that the model only works if certain inputs are assumed constant. (As long as there are no other planets out there, Uranus should appear at a particular point in the sky on a certain date...)

It might also be that the researcher doesn't have enough historical data to test his model against. (How did changes in solar output, cloud cover, and CO₂ levels influence the Holocene Climatic Optimum? If we don't know that our model is right, we can't really test the model against that time period unless we have good data on all the variables involved.)

Usually, the process of peer review and other researchers testing the model with new data solves these problems. But that process doesn't always make it into public awareness.

Models are useful things. Computer models of amazing complexity can be constructed, and can be very useful. But that usefulness depends on whether the model has been tested properly against the real-world system it claims to represent.

Posted by: karrde at 19:59 | No Comments | Add Comment
Post contains 685 words, total size 4 kb.