IFTF's Future Now

Deconstructing turbulence

via FutureFeeder-

Richard Feynman once called turbulence the most important unsolved problem in classical physics - so it's notable that MIT researchers are making some headway in understanding its underlying structure.  They tracked the motion of minuscule particles in a turbulent fluid using lasers, high definition cameras, and software to follow the particles over small increments of time.

MIT researchers report that they have visualized for the first time a convoluted tangle underlying turbulence. This work may ultimately help engineers design better planes, cars, submarines and engines.
...
The researchers identified a complex network of two types of curves formed by two distinct groups of particles. The first type of curve, which the researchers colored red, attracts other fluid particles. At the same time, the second type, colored blue, repels other fluid particles. Both sets of curves evolve with the flow.
...
The MIT researchers call their discovery the "Lagrangian skeleton" of turbulence because their particle-based approach is motivated by the work of 19th-century mathematician Joseph-Louis Lagrange. "Lagrange developed mathematical tools still used today for calculating mechanical and fluid motion," said Peacock.

Swarm Agent Modeling Wiki

Swarm is an agent modeling system originally developed at the Santa Fe Institute. Its now run by a public consortium, the not-for-profit Swarm Development Group. They have just re-designed their wiki site, which is rich in content about Agent Modeling and the Swarm system.

Simulating Realities

All business is a forecast, and all forecasts are a kind of simulation of the future. In recent years there has been increasing interest in how to do simulation and how simulation can get you a leg up on what the future will be. James Swain, a former professor and advisor of mine, publishes an overview article on simulation capabilities in ORMS/Today. Included is a survey of dozens of packages, their capabilities and uses. This is more about packages than it is about vendors, but many package makers will do work for you or point you to those that can.

Cellular Phones and Cellular Automata

A colleague recently sent me a pointer to WolframTones, a clever site developed by Wolfram Research that uses cellular automata concepts to generate musical compositions that can be used as cellphone ringtones as a new kind of music. We have followed Wolfram's work for some time. Most notable is his book A New Kind of Science, which seeks to replace standard science with lots of cellular automata. Cellular automata are totally unrelated to cellular phones. The invention of CA's are often attributed to Manhattan Project physicist Stanislaw Ulam. In the 80s they were reapplied by Los Alamos Labs scientist Christopher Langton, to create a discipline called artificial life. We met Langton in the 90s as he pushed the modeling capability of AL, which then morphed into the modeling called Agent-Based Modeling, or ABM. That technology is being used to build corporate simulation models. The kind of models that Wolfram is proposing to replace conventional science are very different from those used for ABMs, but the general concept is the same: individual agents interacting in relatively simple ways generate complex behavior. Amazing that Ulam's phsyics would end up in cell phone tones and corporate models.

Making Stick Figures Walk

A clever animation of a Genetic Algorithm process (GA), instructive for understanding how difficult problems can be solved with this method. A GA is designed to solve problems that are too difficult to solve using mathematical optimization .. they search through lots of options by using a method inspired by genetics: using population, mutation and crossover. In this simple example a stick figure is 'taught' to walk using a genetic method. Via the Illinois GA Lab blog. Also, GAs and their application.

Considering the Nature of Chatter

I recently reviewed Patrick Keefe’s new book Chatter .... I won’t describe that book here, but I did post a critical review of it elsewhere. The book did make me think about the term and how it has come to be used to describe the activity of terror groups. How do we think about the changes in such signal intelligence over time, determine when changes are significant, and consider the causal effects that influence the changes?

Security agencies think about chatter in terms of terror level and how we should react, but many other industries are starting to have access to data streams that can be analyzed in terms of signals and their changes over time. Point-of-sale data sources, call centers, increasing use of sensors, video-image analysis and RFID tagging are producing an avalanche of rich data sources, which can be studied in real time. Even the notion of ‘real-time’ has started to evolve. Where in the past analysis of commercial data may have been performed monthly or quarterly, now transactions can be tapped into to provide data directly from the cash register or RFID scanner to use much smaller time slices.

So how do we know when an observed measure has changed? In the 1920’s Walter Shewhart of Bell Telephone Labs developed the Control Chart, a statistical means of observing such streams and determining the significance of changes. Later Deming was to lionize them for understanding the operation of processes. These methods are starting to re-emerge as a means for understanding data streams. These are powerful techniques, but also rely on statistical assumptions that may not hold for new forms of data.

How do we determine what has caused changes in a data stream? Intelligence communities have to worry that terrorists may be manipulating the chatter level. In the same way we need to understand what causal elements can be influencing any set of signals we are observing.

Also important is the idea that processes are not just all about a single measure, they are often dependent on multiple streams of information, with varying time dimensions, accuracy and availability. Newly purposed Bayesian techniques are now being used to model and analyze such multiple data-stream systems.

Companies are emerging to address these problems. Here is a recent article on some vendor activity in this area. Notable is the University of Illinois startup Riverglass, which "... has developed software that merges data from multiple, disparate data streams--including unstructured text and numeric data--and applies real-time data modeling and analysis techniques to those streams. The goal is to detect patterns in the data to identify potential investment risks and opportunities ..."

Chatter is now all around us, and as we continue to develop new means to generate data though sensor networks, we will need to understand what it all means, and link it to decisions we can ultimately make. There is much work to be done.

Social Modeling: JASSS Journal

I neglected to mention it in my previous note on agent-based simulation, but an excellent source of information on social simulation, agent-based and otherwise, is the online Journal of Artificial Societies and Social Simulation. I have followed it for years. To get a good feeling for how this area is progressing, its useful to browse some of their recent titles. It shows current progress and also the challenge of using social simulation to understand systems of people.

Agents of Change: Social Modeling

A good overview article on Agent-Based Modeling (ABM) philosophy, techniques and applications, from the February ORMS Today, by Douglas Samuelson. He makes the case that this technique will transform social science because of the natural linkage between social and agent entities and their inclusion of behavioral changes. He also links ABMs to classical simulation techniques and Systems Dynamics. Some useful further links.

A Wisdom of Models?

I recently presented at the Informs Roundtable Meeting on Data Mining. I will post several short individual items on useful learnings.

Data mining is about finding useful patterns in data, frequently from data that was not gathered explicitly to solve a current problem at hand. Modeling methods for data mining include neural nets, regression, genetic algorithms and classification approaches. Dr John Elder of Elder Research promotes the idea of trying a number of models and then combining the results in an ensemble answer, the simplest approach would be to average the results ... a kind of Wisdom of Models, see his paper on why an ensemble of models can do better.

Genetic Blogging

The comments on my recent post on the Value of Genetic Design pointed me to an interesting blog on Genetic Methods, see: IlliGAL Blogging: Life, Liberty, and the Pursuit of Genetic Algorithms. David Goldberg and others are contributing, contains useful technical details and excellent application examples. I see also that they are now pointing back to us as well, welcome.