« Mashed Up | Main | links for 2005-04-22 »

Diversity and Problem Solving

What if, as part of the human condition, we naturally underperform when trying to solve problems together?

There's a fascinating paper by Lu Hong and Scott E. Page called "Diversity and optimality"[PDF]. As they say in the abstract,

In this paper, we construct a general model of cognitively diverse problem solvers. We use this model to derive two main results: (1) a collection of cognitively diverse problem solvers can locate optimal solutions to difficult problems and (2) a collection of diverse problem solvers with limited abilities tends to outperform a collection of high ability problem solvers, where a problem solver's ability equals her expected individual performance
[thanks to Dennis Pearce over at AOK for the pointer]
In other words, to have a great problem solving team, you need talent, yes, but you also need diversity.

This seems to sit very uneasily with homophily and the whole "birds of a feather flocking together" shebang. For reference, here are some views from sociology and psychiatry/psychology.

"attitude, belief and value similarity lead to attraction and interaction."
- McPherson, Miller; Smith-Lovin; Cook. 2001. "Birds of a Feather: Homophily in Social Networks." Annual Review of Sociology 27: 415-444.
And from the same piece, while "similarity breeds connection," it also seems to hold that "Ties between nonsimilar individuals ... dissolve at a higher rate." From the psychology field, a quick browse turned up this:
People who suffer from the same problems tend to seek together, either unconsciously or for support. Once to many, I've seen people who have the same problem refuse to confront it, and as a result actually accelerate each other's psychopathology.

This occurs relatively fast and is one of the reasons why group therapy may be very effective. In group therapy, the interaction between people and their symptoms are supervised by the therapist, who (hopefully) will confront the clients when appropriate.

And lastly, as Dennis Pearce was (again)quoted as saying last September,
(Tom) Allen also found that the engineers he surveyed got their knowledge from the sources that were easiest and most familiar to them, not the ones that were most reliable or accurate, even when they themselves were aware of this. In other words, they knowingly sacrificed accuracy for expediency.
[thanks Suw for the pointer]
[The work in question is Allen, T. (1977). Managing the Flow of Technology. Cambridge, MA: MIT Press.]

In short, to optimise our group problem-solving we need to make sure our groups are diverse, but nature, or rather our natures, shoot us in our feet.

Other random associated thoughts, if not particularly thought out, and references:

  • What if flaming was a sign of a vibrant problem-solving community? Is there a point at which debate tips and damages the chances of a solution?
  • How fine is the line between group kitsch and group ethos?
  • What does all this mean about the value of the outliers (like the madman in the corner) to the group in terms of solving their problems
  • Can and if so how would one design for optimal unfamiliarity? [That phrase has really stuck, Ton ;)]
  • Do both hierarchies and "bottom-up" modes of organisation suffer equally from the problem of natural underperformance? (if it is a problem)
  • Do some of the intuitively great ideas about Open Space conferences actually accelerate group "psychopathology"?
  • Implications for HR etc

Hmm. Very possibly all non-starters, but hey.


TrackBack URL for this entry:

Listed below are links to weblogs that reference Diversity and Problem Solving:

» Knowledge Retention and the Importance of Forgetting from Monkeymagic
Three things happened in conjunction this morning that made me wonder whether memory and experience have some underestimated drawbacks. In order, I saw Jack's post about Knowledge RetentionMy server stopped working because it was full, andI read a pape... [Read More]


See also Johnnie Moore's thoughts on diversity in groups that is hidden but nevertheless still there.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)