King and Levin take on aggregation

Brayden King of orgtheory.net cites a helpful passage from Peter Levin on aggregation and crowdsourcing:

More theoretically, it has never really be adequately explained why a ‘market-like’ information crowd-sourcing should work. I understand why markets might produce a price that incorporates most public and private information about a commodity. But the widespread substitution of expertise with data mining and crowd-sourcing is a market metaphor more than a market. Why should a metaphor work? This is at the heart of someone like Daniel Davies’ criticism. And I get that sometimes aggregation does work. But there’s no good reason why.

My own feeling is that, using March’s metaphor of ‘exploitation’ and ‘exploration’ (where the first is the plumbing of existing knowledge/arenas, and the second is the seeking out of new opportunities), aggregation mechanisms are better at exploitation than exploration. They do better with existing standards of knowledge, of tastes, of commodities, than they do with something that is new.

Brayden is not totally buying it, by the way:

The key to channeling knowledge usefully is to find the “right” aggregation method. Sorting through opinions in a discussion board is probably the least effective way to aggregate opinions. Averages, as long as opinions are independent from one another and representative of the population, are pretty useful in figuring out what the general perception of something is. If you want to know if a movie is any good or not, check out Rotten Tomatoes or Metacritic and pay attention to the average rating. Pure averages though, while probably useful for the average consumer, can also be less than useful if your tastes happen to deviate in non-random ways from the rest of the population. And so what we’d like, ideally, is an average opinion or rating from someone who has the same taste set as we do. Peter’s frustration with aggregation probably reflects to an extent the fact that his tastes deviate quite a bit from the average consumer.

I can see both sides of the issue.  I’m pretty sure my tastes “happen to deviate in non-random ways from the rest of the population”, but it may be that most of us think that (at least in Western cultures) because we like to think we are “special”.  I also think crowdsourcing is overhyped, though useful in specific applications.  It’s an intrguing dialogue.

MORE:  Peter Klein has an opinion, too.