Tuesday, September 12, 2006

Why The Top of the Long Tail Fails

Tim Bray opines


linking to Bruce and saying what some way out of the top of the long tail curve have been saying for five years: terrorism is mainly a law enforcement problem. I'm glad to see the top of the tail finally gets it, particularly some that were rooting for the US to go stomping into Iraq, guns blazing, while their citizens sat safely at home blogging.

It really doesn't surprise me that the so called top bloggers tend to be the slowest at actually 'getting it' on almost any topic. That is what a inbound-link based system does: it popularizes an idead the same way a media machine does it and it creates the same fisheye lens effect that distorts even as it appears to cover more information. The long tail is a negative power law when it comes to early detection of important emergent effects. The curve is inverted.

Emergent effect comes from the edges of the network, but that means in the beginning, it is a very weak signal and if the effect of the fisheye is to marginalize and distort, it prevents it from being noticed until it moves to the center, to wit, into a top blogger's blog.

So it keeps us from doing the smart thing early because the center is by effect and definition, slow and often mediocre thinking. After so many years of studying information systems and their en masse effect, I repeat what I've said before: the values of one's values are all that is really important because judgement and justice are individual contributions into a system that by its very nature has neither of these as native qualities.

Take a position, take a chance, pick up the prize or the consolation of knowing you are giving it your personal best, and not just following the herd over the cliff.


John Cowan said...

Not really fair to Tim, I think. This is his 9/11 essay: it makes basically the same points, although not as clearly.


len said...

It's fair, John.

1. We did know who attacked us. We knew within 24 hours. Why? Data mining. Also, we had watched the run up to the attack. Smart analysts did not fault Hussein. They knew this was Al Qaeda. I knew as soon as I heard the first plane hit the WTC because a former military pilot explained to me how to do it years earlier as we flew into DC over the Pentagon. It wasn't unimaginable; it was untracked.

2. My first reaction was and has continued to be that this was a law enforcement problem because they have the policies and nascent systems for doing the ground work necessary. I said as much on lists. Folks like Dare Obasanjo went ballistic screaming for war.

Better coordination and systems were needed. The administration wanted to stomp Iraq. I chewed Bray royally for supporting that. Smart analysts knew it was a repeat not of VietNam but of Israel's stomping of Lebanon the first time. It would break the country and lead it to civil war.

America was not historically a first strike country. By misleading the public, the administration and their foolish supporters destroyed our advantage and our credibility while sacrificing thousands.

Tim claims to be a Middle East expert by dint of having lived there and by being an amateur historian. What he is not even if he has spent time at Ft Meade is a smart tactical or strategic analyst. He spins the conversation to fortify his emotional attachments and any smart intelligence analyst will tell you that is the worst posssible approach unless one is attempting to direct events contrary to the results of analysis.

I like Tim. We are old colleagues. That doesn't mean we don't disagree privately and publicly. But my point was that the long tail is inverted. As in so many cases of popularity driven elections, the more narrow the demographic, the less flexible the mind and the more easily funneled is the output.

So it comes down to one's personal values, not the values of ego, but of outbound links to every point of view that expresses a reasonable and clear analysis.

Comment Policy

If you don't sign it, I won't post it. To quote an ancient source: "All your private property is target for your enemy. And your enemy is me."