Shreddin' The 'Natch

The title phrase comes from a Doonesbury cartoon in which Gary Trudeau's archetypal musician, Jimmy Thudpucker, returns to the studio after a long hiatus only to discover that his bandmates and the session musicians have all been replaced by midi-driven synthesizers. The producer tells him that employing real musicians is 'too many ways to shred the 'natch' which in mortalSpeak means, he would have to pay them union wages and that given the high costs of doing business, automation is cost-effective. Accepting that, Jimmy asks how the tracks will be 'sweetened' and the producer replies, "I got strings that'll give you diabetes."

Cost and quality have become the battleground in the wars between open source and proprietary systems. Insidiously, the one thing these competing systems shared in the technological ecosystem has been open standards. Now we see corporations beginning to walk away from the consortia who sponsor these efforts as the impact of policies regarding royalty-free contributions are measured against the price drops in software driven by open source systems, commoditization pressures from these very standards, and the intellectual property wars where the strategy to hold off predatory lawsuits using the tactics of pre-emptive patents is acceleration.

Ecosystems are about competing for energy. Tactics vary but the goals are always survival and growth. Species within these systems will apply tactics that will cause them to evolve towards symbiotic relationships or parasitic ones. In a technological ecosystem, a company or product can be one or both of these at the same time depending on the tactics applied in response to various conditions in an environment made up of themselves, their customers, and the ever present press all of whom can be contributors or predators.

Understanding the effect of one's tactics is essential to self-directed adaptation. The Hindus call this karma, but in this model, it is just the effect of feedback into a system that by virtue of its complex relationships strengthens or weakens the aggregate effects of acts selected by operations over those relationships. In other words, you aren't exactly what you eat, but the supply of what you eat is determined by how well you care for it. Basically, tend the garden or watch it go to weeds. Smart farmers know how to rotate crops, keep certain crops near or far away from each other relative to pollinization and predator control, how to select by markers for crops to cross-breed, which fertilizers cause rapid growth but burn the soil, and so on.

Farmers are the original ecologists, and not surprisingly, originally most were women because the men had manly pursuits such as whacking buffalo and deer for protein. Animal husbandry came later because mammals do the herd trick naturally and any male can follow a herd. So men eventually came to own most of the money, and like a herd, money follows fashion. Thus began the extinction of species. At the same time, the numbers of crops bred for food slimmed down to a very few. According to the August 2004 edition of Scientific American, no new cereals have been domesticated in more than 3000 years and today's population of mammals is fed by 24 plant species with corn, rice and wheat accounting for most of the calories consumed.

Meanwhile, further down in the system, the mammalian sex drives increased to replaced units damaged by war, famine, pestilence and bad beer. Sex is a means to repair damaged DNA and evolution, a friend of mine tells me, made it as much fun as possible to ensure we keep on having it. This increased the mammal supply, so increased the need to feed more mammals, and because no new cereals were being domesticated, we've cleverly learned to increase the yields of the few plants we use. In effect, the mammals bred the diversity out of the plants and increased the yield of the chosen few. From time to time, genetic breeding experts go into the wild and find a plant with markers indicating some desirable trait and introduce the gene responsible for that trait from the wild plant into what is known as an elite cultivar, essentially, the template plant for a new strain. The original population of plants from which all of today's corn crops are descended was likely less than 20 plants. Consider that next time you sit down to eat a bag of popcorn at a movie. There are more kernels in the bag and more people in the first two rows than all of corn's original ancestors. Grass fire anyone?

What does this have to do with music, standards, and source code? Glad you asked.

In the music world, open source is not the mp3 files people are busily swapping. That is the harvest of the crops grown by musicians. Midi is the open source. I am a musician. Musicians make and swap midi files with wild mammalian abandon. We often take old recordings and laboriously notate them in midi format so we can perform these with approximately the same sounds as the originals. Yes, like Jimmy Thudpucker, we'd rather use our fellow musicians but given that the pay for a gig has not increased in real or virtual dollars since 1972, we can't take the nine to fifteen people required to reproduce a 70s Motown hit to a gig that pays $300. It is simply too many ways to shred the 'natch. We'd like to make that up in CD sales but the real costs of producing a marketable CD even with all of the neat technology which includes midi has settled at the optima of about $10 a copy. Yes, a CD can cost less, but all megabytes are not equal so the geek math some use to criticize the industry doesn't reflect the reality of the costs of the harvest.

What midi really does is give us a chance to share open source music in the form of reproducible notes and orchestration. We pull these into cheap editors (thanks to the musical geeks) and study them to learn orchestral and composition techniques. We steal from each other, taking a drum part here, a lead lick there, and we compose new music. We've been doing this for centuries but midi made it cheap and fast. The quality of production actually can go up because we are as Paul Graham notes in his excellent article on Hackers and Painters, learning by example. It's a good thing. We found a way to make the bad effect of shreddin' the 'natch work for us. Now three guys can go play a decent sounding gig and still feed their kids. Where we resisted midi initially, we eventually embraced it and manage to live well in a shallower money pool.

Now we come to the issue for the geeks. If you want to keep open source open, you have to confront the dilemma of standards and code versus the reality that as you drive down the price of software, you force the system to get energy elsewhere. The powers of companies large and small found it in intellectual property cross-licensing. They also discovered that it could be used as a weapon against other competitors, so they are amassing it as fast as they can file it knowing full well that they are overwhelming the patent inspectors around the world and failing to meet the due diligence requirements. They use the tactics of the goombahs; as long as you can't resist them, the legal niceties don't matter because they have found a way to shred the 'natch in their favor.

The tactic coming into favor is pre-emptive patents. Pick a technology that you think is on the rise at some time in the future (to surf, you swim ahead of the wave), and get as many patents as you can. This works. Corporations today are defending themselves from IP suits with twenty and thirty year old users manuals dragged out of the garages of engineers who held on to them for keepsakes. Not being dumb, companies who see that trend are now building documentation repositories because they now know they need to keep all of the code they write, the documents they manage, and the contracts they sign just in case. Meanwhile, they are securing patents at a furious rate.

This won't stop because it is working.

Coming back to cereals, languages that we use on the web have been increasing steadily, but my bet is that at some tipping point, this will reach a steady state. It will winnow down to a handful of languages that we all use for the majority of our work. The bad part is that we haven't yet reached that state and some niches are being fought over furiously by old consortia and new faux consortia sponsored by BigCos who need to both justify their future products with a patina of cooperation while establishing their patent libraries as hegemonic over these faux standards. Meanwhile, champions of languages in the open source world are at war with each other over which languages shall be the elite cultivar of the future. This is bad because it means that very soon, two or more languages or systems that have to interoperate can't be bundled together because the licensing for each is incompatible with the other.

Yesterday, I responded to the request from a major figure in the development of open standards for 3D systems to introduce him to the editor of an up to now well thought of online magazine who's founder is a well-known champion of open source and open standards. The idea was to launch a series of technical articles about X3D, the only open source, open standard for 3D on the web. Here is the reply:

"Sorry, but from my perch, SVG is several orders of magnitude more used than X3D. *That's* what warrants series of articles, rather than inherent technical merit."

After swapping a few gratuitous insults, I realized this bird on his perch simply doesn't get it. SVG as an open standard and open source system isn't in as good a shape as he thinks in the market. X3D being a new language isn't either. What he also doesn't know is that patents and closed standards for 3D on the web are being pursued by invitation-only consortia with furious speed. In effect, the door for open systems based on 3D and 2D vector graphics is slamming shut while this guy is worried about the two formats competing. This isn't the first time I've heard this in the SVG camp, and it worries me. None of this stuff is new.

It is likely that one of the X3D vendors will implement 2D extensions to X3D for layers where 2D is a better and more efficient authoring and rendering technique. Think HUDs, data displays, 2D controls and the like. The work being done on shaders and a CAD Distillation Format will make this a powerful cross-bred set of capabilities. Some would like this layer to be SVG or at least a reasonable subset. That may well be the way this goes. Some in the SVG community want to
extend SVG into 3D and have done work in that direction. Graphics experts tell me that this is futile because the object model for SVG isn't designed to support real time 3D.

Why wouldn't these groups work together?

If the number of languages and elite cultivar are going to shrink, and if we as information farmers want to get more yield out of these virtual fields while protecting them from the predators on the near horizon, that would seem to be the best strategy. Yet in all my years of doing this kind of work, I am constantly amazed how the focus is on trying to get more yield from the field rather than improving the crops. This kind of bad planning dominates the thinking of some of the open source, free range designers and standards authors.

If we want to be able to feed ourselves in a time when the pay for the gig is falling, we need to learn from the musicians about creating formats that work together instead of fighting over who is paid to be in the studio. Our resources aren't that many and our products aren't that strong. Technical excellence still has a role to play on the World Wide Web, but for this to work, we should learn how to better shred the 'natch. X3D and SVG should be allies instead of competing alleles because extinction is just two releases away given increases in the parasitic qualities and a failure to breed symbiotes in the graphics ecosystems.

Why do it? Because Flash will give you diabetes.