Friday, August 27, 2004

Shreddin' The 'Natch

The title phrase comes from a Doonesbury cartoon in which Gary Trudeau's archetypal musician, Jimmy Thudpucker, returns to the studio after a long hiatus only to discover that his bandmates and the session musicians have all been replaced by midi-driven synthesizers. The producer tells him that employing real musicians is 'too many ways to shred the 'natch' which in mortalSpeak means, he would have to pay them union wages and that given the high costs of doing business, automation is cost-effective. Accepting that, Jimmy asks how the tracks will be 'sweetened' and the producer replies, "I got strings that'll give you diabetes."

Cost and quality have become the battleground in the wars between open source and proprietary systems. Insidiously, the one thing these competing systems shared in the technological ecosystem has been open standards. Now we see corporations beginning to walk away from the consortia who sponsor these efforts as the impact of policies regarding royalty-free contributions are measured against the price drops in software driven by open source systems, commoditization pressures from these very standards, and the intellectual property wars where the strategy to hold off predatory lawsuits using the tactics of pre-emptive patents is acceleration.

Ecosystems are about competing for energy. Tactics vary but the goals are always survival and growth. Species within these systems will apply tactics that will cause them to evolve towards symbiotic relationships or parasitic ones. In a technological ecosystem, a company or product can be one or both of these at the same time depending on the tactics applied in response to various conditions in an environment made up of themselves, their customers, and the ever present press all of whom can be contributors or predators.

Understanding the effect of one's tactics is essential to self-directed adaptation. The Hindus call this karma, but in this model, it is just the effect of feedback into a system that by virtue of its complex relationships strengthens or weakens the aggregate effects of acts selected by operations over those relationships. In other words, you aren't exactly what you eat, but the supply of what you eat is determined by how well you care for it. Basically, tend the garden or watch it go to weeds. Smart farmers know how to rotate crops, keep certain crops near or far away from each other relative to pollinization and predator control, how to select by markers for crops to cross-breed, which fertilizers cause rapid growth but burn the soil, and so on.

Farmers are the original ecologists, and not surprisingly, originally most were women because the men had manly pursuits such as whacking buffalo and deer for protein. Animal husbandry came later because mammals do the herd trick naturally and any male can follow a herd. So men eventually came to own most of the money, and like a herd, money follows fashion. Thus began the extinction of species. At the same time, the numbers of crops bred for food slimmed down to a very few. According to the August 2004 edition of Scientific American, no new cereals have been domesticated in more than 3000 years and today's population of mammals is fed by 24 plant species with corn, rice and wheat accounting for most of the calories consumed.

Meanwhile, further down in the system, the mammalian sex drives increased to replaced units damaged by war, famine, pestilence and bad beer. Sex is a means to repair damaged DNA and evolution, a friend of mine tells me, made it as much fun as possible to ensure we keep on having it. This increased the mammal supply, so increased the need to feed more mammals, and because no new cereals were being domesticated, we've cleverly learned to increase the yields of the few plants we use. In effect, the mammals bred the diversity out of the plants and increased the yield of the chosen few. From time to time, genetic breeding experts go into the wild and find a plant with markers indicating some desirable trait and introduce the gene responsible for that trait from the wild plant into what is known as an elite cultivar, essentially, the template plant for a new strain. The original population of plants from which all of today's corn crops are descended was likely less than 20 plants. Consider that next time you sit down to eat a bag of popcorn at a movie. There are more kernels in the bag and more people in the first two rows than all of corn's original ancestors. Grass fire anyone?

What does this have to do with music, standards, and source code? Glad you asked.

In the music world, open source is not the mp3 files people are busily swapping. That is the harvest of the crops grown by musicians. Midi is the open source. I am a musician. Musicians make and swap midi files with wild mammalian abandon. We often take old recordings and laboriously notate them in midi format so we can perform these with approximately the same sounds as the originals. Yes, like Jimmy Thudpucker, we'd rather use our fellow musicians but given that the pay for a gig has not increased in real or virtual dollars since 1972, we can't take the nine to fifteen people required to reproduce a 70s Motown hit to a gig that pays $300. It is simply too many ways to shred the 'natch. We'd like to make that up in CD sales but the real costs of producing a marketable CD even with all of the neat technology which includes midi has settled at the optima of about $10 a copy. Yes, a CD can cost less, but all megabytes are not equal so the geek math some use to criticize the industry doesn't reflect the reality of the costs of the harvest.

What midi really does is give us a chance to share open source music in the form of reproducible notes and orchestration. We pull these into cheap editors (thanks to the musical geeks) and study them to learn orchestral and composition techniques. We steal from each other, taking a drum part here, a lead lick there, and we compose new music. We've been doing this for centuries but midi made it cheap and fast. The quality of production actually can go up because we are as Paul Graham notes in his excellent article on Hackers and Painters, learning by example. It's a good thing. We found a way to make the bad effect of shreddin' the 'natch work for us. Now three guys can go play a decent sounding gig and still feed their kids. Where we resisted midi initially, we eventually embraced it and manage to live well in a shallower money pool.

Now we come to the issue for the geeks. If you want to keep open source open, you have to confront the dilemma of standards and code versus the reality that as you drive down the price of software, you force the system to get energy elsewhere. The powers of companies large and small found it in intellectual property cross-licensing. They also discovered that it could be used as a weapon against other competitors, so they are amassing it as fast as they can file it knowing full well that they are overwhelming the patent inspectors around the world and failing to meet the due diligence requirements. They use the tactics of the goombahs; as long as you can't resist them, the legal niceties don't matter because they have found a way to shred the 'natch in their favor.

The tactic coming into favor is pre-emptive patents. Pick a technology that you think is on the rise at some time in the future (to surf, you swim ahead of the wave), and get as many patents as you can. This works. Corporations today are defending themselves from IP suits with twenty and thirty year old users manuals dragged out of the garages of engineers who held on to them for keepsakes. Not being dumb, companies who see that trend are now building documentation repositories because they now know they need to keep all of the code they write, the documents they manage, and the contracts they sign just in case. Meanwhile, they are securing patents at a furious rate.

This won't stop because it is working.

Coming back to cereals, languages that we use on the web have been increasing steadily, but my bet is that at some tipping point, this will reach a steady state. It will winnow down to a handful of languages that we all use for the majority of our work. The bad part is that we haven't yet reached that state and some niches are being fought over furiously by old consortia and new faux consortia sponsored by BigCos who need to both justify their future products with a patina of cooperation while establishing their patent libraries as hegemonic over these faux standards. Meanwhile, champions of languages in the open source world are at war with each other over which languages shall be the elite cultivar of the future. This is bad because it means that very soon, two or more languages or systems that have to interoperate can't be bundled together because the licensing for each is incompatible with the other.

Yesterday, I responded to the request from a major figure in the development of open standards for 3D systems to introduce him to the editor of an up to now well thought of online magazine who's founder is a well-known champion of open source and open standards. The idea was to launch a series of technical articles about X3D, the only open source, open standard for 3D on the web. Here is the reply:

"Sorry, but from my perch, SVG is several orders of magnitude more used than X3D. *That's* what warrants series of articles, rather than inherent technical merit."

After swapping a few gratuitous insults, I realized this bird on his perch simply doesn't get it. SVG as an open standard and open source system isn't in as good a shape as he thinks in the market. X3D being a new language isn't either. What he also doesn't know is that patents and closed standards for 3D on the web are being pursued by invitation-only consortia with furious speed. In effect, the door for open systems based on 3D and 2D vector graphics is slamming shut while this guy is worried about the two formats competing. This isn't the first time I've heard this in the SVG camp, and it worries me. None of this stuff is new.

It is likely that one of the X3D vendors will implement 2D extensions to X3D for layers where 2D is a better and more efficient authoring and rendering technique. Think HUDs, data displays, 2D controls and the like. The work being done on shaders and a CAD Distillation Format will make this a powerful cross-bred set of capabilities. Some would like this layer to be SVG or at least a reasonable subset. That may well be the way this goes. Some in the SVG community want to
extend SVG into 3D and have done work in that direction. Graphics experts tell me that this is futile because the object model for SVG isn't designed to support real time 3D.

Why wouldn't these groups work together?

If the number of languages and elite cultivar are going to shrink, and if we as information farmers want to get more yield out of these virtual fields while protecting them from the predators on the near horizon, that would seem to be the best strategy. Yet in all my years of doing this kind of work, I am constantly amazed how the focus is on trying to get more yield from the field rather than improving the crops. This kind of bad planning dominates the thinking of some of the open source, free range designers and standards authors.

If we want to be able to feed ourselves in a time when the pay for the gig is falling, we need to learn from the musicians about creating formats that work together instead of fighting over who is paid to be in the studio. Our resources aren't that many and our products aren't that strong. Technical excellence still has a role to play on the World Wide Web, but for this to work, we should learn how to better shred the 'natch. X3D and SVG should be allies instead of competing alleles because extinction is just two releases away given increases in the parasitic qualities and a failure to breed symbiotes in the graphics ecosystems.

Why do it? Because Flash will give you diabetes.


Anonymous said...

Right Arm, Len.
I think anyone that studies it with any interest
can see how important SVG can be for X3D, and
what a dead end SVG will be without X3D.
Best Regards,

Anonymous said...

I've been trying to figure out what to say.

I wanted to talk about time, and how it plays a role (short term vs. long term profit). Genetic breeding is long term, but monoculture is short -- so perhaps the bias is to short-term profit.

It's also about ego, and sharing -- too much open source seems ego bound, while the benefit of open source - like the benefit of open ideas - is based on sharing, open debate and the exchange / mutation / integration of ideas. Ego can get in the way of that.


len said...

>I wanted to talk about time, and how it plays a role
>(short term vs. long term profit).
>Genetic breeding is long term, but monoculture is
> short -- so perhaps the bias is to
>short-term profit.

Of course but that is a baby that needs parents to get it to self-sufficiency. Ecology talks about isolates, or monocultures. They do fine as long as they
are in an energy rich environment. Among non-isolates, there is competition or energy. The environment is the Internet, the energy measure is URI tokens but the energy expression is clicks. That's why adSense works. At the interface of the Internet, it converts clicks into cash, the measure of energy among the mammals.

>It's also about ego, and sharing -- too much open
> source seems ego bound, while the benefit of open >source - like the benefit of open ideas - is based on > sharing, open debate and the exchange / mutation /
> integration of ideas. Ego can get in the way of that.

Yes, enlightened self interest. Self-interest alone is parasitic. Enlightened self-interest understands as the song says, that "Oz never did give nothing to the Tin Man.". or as Oz said, 'It's not how well you love but how well you are loved by others." In the token exchange pattern of the Internet, that is URIs that point to you, not the URIs you point to. In a sense, that is just a measure of popularity, but just as love among mammals is expressed many ways, one expression can express many things. Popularity is one, authority is another, respect is another, cowardice is another, but enlightened self-interest learns to choose, so if
you keep the elite cultivar for the strain you wish to create and are selective about traits acquired from the wild, you will learn to recognize which tokens
are valuable enough to point back to and which simply point your way.

Ego is the mind fooling itself that it is independent of the soul when all mind is is a mirror of the soul. Ego is not without use. It is about naming the the items in the mirror. Ego is not without value because these names are also tokens of recognition.

The reasons for X3D taking on an SVG layer are simple:

1. It is easy to use declarative systems. It is only
effective if the declaration when actioned is expected.
When we insert <p> anywhere in any XML application, we
should get the same effect.

2. X3D needs a 2D layer. If we use SVG:

a) The author has to learn fewer XML tagsets.
b) The programmer has to implement fewer tags in
the framework.

3. Easier content means more content. More content
means more URIs. More URIs means more energy that
can be converted across the Interent to mammals.

That is straightforward. If the programmers find they
have to fudge semantics, they should. If XML and object
oriented design work as they should, semantics should be in so far as possible, hidden in the framework. All
the author should need to know when they need to know
it is the namespace they are composing in.

It would be cool if the standard were compatible in the
legal sense, but standards can catch up. What is important is for the authors to tell the programmers that this is a cool thing, and the programmers to implement it so the authors can get on with building with it. Examples fed back into the pool always increase the common knowledge, that is, the number of URIs. This is a system that can within some limit we haven't defined and don't need to, create its own energy (the practical measure is the conversion rate from mammal energy to Net energy).

This doesn't mean all energy in the system is free. The Internet and XML don't care. It enables an infinite set of sets to expand into. The mammals can't. The lifeforce of the mammals is provided by the biological systems and these need more than URIs. The Internet conversion back to their domain is a contributor but not all they need by type. Geeks do not live by bytes alone. So resources are guarded. A current dispute is how long that protection is considered inviolate before a resource that is increasing in value by URI aggregation should become common. There are issues of type here. So we will have binaries and brands and so on. The politics of pooling resources are mammal politics. The Internet and XML don't care. They are means, not objectives.

There are technical systems emerging for that and it is
argued elsewhere in my blog.

But the obvious simple bits are easy to share and if we
want the best upward curve on URI aggregation, we'll come to understand how that can be amplified through sharing the symbol sets for creating resources. It's a force multiplier. Any namespace community that finds it is overlapping another namespace community can waste
the opportunity over who has the bigger numbers or
schmucks (look it up), or they can recognize that
these overlaps are a source of amplification and use
them to increase their penetration.

VRML still has surprisingly good numbers for use.
X3D is new. If the XML community wants to use it,
it's there. The X3D community is where new VRML
work is being done, so expect the VRMLers to move
to it. Today they have good enough so they may
not pack up and go in a single weekend but as
more of them learn XSLT and discover the database
features that emit and accept XML, they'll expand
their chops and their resources that way. It's
a natural painless evolution of tools and content.

If the SVG community and the X3D community are
wise or at least greedy, they will evolve into
symbiotes to increase the overall energy resources
available to both. If the SVG community doesn't
wish this, we'll steal their tags anyway.

Open source. :-)


Anonymous said...

Just use the mailing list you twit.

Zeke said...

Hi o)

I (webmaster) am writing to let you know I love the ideas you
present on your web-page and I also; love the structure and layout of the
whole web site ... I would be interesting in using a few of your ideas; with
your permission and would also like to be subscribed to your mailing list...

common blog

Anonymous said...


Thank you for the info you provide on your blog.. I would like to
learn more about this concept, I find it truly moral.. Could you show the full details as soon as you can, please?

Truly yours,

general contractors said...

How do I get information for general contractors... Some people believe that to create is great but build is like changing the past forever... If you want to create a beautiful home Visit general contractors and you can see what a little change can create..

private label rights said...

Now that is food for thought!

Strangely intriguing...

Keep it up :)

PS: I'm making a killing with private label rights ... find out how you can get extra income from private label rights

Comment Policy

If you don't sign it, I won't post it. To quote an ancient source: "All your private property is target for your enemy. And your enemy is me."