Little Big Ways

I often miss the thrill of writing my own software.

It's not that I don't often have the opportunity to write software at all. Work provides amble opportunities to code applications at my own discretion. In fact, late this week I was prodding at an java application that parsed, and translated message formats. It's an interesting piece of coding, but I wouldn't necessarily call it "fun".

What I miss is being able to code a program something interesting and challenging. My last major coding project was Gazelle -- a website content manager written in PHP. It was a sizable project, but I can certainly see the limitations I imposed on it today. After being exposed to "enterprise" grade software code, I can see a number of poor design choices I had made. Prior to Gazelle, I worked on a 3-D rendering engine written in C++. That project was indeed challenging on a number of fronts. It had to be well optimized so as to draw each frame quickly. It had to manage a large database of objects and vertices. Lastly, it had to be cross-platform. 

Both of those projects, however, shared one major design oversight: The weren't data-driven. The concept of a data-driven application can be difficult to explain to a non-programmer. One author described it in terms of flow control. You can think of a program as a water flowing through a series of pipes. In order to change where the water is going -- what the program is doing in this example -- you need to open and close particular valves in sequence. The confluence of valves controls the flow of water. Thus, you see where the term gets it's name. In many programs, the flow control is written into the logic of the application. A certain sort of data goes in, and a certain sort of data comes out. In a data-driven application, however, the flow control is (largely) included in the file input. The input can then modify how the application behaves.

Now, there are little ways to make an application data-driven, and big ways. The little ways may mean to use a block of memory containing key-value pairs instead of a lengthy series of if-else statements. Or, it could mean storing your preferences in a file instead of in global variables. I'm familiar with a lot of these little ways and feel comfortable using them. It's the big ways that are daunting.

Using a hashtable instead of if-elses are one thing, it's designing a software engine that's hard. Both of the projects I mentioned, the content manager and the 3-D renderer, work best when designed as an engine. You can think of an engine as an ultimate form of data-driven application. The input data defines not only how the program looks, but how it works, and what it can do. Games are particularly effective as software engines. A properly written game engine can be used again and again in multiple products with only minor changes or improvements. Many games today are so versatile they can be modded to look and work like different games. 

In college, I held a brief fascination with game engines. I read several websites and a few books available in the school library.  Even while I studied them fervently, little if any seem to actually get through. Today I remember very, very little as to the actual coding of the engine. I certainly remember some of the theory, but little else. Perhaps I was too naive at that time to read other's code in detail.

Gazelle's biggest problem was it required extensive coding to add features and content types. Toward the end of the project, I had considered a massive redesign that would have solved some of these issues. The sheer complexity of it seemed too exhausting by that late juncture. A few short months later I tore down the system and put Drupal up on deninet instead. Drupal has an inherent flexibility that I admire in a piece of software. Today, it's possible to create new content types, data displays, and more without touching a line of code. Lately, I've taken to diving around in Drupal's core code (thanks to it being open source) so as to understand how it achieves that flexibility. Remarkably, Drupal's core is quite small and leaves most critical functions to additional modules.

The 3-D renderer suffered from similar, but different, issues. The main core of the application would draw whatever was pushed into its object database. Making those objects do more than fly around the screen in a consistent fashion was markedly difficult. I realized the system was capable of doing more than that, but making that happen seemed beyond my skills. I never quite figured out how to transform it from a renderer to something interactive. Just like Gazelle years later, I met this challenge after two or more years of effort. I was simply too exhausted to continue.

For years I thought these two projects failures. Sometimes I still see them that way, but tonight I can see the value in the experience. I daresay they demonstrate considerable skill in their own right, even if they fell short of their intended goals. 

My biggest problem today doesn't seem lack or forethought or naive design, but mostly a lack of interest. Whenever I think of a software project today, my first thought is to see if there's someone else on the Internet already working on something similar. Nine times out of ten, the answer is yes. Most of the time what I find is more than adequate to fill my needs. Less often, what's available needs serious improvement.

Perhaps I'm going about this in the wrong way. Instead of looking for large applications to write, I should focus on coding small programs for my own entertainment. Intead of coding a truly innovative game, I could write something simple but interesting. Intead of writing my own content manager, I could write modules for Drupal to solve a specific need (of which there are many).

Even though I'm not yet middle-aged, I've been writing software for the better part of a decade. For the first few years I was learning languages and skills in order to write "real" programs. Then, I focused on "making it" in the public eye, or in my career. While there are always things to learn, I already have "made it" in a sense. Perhaps now is the time to begin a new phase of my programming life.