Failing Forward

From a now not-so-recent Slashdot article:

The company [Volvo] will introduce a concept car based on the S60 this month at the Detroit Auto Show, looking ahead a few years to the goal that by 2020 “no one should be killed or injured in a Volvo car.” The concept car will have forward-looking radar as a proximity sensor, and the ability to brake if a collision is imminent. When the car senses a collision, a light flashes on the windscreen display along with an audible warning. If the driver doesn’t act, the car will brake automatically.

Now, I consider myself pretty competent behind my wheel.  I consider most other drivers incompetent behind theirs.  But I would trust a human driver over a computer driver any day of the year.  Let me amend that: I would trust a human driver over a computer driver any day of the year when the road system contains any human drivers.  If the entire road system is being controlled by computer, then I think we might be on to something.  But so long as there are any humans driving any of the cars, having computers control any of them can only be trouble.

Humans don’t operate on rules. At least not from a computers’ perspective.  Human behavior has to be regarded as nearly completely random on the road, because the road system is not a closed system.  Any number of things, as far fetched as a plane crash, can happen while you’re driving, and your response is, in a certain respect, random.

I’ve slowly been finding the time to work through a new idea of mine, a data processing application where the focus lies on automation of processes to increase system throughput over large (not necessarily “clean”) data sets.  I started to wonder… exactly HOW much should I be leaving up to the user, and how much can I safely stuff into the black box labeled “super secret algorithm”?

The answer, to me, seemed clearly evident no sooner had I questioned myself.  The answer is quite simple: pass to the system all repetitive tasks.  After all, that’s what programs do, isn’t it?  By definition?  A program exists to take the same discrete set of actions time and time again, and branch here and there when the data changes. This is the very power the developer wields, at it’s most basic.  Where someone has to add the same numbers again and again, we can write a program for that.  Where someone has to verify names on a list against a second list, we can write a program for that.  These are basic examples, but you can easily extend the logic to more targeted and advanced tasks.

For example, where you have an authoritative list of spellings for some type of entity… say.. city names (“Chicago”, “Philadelpia”, “Portland”, etc) there are times when you’re bound to come across misspellings from your input sources. Such as “Philadelpia” in the above list. Since your algorithms may not be intelligent enough to figure out that that is a misspelling of “Philadelphia” the first time it sees it, then that’s something you have to pass off to a human. But here’s where you can pass the buck. By adding “Philadelpia” to a list of known misspellings, and attaching it to “Philadelphia”, even if your application may not be smart enough to pick up the mistake the first time around, you know you won’t be bugging your users about it again. This is a very basic example, but if you take the idea and apply it elsewhere, not only may you find yourself having fun writing some interesting algorithms, but you’ll be slimming down the work your users have to do.

The real trick in this is recognizing a repetitive task when it’s wearing a different costume.