Startling new proof that Ruby sucks at scripting!

Scripting languages should launch quickly, run at acceptable speeds, then exit quickly. Consider this script:

#!/usr/bin/env zsh


cat << SNOBOL4 > bogobench.sno
    OUTPUT = "Hello, world!"
END
SNOBOL4


cat << REXX > bogobench.rexx
say "Hello, world!"
REXX


cat << AWK > bogobench.awk
BEGIN {
    print "Hello, world!\n"
}
AWK


cat << RUBY > bogobench.rb
puts "Hello, world!"
RUBY


cat << LUA > bogobench.lua
print "Hello, world!"
LUA


cat << PROLOG > bogobench.pl
main :- write('Hello, world!'), nl.
PROLOG


cat << ERLANG > bogobench.escript
main(_) -> io:format("Hello, world!~n").
ERLANG


cat << JAVA > bogobench.java
public class bogobench
{
    public static void main(String[] args)
    {
        System.out.println("Hello, world!");
    }
}
JAVA
javac bogobench.java


/usr/bin/time -p ./bogoloop awk -f bogobench.awk
/usr/bin/time -p ./bogoloop rexx bogobench.rexx
/usr/bin/time -p ./bogoloop snobol4 -b bogobench.sno
/usr/bin/time -p ./bogoloop lua bogobench.lua
/usr/bin/time -p ./bogoloop swipl -q -t main -f bogobench.pl
/usr/bin/time -p ./bogoloop java bogobench
/usr/bin/time -p ./bogoloop escript bogobench.escript
/usr/bin/time -p ./bogoloop ruby bogobench.rb


rm bogobench.*

It’s simple enough: it generates the test scripts using here-documents. In the case of Java, placed as an example of a well-known start-up time hog, the test script is then compiled. The script is then run 100 times in a tight loop using this script:

#!/usr/bin/env zsh


max_iter=100


a=0
echo "Running '$*' $max_iter times…"
while [ $a -lt "$max_iter" ]
do
    a=$((a + 1))
    $* > /dev/null
done
echo "…done."

Very straightforward. And, of course, we’d expect a few things:

1. Execution time should be negligible. All that’s being done, after all, is outputting a string, something scripting languages should excel at.
2. SWI-Prolog and Erlang both should be pretty damned slow, given that they’re massive languages with massive run-time systems behind them (and in both cases the “scripting” is mostly an afterthought).
3. Java, with its notorious JVM load time, should be incredibly slow.

Results

The two numbers that are meaningful in the time output are the user and sys values. Take a look at the raw results:

./bogobench
Running 'awk -f bogobench.awk' 100 times…
…done.
real 0.66
user 0.03
sys 0.03
Running 'rexx bogobench.rexx' 100 times…
…done.
real 0.36
user 0.00
sys 0.05
Running 'snobol4 -b bogobench.sno' 100 times…
…done.
real 0.52
user 0.00
sys 0.05
Running 'lua bogobench.lua' 100 times…
…done.
real 0.28
user 0.01
sys 0.03
Running 'swipl -q -t main -f bogobench.pl' 100 times…
…done.
real 2.40
user 1.32
sys 0.70
Running 'java bogobench' 100 times…
…done.
real 12.03
user 8.75
sys 2.00
Running 'escript bogobench.escript' 100 times…
…done.
real 18.89
user 13.77
sys 2.46
Running 'ruby bogobench.rb' 100 times…
…done.
real 49.53
user 43.10
sys 3.74

Guh… what?!

Results

In EVERY SINGLE METRIC Ruby came in slower—often by two orders of magnitude!—than EVERY OTHER LANGUAGE. Let’s consolidate the results, adding together user and system time:

Consider what this means. Lua is a better scripting language than Ruby in that it is just the fastest of the bunch while being easily as powerful. Rexx, a language made in the ‘70s on ancient, creaking mainframes is a better scripting language than Ruby in that, while it may be a little bit less powerful, it sure as Hell is faster at loading. (And how much power do you actually need for scripting?) Awk, too, another language from the ‘70s, is more than powerful enough for most scripting needs, and well within acceptable load speed parameters.

More embarrassingly, Prolog, a language not intended for scripting, and, further, an implementation that’s not noted for being quick, is almost acceptable as a scripting language, coming in a good twenty times as fast at loading as Ruby. Erlang, too, even though it’s a painfully slow startup, is over three times as fast as Ruby at loading.

Even Java, a language not intended for scripting and infamously slow at loading is almost five times as fast as Ruby (and about 1.6 times as fast as Erlang).

Ruby, it turns out, is worse as a scripting language than three scripting languages from as far back as the ‘60s, not to mention three languages that aren’t even really intended for scripting.

Versions

Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
REXX-Regina_3.7 5.00 14 Oct 2012
CSNOBOL4 version 1.4.1 (January 19, 2012)
GNU Awk 3.1.7
java version "1.7.0_17" Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
rubinius 2.0.0.rc1 (1.8.7 release yyyy-mm-dd JI) [x86_64-unknown-linux-gnu]
Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:true]

Rage at Connecticut

There were two major events in the same year that had me questioning the sanity and the humanity of Americans during my time in China.

Rock my world

The first of these was the 2008 Sichuan Earthquake where great American luminaries like, say, Sharon Stone, opined that the victims of the earthquake somehow deserved to die or to have their lives destroyed because of the way the Chinese government treated Tibetans.

Were it only that overrated Botox® bimbo I’d have laughed it off, but all over my social media feeds of the time I saw people who seemed to be normal, decent human beings suddenly start spouting off exactly the same sentiments. The stunning hypocrisy, stupidity and inhumanity of these people had me exercising a lot of blockage. They fell straight into the “human sacks of shit” box and were buried from my social media stream forever.

The milk of human kindness

Even worse was the dairy scandal that broke only a couple of months later, however. Here, you see, I found similar seemingly-decent people suddenly spouting the homily that “the Chinese just don’t value life like we do”. Reading that from dozens of people from both the USA and Canada led to a severe adjustment of my opinions toward both nations as I watched everybody around me race to find safe supplies of milk powder (I got mine at tremendous expense from overseas, as did most others who could afford it) because they were afraid for the lives of the children whose lives they “don’t value”.

Keep that asinine, sanctimonious bullshit line in mind: “The Chinese just don’t value life like we do.”

No, seriously: rage at Connecticut

On December 14th of 2012 there was a horrific shooting at Sandy Hook Elementary School. It was one of many mass school shootings in the USA since the tainted milk scandal. The death toll for the tainted milk scandal was 6. The death toll for mass school shootings alone in the USA since then is over 50. (That’s mass school shootings, note.)

<sarcasm>I guess Americans just don’t value life like the Chinese do.</sarcasm>

Hurts, doesn’t it?

Do I seriously believe Americans value life less than the Chinese? No, not really. Of course not. I do believe, however, that Americans (and Canadians) have a very strong tendency to be smugly sanctimonious at the specks in other peoples’ eyes whilst ignoring the beams in their own.

Do you really want to criticize how “murderous” other cultures are? Take a look at your own violence statistics. (Let’s not forget the people you kill in your wars! Or the people your allies kill for you in their wars!) When you’ve looked into these, and taken in the shocking numbers (if you’re not shocked, you’re not human as far as I’m concerned!), then, maybe, you have the background to be sanctimonious.

I doubt you’ll be that smug afterwards, though.

Closing words by the Boomtown Rats

I Don’t Like Mondays

Thinking the unthinkable

I’ve been using Linux 99.44% exclusively for almost ten years now. In those ten years, however, I’ve developed an almost anaphylactic reaction to its total fuck-ups in the GUI world. Contrary to the almost annual promise “this year is the year of the Linux Desktop”, Linux has absolutely no future visible on the desktop for average end-users. The reason for this is obvious: contempt for the whole end-user experience is baked deeply into the genes of your average freetard developer.

As a result I’m thinking of flipping my machine.

Currently I run a Linux desktop (Linux Mint 11) for everything and fire up a VirtualBox machine for Windows XP for those rare times I need to use Windows specifically. I’m now thinking that it’s time to switch my main OS to Windows 7 and fire up a VirtualBox machine for those times that I need to use Linux.

WTF?! What led to this?

I tried to install a piece of software today. It’s a pile of fucking horse dung called KVpnc. Every step of usage is a serious WTF on a GNOME desktop. Take, for example, what happens when you launch it. Up comes KDE‘s “wallet” dialogue asking for permission to… do what, precisely? Who the fuck knows!? Freetard GUI developers seem to believe that “dialog box” is just an accidental name; that surely there’s no expectation of a dialogue with the end-user there! It just asks me for my kwallet password and tells me that KVpnc wants to use it. For some reason. That it won’t deign to tell me.

But that’s not the only piece of WTFery associated with this festering, manure-smeared outhouse masquerading as software. Because not only does the wallet dialog pop up, but so does another dialog box telling you that KVpnc wants to access the wallet, asking you if you want to allow it or if you want to kill the KVpnc application.

Two dialog boxes for one action. With no clue as to which should apply. Brilliant UX design there guys! (Before the reddit apologists and the HN wannabes chime in, I’m aware that the kwallet crap is part of the other festering pile of horse dung called KDE. This doesn’t matter. The point is that the UX is a horrible piece of stinking faecal matter. Where the specifics of the horror are implemented is of absolutely no interest to the user.)

Anyway, that alone isn’t worthy of me wanting to just give up on F/OSS desktops. No, what happens next is the reason. Because what happens next is that I try to import some settings given to me in an XML file. I sneaked a look at the XML file and it’s perfectly well formed. It has, to my quick eyeballing, a valid set of options. Too, it was made by exporting from a KVpnc instance that was the same version, so no versionitis is to blame. What did I get from trying to import that, though?

“error: Failed to read ”/home/michael/Desktop/KVpnc-connection.xml”

Yes, that’s all I got. “Failed to read.” No hint as to what the problem was. Permissions? Bad format? Doesn’t exist? Cosmic rays from outer space hit the CPU at just the wrong point? No fucking idea.

“Ah,” I said to myself. “This is just the traditional optimism of freetard developers. They can’t conceive of their code failing so they put in a catch (or whatever) at the topmost level and print a generic error message. Surely I just have to hit the web and find that error message and read what the problems could be.”

Boy was I a naive idiot.

Here’s a fun exercise for you. On Google search for “Failed to read” (with the quotes). I forgot that making stupid, meaningless, unhelpful error messages is an entire cottage industry in the F/oSS world. The old-school UNIX ethos of just saying “an error happened” has pervaded the culture to the point that even when software intended for end-users is made the whole “experts will undoubtedly know what’s wrong” meme takes over. Even searching for that and vpnc gets you nothing meaningful for help.

And that’s the problem.

In searching on this error message, I was reminded of the myriad of times I was faced with crappy software tools of all stripes, ranging from developer tools to end-user tools, whose sole approach to user interaction on error was to say “an error has transpired” and then nothing else. Then I compared this to my Windows XP days, short as they were, and started to reminisce.

Yes. You read that right. I started to reminisce over WINDOWS XP. (Shoot me now, please!)

And that’s how it stands now

I’m desperately trying to find a reason not to switch back to Windows. I’m trying to find a way to not use software written by poorly-socialized geeks who hold their users in contempt while still not feeding the Microsoft machine. (Don’t bother suggesting Apple anything. I will never purchase Apple kit. I find them too toxic an influence on the world of computing and software.) I’m trying, but really failing badly because the state of F/OSS GUI is so fucking horribly bad it hurts.

Maybe if I drink enough 白酒 tonight I’ll damage the brain cells that hold the memories of all this F/OSS UX shit and I’ll stave off the decision to toss away Linux for another month.

Why inbreeding is bad for your community

I decided to try out the Yesod Web Framework for Haskell today. I did this knowing full well that the Haskell/Hackage/Cabal ecosystem and infrastructure is almost gloriously broken. “Hey,” I thought. “How bad could they possibly make what is supposed to be one of the flagship packages showing off the strengths of their language?”

Very bad indeed, my droogs

The Cabal setup is fundamentally broken in Haskell. Packages in the Hackage database are extremely picky about their version requirements (which is fair enough, given that even bumping up a minor version number in a Haskell package frequently involves massively-incompatible API changes—apparently semantic versioning isn’t a thing in the community) and Cabal does absolutely no version management whatsoever. The only thing it can tell you is that if you install a package that will force a version upgrade that another package might get broken. (When you read these messages you might as well s/might/will most fucking definitely/g because I’ve never seen a case where an upgrade didn’t break another package insisting on a previous version.)

I made the mistake of trying to install Yesod after I’d installed a few other Haskell packages. Cabal flipped its lid and simply refused to continue past this point. Trying to force it, as expected, broke the whole ecosystem so badly I simply couldn’t get back to anything that worked at all.

Initiative comes to thems that wait

So, after a brief discussion with some nice, earnest people in #haskell (props to both ivanm and byorgey for your help!) I just decided to cut my losses and do everything from scratch. I mean it’s not as if I actually do any real programming in Haskell. (Mostly because of the Cabal fuck-up, mind, but still.) So off I go and delete $HOME/.cabal and $HOME/.ghc to start from a clean slate. I execute cabal update to get the package list and…

…wait…

…until it finally finishes. It suggests I upgrade my cabal-install package to the latest and greatest. cabal install cabal-install (this command line brought to you by the Department of Redundancy and Unnecessary Repetition Bureau) gets run and…

…wait…

…until it finally finishes. Then just for safety I do a cabal update again. Only to be told that there’s an update available to cabal-install.

This isn’t looking so good, is it?

I decide to ignore that error and continue with the advised installations. I cabal install cabal-dev and…

…wait…

…until it finally finishes. Then I cabal install yesod-platform and…

…wait… ...wait… ...wait… ...wait…

…until it finally finishes RECOMPILING THE ENTIRE FUCKING WORLD!

But, at least, it now worked. The installation succeeded and I can run the yesod command!

Oh bliss, bliss and heaven…

yesod init        # answer the questions
cd yesod
cabal-dev install

Oops. Fucking major oops here. Because, get this, cabal-dev RECOMPILES THE ENTIRE FUCKING WORLD ALL OVER AGAIN. EVERY library I’ve ALREADY installed via cabal gets recompiled again. “Initiative comes to thems that waits” indeed! I had a whole lot of fucking initiative going on here, let me tell you!

Still, surely this was all worth the wait, right? I could finally, after the successful build, finally get my type-safe web-app building funk on, right?

Suddenly, I viddied what I had to do, and what I had wanted to do…

Not a fucking chance in Hell.

yesod --dev devel

And…

…wait…

…until it finishes building shit. (How many times have I had to compile shit now just to get a web app started?!) Mercifully this was a far shorter build. It only had to recompile 8 files this time. Unfortunately it crashed hard on the last one. Apparently cabal-dev does something weird with ghc, wrapping it in something else, and that was doing stuff that made it throw up its hands and just give up.

…and that was to do myself in; to snuff it

And that’s just what I chose to do. I threw up my hands and just gave up. The Haskell ecosystem is a worthless pile of shit, as far as I’m concerned. As someone put it in #yfl:

Haskell people don’t know when they’re shooting themselves in the foot because they can’t remember what it’s like to have a foot without a bullet wound.

My off-and-on experiences with the community over the past six or seven years bear this out. The intellectually inbred world of Haskell is such that people within it honestly can’t even see just how bad their ecosystem is. They’ve just internalized all these fugly quirks and failures and adopted what is to them minor little workarounds to each of them without once thinking of what it feels like to someone coming in from outside.

And keep in mind that Yesod is one of their points of pride; one of the things they point to as a success story for Haskell. (The other one they point to—xmonad—has very similar problems as soon as you start trying to use any kind of contributed modules.)

This, guys, is really not how to convince people that your platform rocks. Even if your platform is spelled E-R-L-A-N-G or R-A-I-L-S or N-O-D-E or…

A (thinly-)veiled parable of failure

Every once in a while, in the religious world of software, you get a schism. They happen everywhere you can imagine, no matter how stupid they may seem. It even gets down past the wars like vi vs. emacs and into emacs vs. xemacs or vi vs. vim or the like.

There’s a holy war and a schism happening in a language community I peripherally inhabit. Just to keep this as non-personal as I can get I’m going to be using pseudonyms for everybody except me. I’m even going to be using pseudonyms for the software technologies involved.

Dramatis personae

An important afterthought based on feedback

The name Shen used in this story is not the Shen programming language. The selection of the name Shen was done in ignorance of the existence of the latter and I apologize if any were confused.

Software zealotry

There is something strange about software geeks that makes them form emotional attachment to their tools. Dispassionately viewed, the following facts are obvious to all… except the software geek:

• compilers are tools;
• programming languages are tools;
• editors are tools;
• debuggers are tools;
• libraries (even when they’re called “DSLs” or “frameworks”) are tools.

It really is that simple. Anybody who disagrees with the above is an idiot, pure and simple. (That means if you’re reading this and you disagree with the above, you’re an idiot, yes.)

Now imagine if, say, an automobile mechanic were to get so emotionally attached to his 9mm socket wrench that he won’t use anything else but that 9mm socket wrench. Is there a 10mm bolt that needs tightening? Well, he’ll grind down the head of it until it’s (approximately) 9mm because “everybody knows” that the 9mm socket wrench is the ONLY tool worth knowing, right?

We would, rightly, think any automobile mechanic who held these views was a lunatic, possibly a dangerous one. Yet most people who work in software act in exactly this way and, indeed, in worse ways. After all even if there were an automobile mechanic who fixated on his 9mm socket wrenches, there’s a very good probability that he wouldn’t go insane and start pounding the skulls of people who dared to use a 10mm socket wrench (not to mention those motherfuckers who sometimes use SCREWDRIVERS!). Software people do the equivalent, however all the fucking time.

The story so far

So, there is a schism brewing in the Shen community. Shen is old and crusty and, despite it being proven technology, many young bucks who use it chafe at its limitations and ugly corners and yearn to be free. And their cries did not go unheard for someone created for them the Panacea tool. Panacea is magical. It is “modern” and “productive” and is the cure to all of Shen‘s problems. Even better, it is built atop Shen so Shen‘s userbase can shift to Panacea without difficulty and without giving up all the wonderful things that made Shen the proven, strong technology it is. (You know, like how becoming a Mormon doesn’t mean you stop being Christian, right?)

Of course, as is usual when a new religion comes to town, there are zealous converts and the zealous convert of our story is named gagarin. Gagarin was once a pretty well-known figure in the Shen community, famous mostly for churning out a lot of work using Shen. (The fact that much of this product was half-assed, poorly thought-out, and/or incomplete is beside the point.) He, however, also constantly chafed at the restrictions Shen put on him; restrictions he believed suppressed his creativity and his productivity.

The elders of the Shen community, among them livingbuddha and immer, shushed him quietly and explained patiently why the things he lamented were good for him, not bad; why he should learn to love Shen for what it was and even to forgive its occasional quirks. This failed to satisfy gagarin, however. He looked outside the community for answers and found them in Panacea.

Cue that bit from the Rush song where the guy plays his guitar to the priests.

Yes we know it’s nothing new.
It’s just a waste of time.
...
Forget about your silly whim,
it doesn’t fit the plan.

Because much to gagarin’s shock and dismay, the Shen community reacted to his enthusiastic (verging on bombastic) proselytizing of his new faith with a collective yawn. People he’d thought of as his equals, (indeed at least one person he’d looked up to!) in the community were pooh-poohing his wondrous new tool.

I can’t believe you’re saying…these things just can’t be true!
Our world could use this beauty. Just think what we might do.
...
There’s something here that’s as strong as life.
I know that it will reach you.

Gagarin was flabbergasted! Maybe it was understandable that livingbuddha didn’t want to convert; he was practically a high priest of Shen, after all, but immer?! Why didn’t immer want to join him in the new faith?

Slàinte Mhath

Of course things did not go well from there. Gagarin, like most new converts in religion had no idea how to go about proselytizing intelligently. He spammed the Shen IRC channel with his Panacea code and discussions using the disingenuous excuse that because Panacea used Shen underneath it all, it was Shen and anybody who didn’t see that was just not looking closely enough. So… more Panacea spam was clearly the answer, right?

This, sadly (for gagarin, I mean) did not work. All he did was annoy his former coreligionists in Shen. Eventually people got sufficiently annoyed by his behaviour that they hounded him out of the Shen channel (while he ragequit the Shen community at large, it seems). Of course anybody who’s ever seen the reaction of a religious community to someone who converts out of it should not have been surprised in the slightest.

And the backbiting begins

Your humble narrator decided to peek into the Panacea community channel after this out of a sense of morbid curiosity. Sure enough the bitter gagarin was in there backbiting his former fellow travellers in the Way of Shen. Your humble narrator, for example, was merely “book smart”. (This is a peculiarly North American phenomenon where someone who bothers to educate himself is viewed with suspicion verging on derision. The fact that I’d been coding things people actually use since before gagarin was even born, naturally, fell by the wayside in his assessment.) My opinion having been safely dismissed as being based on knowledge (?!?!), he progressed to backbiting others in the community. Immer was very smart and able but too stubborn to see the Truth and the Light of Panacea. And, of course, livingbuddha was too much a fan of Shen (the implication being that he had an unnatural and perhaps even unhealthy attachment to it!) to ever be open-minded.

You know. Exactly the same kind of vicious and unhealthy backbiting you’ll hear from converts to, say, Islam talking about their unconverted family and friends.

So what the fuck is actually going on here?

I wish I had a decent conclusion to this story. I really do. But I don’t. Because I’ve seen this story play out so many times in so many ways with so many software technologies that I could have typed out this PRECISE sequence of events even had I not just experienced it (again!). It’s one of the reasons, in fact, that I decided to use pseudonyms for this story. There’s nothing particularly special about it. I’ve encountered dozens (maybe even hundreds!) of gagarins in the past and I expect to encounter dozens more before that brain tumour I’m hoping is there gets me to the point where I go on a massive shotgun rampage in a software company.

There is something fundamentally wrong in the head of software people; something that turns them into raging retards whenever they encounter an opinion that differs from their own; something that renders them utterly and completely incapable of just living and let live.

I don’t wish gagarin (in any incarnation!) any particular ill will (although I’m happy this particular one ragequit the Shen community channel given his proclivity toward being an ass). I’m happy he’s found something he likes and enjoys. I’m just not so happy that he feels the need to either convert everybody to his new religion or to pour derision on them when he thinks their backs are turned just because they disagree with something he likes.

If you’re reading this and you think you might be (one of) the gagarin(s) I’ve mentioned? You probably are. Stop being a cunt. If you’re reading this and you’re thinking of going the route the gagarins have taken? Stop being a cunt. Just be happy you’ve found a new tool.

And here’s the big secret, fellas (because the gagarins are always male!): having a new tool doesn’t mean you have to throw the old one away. It’s possible to have both a 9mm socket wrench in your toolbox and a 10mm one. And, if you’re brave enough, you can even have a hammer, a screwdriver or two and even a tape measure!

The ontology of the lambs

There is a lot of buzz being generated on Twitter right now as a result of some ignoramuses, led by the inimitable Zed Shaw, taking to launching multiple-tweet angry tirades (filled to the brim with as much profanity as 140 characters will permit) against advocates of strong static typing in programming languages. In addition to the profane personal attacks, there is a strong tendency to use straw man fallacies to the point that you’d be forgiven if you’d thought you’d wound up somewhere in Bizarro Earth where up is down, black is white and the Republican Party is the bedrock of tolerance and diversity.

What is astonishing to me about all this is the raw anger these people demonstrate. The raw hatred they spew for those who favour and use strongly and statically typed languages. Or, rather, it would astonish me if I didn’t already know the why. For the rage of Zed (and others like him) is the “rage of Caliban, seeing his own face in a glass”.

The ontology itself

Now keep in mind here that I do not mean the following three terms in their literal meaning. I’m invoking Forms of sorts, albeit fairly vaguely-defined and ephemeral ones. I’m using the three terms, in short, for a form of symbolic meaning, not literal truth.

The terms are:

• Technician
• Engineer
• Scientist

These three forms will be capitalized to distinguish them from their prosaic, better-defined real-world terms. I submit, however, in brief, that programmers fall into one of these three Forms (predominantly, I mean, there is overlap) and that accepting and working within these Forms will improve the world of software in measurable and important ways.

The Technician

In one of my earliest jobs in tech, I worked for a (very) small hardware manufacturing firm. How small was it? In the firm proper there were three people: the owner/boss, a technician and me as the software dogsbody. The technician was a nice, gentle Polish guy who was very good with his hands. His job was to populate our trial boards with components (assembly, in a word), to diagnose flaws and errors in the boards, and to, within limits, perform repairs on the prototypes to get them into functioning order for the next revision of the actual PCB.

At no point did this nice man, however, ever do anything creative. He was remarkably well-trained in his field, had a broad knowledge base, but he had no actual education. It was all training; a sophisticated intellectual toolbox and a sophisticated set of tools to use, but in the end still pre-formatted, pre-arranged, work using pre-formatted, pre-arranged components and tools.

In any engineering discipline, you’ll find technicians outnumbering engineers by anywhere from 2:1 to 10:1 and beyond. I submit that this is the same in software, but that we haven’t (yet) reached the stage of actually distinguishing between Technicians and Engineers in any formal fashion.

(And nor have I yet actually talked about what I mean by Engineers. I’ll correct that now.)

The Engineer

Again, going back to that job, my boss in this tiny firm was an engineer. He didn’t (often) assemble or repair things. He designed them. He saw a capability need of some form, figured out how to meet that need, designed the boards for it, selected the required components to go into them, etc. He created in short, building things that had not yet been built before. He may have used predefined concepts and components, and he may have rigidly applied some well-known conventions in the process, but in the end he created new things. His skills, knowledge and education were a step above our technician’s in that he solved problems using new, as yet uncreated technology.

Now before the straw goes flying fast and hard, please note that I’m not saying that this engineer was in some way a better person than the technician. I’m not even saying that he was more valuable to the company! The fact is that our technician was probably the most valuable to the company in the short term because his hard, boring drudge work was what allowed the boss to get on with his own work and gave him the time needed to create things. If my boss had to do all the assembly, debugging and repair work himself we’d never have made it to market.

Again, as in this electronics field, I submit that there is a distinction between the Technicians of coding and the Engineers of software development. I also submit that, depending on context, as with my former employer, both are actually necessary. The Engineer is not a better person than the Technician, nor is the Engineer necessarily more valuable to a company. It is, however, important to note that there is a difference and that this difference necessitates different handling of each’s working life, supervision, etc.

The Scientist

In the context of electronics, the technician, you recall, assembled the boards, and the engineer designed the boards. So what then is the role of the scientist? Well in this context the scientist is the person who invented the transistor (or whatever other ground-breaking piece of technology you choose to use—it’s not as if there’s a paucity of examples here!).

The Scientist’s main area is inquiring in nature. Where the Technician and the Engineer are concerned with making things that work, the Scientist is more concerned with pushing back the boundaries of human knowledge, even if it doesn’t necessarily have immediate practical benefits.

Again, as before, I submit that the realm of software has its Scientists. These are the people who invent the new techniques the Engineers use to design software systems and the Technicians use to implement them. They invent the algorithms, the “esoteric” languages, the paradigms and put them on a firm theoretical foundation to build up from. And again I submit that these people are not intrinsically better or more valuable. It’s just a different focus of interest and intent.

Examples

Examples of the Scientist Form in computing would be people like Knuth or Dijkstra or McCarthy for the big names. It would involve, too, the lesser-known names like Parr or Peyton-Jones or Bone or their ilk. All of these people have, to one degree or another, pushed back the boundaries of human knowledge and added tools, techniques and even simply thinking patterns to the field of software.

Examples of the Engineer Form in computing would be people like Ritchie or Moore. Ritchie, for example, together with Thompson and others of that team, created Unix. Unix was not groundbreaking in any of its features. Most of the “firsts” that are cited for it by its unthinking fans (and, thankfully, not by any of the principles involved) are not firsts. Unix was a product of taking the best known practices of the time and putting them together into a new and unique whole. This is the Engineer in a nutshell.

Moore, similarly, in inventing Forth, was not inventing anything new at a conceptual level. Stacks were known. Dictionaries were known. Modular programming was known. Even implicit operands or rapidly building abstractions upon earlier abstractions were all known techniques. (APL, for example, predated Forth by about four years.) All the techniques were old hat (relatively speaking), but Moore put them together and created something new with them.

By their very nature, Technicians are not going to be well-known, so it is hard to name examples. Instead I’ll point to the kinds of people who are the technicians in software. Think of the body shoppers in big consulting firms like Accenture/Andersen or EDS or their ilk. I’m not talking their “designers” or “architects” or that kind of person but rather their coding drones who are given a function/window/table/whatever specification who then type out the code that implements said function/window/table/whatever specification. This is the Technician of software development.

The problem

The problem is one of communication and identity. Explaining the communication problem is simple. For a Technician to be useful, they have to understand enough of the vocabulary (and thus thinking) of the Engineer to follow explanations and instructions; to discern intent from vague pronouncements and to build up their work from that. A Technician that can’t understand Engineers is useless and, likely, in the very near future, out of a job.

Similarly, for an Engineer to make new things, they have to understand what the Scientists are talking about sufficiently to turn the academics’ ideas into something practical and useful. An Engineer who can’t understand what a Scientist communicates cannot make new ideas into practical, useful or even beautiful things.

The problem, I think, is that people like Zed Shaw (and those who lionize his angry person/a) are at their core Technicians. They may be very good technicians. They may have a large tool belt with a lot of specialized tools. They may have a great deal of in-depth training. In the end, however, they’re Technicians. They assemble things based upon the specifications and instructions given by others. Just like the technician in the small hardware firm I worked for they may occasionally dabble in making things of their own (I did mention that the categories don’t have hard boundaries above, after all), but overall they remain Technicians: the assembly line workers of software.

The thing is that they don’t perceive themselves to be Technicians. They are emotionally invested in being Engineers and/or Scientists. And yet they cannot actually understand the output of software’s Scientists (this link alone should be ample evidence of this!) and this holds up a glass to their face that reflects something they don’t want to know: that they are not truly the Engineers they believe themselves to be. They rage at the looking glass because it doesn’t show them what they believe it should; they rage at it because it shows them what is, not what they desire to be.

The solution

For people like Shaw and company there is no solution. Reality faces one way, they face another. Their rage is real and exists because it strikes at the heart of their perceived identity.

But why is that their perceived identity and why are they so invested in it?

I think it’s because we, in software, don’t value our Technicians enough so nobody wants to be one. In that small hardware firm the technician was a highly valued member of the team; without his work there would have been no company at all. He was not looked down upon. He was not sneered at. He was not derided. In software, however, the Technician is given all sorts of epithets: code monkey, keyboard jockey, remote keyboard, technical stenographer. Their work is described as “a simple matter of typing” or even worse.

Even worse, I think in software by not recognizing these three Forms, we do egregious harm to the whole software development process. Because even on something as simple as supervision and assignment of work you should be treating your Technicians differently from your Engineers and your Scientists differently from both.

I’ve seen a company that almost got this right. They definitely recognized their Scientists at Entrust in that the cryptographers on staff, some of whom were heavy movers and shakers in the field, were treated with the deference and the care you should be treating those whose job is primarily thinking and expanding human knowledge. Sadly it all fell apart with the Engineers and Technicians. There were people who were obviously Technicians (the clue lay in their proclivity toward cut-and-paste programming and the utter disasters that were their unfactorable, unmaintainable, unalterable code) given roles of Engineers while obvious Engineers were treated like they were Technicians who needed close personal supervision of detailed, closely-specified drudge work.

It doesn’t help that educational institutions blur the lines too. You can get degrees in “Computer Science” that show no hint of academic rigour at all. (When you graduate from a “Computer Science” program without exposure to at least five entirely different ways of thinking about problems, including at least one purely abstract way, you’re not a Scientist!) Indeed some trade colleges give better theoretical grounding than some universities. There has to be better stratification of education so that those suited by temperament or preference to Technician-grade work go to trade colleges, those suited to thinking of new ways to piece things together to solve problems can go to engineering schools and those suited to pushing back the foundations of abstract thought enter computer science programs.

But just recognizing the differences (and acting accordingly) is a good start.

Evil Spock

I’ve always had a bit of a fetish for evil code. Note that evil code is not the same as obfuscated code. Evil code is perfectly readable… syntactically. It just fucks with expected semantics when put to actual use. This Haskell snippet is a very simplistic example of what I’m talking about:

-- The types are right: why won't this work?
double :: Integer -> Integer
double = (*) 3

Syntactically there is nothing wrong with this code (provided you know Haskell’s syntax). It’s the semantics of a function called double that triples instead that are evil. In this case the evil is very mild because it’s very obvious. You would (rightly) get smacked for this if you showed it to a Haskell fan and be told to grow up. (You’d also probably be compared to Zed Shaw for not understanding that nobody whose opinion matters has ever claimed that static typing solves all bugs.)

More subtle evil can be found in the Mercury language’s documentation:

:- mode var(in).
:- mode var(free>>free).
var(_::in)         :- fail.
var(_::free>>free) :- true.

This code shatters the purity guarantees of Mercury’s strong static type and mode systems. And just as with the Haskell example above the code is easily readable if you know the syntax of the language, but boy howdy would this suck trying to understand in actual code!

The point of the post

Which leads us to the point of my post. As I said, I quite enjoy reading and laughing at evil code. As a hobby, I mean. When in production code I go get the fire axe and start sharpening it. Lately I’ve been looking for (and creating) evil code in Prolog, getting to the point of actually making a project for an evil module for SWI-Prolog (and, probably with minor tweaks later on, for YAP as well). If you were sufficiently demented, you could actually put this line of code into a Prolog module and use the evil predicates I dream up!:

:- use_module(library(evil)).

Bored of readable conditionals? I can help!

The first pair of evil predicates I take from Perl and Ruby (and probably other languages): tail-placed conditionals. They are simplicity itself to define. Let’s look at the tail-placed if statement first:

if(Term, Condition) :-
    ( call(Condition) -> call(Term) ).

It’s succinct. It’s clear. It’s straight to the point. This is economy mode evil at its finest. With this predicate you too can write code as insane as this:

write('I like evil!'), nl if 1 = 1.

This is the same as saying in proper Prolog the following:

( 1 = 1 -> write('I like evil!'), nl ).

It just doesn’t have that unnecessary crud like “readability” or “sanity”.

Of course you can ratchet up the insanity by using my version of unless:

unless(Term, Condition) :-
    ( \+ call(Condition) -> call(Term) ).

This allows you to write abominations like this instead:

write('I REALLY like evil!'), nl unless 1 = 2.

Compare this to the boring, comprehensible old version:

( 1 \= 2 -> write('I REALLY like evil!'), nl ).

A note for my Perl and Ruby-using readers

Yes. I’m calling Perl and Ruby a major source of evil.

Wanna program with metasyntax?

Sometimes evil code is perfectly valid semantically, but the way it’s implemented is a major WTF waiting to happen. A perfect example of this comes from the SWI-Prolog documentation (and is thus presumably the work of Jan Wielemaker). The code is perfectly straightforward, although you’d better know your Prolog syntax very well:

... --> []|[_],... .

This little line of code is a spiffy little DCG rule that matches, well, anything. For some reason the author of it saw fit to use | for ;, but otherwise this is perfectly normal Prolog code. Well, except for the predicate name, I guess. Still, let’s look at it in use before we decide:

file_contains(File, Pattern) :-
    phrase_from_file((..., Pattern, ...), File).

OK, I’m beginning to see the problem. Can you? Here’s a hint: it’s pretty traditional to use the … sequence in comments where the … represents material left out so the reader can focus on the parts that matter. (It’s also used as syntax in some languages like Lua for variadic parameter lists, but that’s not quite as egregious as this use.)

So why is this a problem? Isn’t it cool that … is being used as an executable way to say the same thing?! Well, yeah. It’s really cool! And it’s evil! Because we’re pretty much conditioned to view that kind of stuff as comment-space fodder, not as code. This is going to cause more than one double- or triple-take. It’s going to add cognitive load every time it’s encountered as we figure out whether this is executable WTF code or comment. So why not just code it like this?

any --> [].
any --> [_], any.

Ignoring the fact that the formatting is simply more readable, the use of it is also more readable:

file_contains(File, Pattern) :-
    phrase_from_file((any, Pattern, any), File).

There’s no double-take here. It’s clear we’re matching a rule name any, then a pattern passed in as a variable, then that same any rule again. It takes no more typing, but it takes a lot less brain power when reading because we’re not struggling against years of conditioning.

No, really, the point

The point is that I enjoy reading and writing evil code as a hobby. When I saw that … predicate the first time (and figured out what it was supposed to do) I laughed out loud. It was a delightful find. And when I wrote the if and unless predicates I was chortling with glee at the fact I’d just turned a clean and consistent language into something Perl- or Ruby-like. (OK, well, not really. There’s no easy, readable way to match the sheer WTF of Ruby monkey-patching in Prolog, nor the whole magic come from thing that is the OOP version of polymorphism.) I’m still thinking of evil things to put into Prolog just for the joy of it.

But please, don’t use this kind of stuff in your real code. As an intellectual exercise, evil code is fun. Seeing that evil code in production isn’t fun. It’s a nightmare.

If you’re using a language that permits tail-placed conditionals, for the love of God don’t use them! Get over your “it’s too much typing!” fetish and use a proper, head-placed conditional, even if it’s the conditional operator (… ? ... : ...). And if your language supports a negated conditional (like a head-placed unless) don’t use it either. Negative logic is difficult to reason about and is the source of many subtle, frustrating bugs.

Play with evil code. Don’t use it for evil.

P.S.

Please send me evil code (not obfuscated code!) from any language. I love that stuff!

Zed Shaw won’t be able to read this code

Zed Shaw posted an epic tweet today that contained a link to this set of code snippets. From context it’s pretty clear that he views the first of the two definitions more readable because they’re “real words” instead of abstract symbols.

Of course some people disagreed. Of course Shaw went off on them with profanity and insults (there’s more than just that link… much more!) instead of rational discourse. This is Shaw we’re talking about after all.

Where I agree

Now at some level Shaw is right: in the main it is better to use real words for variable names because it makes the code read more clearly. In my own code I tend to use descriptive variables—when the description matters.

I don’t always do this, however because, often, when a function is sufficiently short and clear it is, in my opinion, more important to get more of it in front of my eyes than it is to explain the pieces. Additionally, sometimes what you’re writing is very abstract and there simply isn’t any set of names you can give that are going to be meaningful without being limiting at the same time.

Where I disagree

I’ll put up an example of some of my own code; code that Zed Shaw won’t be able to read:

:- func compare_impl(number_impl, number_impl) = dec.
compare_impl(number_impl(S1, Co1, E1, C1), number_impl(S2, Co2, E2, _)) =
    ( S1 = S2, Co1 = Co2, E1 = E2  -> zero(C1)
    ; S1 = negative, S2 = positive -> minus_one(C1)
    ; S1 = positive, S2 = negative -> plus_one(C1)
    ; S1 = negative, S2 = negative -> 
        ( E1 > E2   -> minus_one(C1)
        ; E2 > E1   -> plus_one(C1)
        ; Co1 > Co2 -> minus_one(C1)
        ;              plus_one(C1) )
    ;   ( E1 > E2   -> plus_one(C1)
        ; E2 > E1   -> minus_one(C1)
        ; Co1 > Co2 -> plus_one(C1)
        ;              minus_one(C1) ) ).

I’m sure that Shaw will have an aneurism if he ever sees this code. My God! Not a single fucking descriptive variable name in sight! There’s no way anybody could understand this, even if they know the syntax of the language! (The language is Mercury.)

And he’d be right. Even for people who know the language you can’t figure out everything that’s going on there. There’s two important declarations missing:

:- type number_impl
   ---> number_impl(sign, integer, int, context).
:- type sign
   ---> positive
      ; negative.

(Let’s handwave over the context thing for now because that’s way out of scope here and doesn’t impact the calculation in any way.)

There. Now anybody who actually knows Mercury syntax should have no difficulty reading the code.

Now if I were following Zed Shaw’s school of thought, I would write the code to look more like this (I’ll put the type declarations where they belong this time):

:- type number_implementation
   ---> number_implementation(sign, integer, int, context).
:- type sign
   ---> positive
      ; negative.
:- func compare_implementation(number_implementation, number_implementation) = decimal.
compare_implementation(number_implementation(Sign1, Coefficient1, Exponent1, Context1), 
                       number_implementation(Sign2, Coefficient2, Exponent2, _Context2)) =
    ( Sign1        = Sign2, 
      Coefficient1 = Coefficient2, 
      Exponent1    = Exponent2,          -> zero(Context1)
    ; Sign1 = negative, Sign2 = positive -> minus_one(Context1)
    ; Sign1 = positive, Sign2 = negative -> plus_one(Context1)
    ; Sign1 = negative, Sign2 = negative -> 
        ( Exponent1 > Exponent2       -> minus_one(Context1)
        ; Exponent2 > Exponent1       -> plus_one(Context1)
        ; Coefficient1 > Coefficient2 -> minus_one(Context1)
        ;                                plus_one(Context1) )
    ;   ( Exponent1 > Exponent2       -> plus_one(Context1)
        ; Exponent2 > Exponent1       -> minus_one(Context1)
        ; Coefficient1 > Coefficient2 -> plus_one(Context1)
        ;                                minus_one(Context1) ) ).

Now I’ve tried my best to format this in as readable a fashion as possible and yet, to my eyes, it’s far less readable than my original form. Consider the first conditional clause, for example. In the first case the equality of the three variables is in-line with the result zero(C1). In the second case the relationship is muted and obfuscated. The whole thing now is a wall of text (that goes too far to the right) with words that are “descriptive” (and thus more “readable”) only if you don’t know the data type being manipulated.

And frankly, if you’re working on code without a complete grasp of the data types involved, get the fuck off of my code base!

Cargo cult programming

Programming is this bizarre field that probably has more cargo cult behaviour than any other I’ve encountered. (This even includes education which jumps from fad to fad with monotonous regularity, just not with the alacrity of software.) In this case the cargo cult is the one that says “long, descriptive names are always better than short, terse names” (note the stress there, it’s important!).

I’m not arguing that all code should use terse, “cryptic” names. I’m just arguing that terse names are in many cases less obfuscatory than long ones.

I submit pretty much the entirety of the Java ecosphere as my evidence.

Closing words from Gert M.

“Who’s Zed?” “Zed’s wrong, baby. Zed’s wrong.”

Would you people just stop?!

I’ve said this before, and I’ll probably say it again (and again (and again (and again (and again (and again (...)))))), but Jesus, people stop believing the god-damned press!

So what’s triggered my ire this time?

The so-called “facekini”

This retarded “news” story has been sweeping the net and swamping social media. The story is retarded and anybody who propagates it is retarded as well. (Yes. If you passed on that story without sceptical commentary of any kind you are a fucking retard too!) Here’s an example of the typical reportage surrounding this:

(From the NPR blog)

The headline

“On Chinese Beaches, The Face-Kini Is In Fashion” the headline blares. The obvious inference you’re left to take is that this is a craze that’s sweeping China. Immediately under this “herp derp them furners is funny!” headline you get this picture to hammer it home:

This is no accident. NPR, like all journalists, knows that about half the readership will read only the headline and, if there’s a picture underneath it, will look at the picture and that’s it. The intent of these “journalists” is to make their readership feel good about not being the people reported on; about being, in this case, Americans who are “normal” and not them funny-lookin’ furrners.

But…

Let’s start actually reading

Cynical old me notes that this picture is tightly cropped to only show two people. My “press bullshit alert” alarm starts blaring and I start paying more attention. I then note this paragraph:

In China, it’s the height of the tourist season for Qingdao’s famed beaches. But while many of the town’s visitors want to enjoy the sand and water, they’re not so wild about sunbathing. So they often resort to a local tradition: the face-kini, a sort of light cloth version of a ski mask.

Oops. My cynicism is already showing grounds, isn’t it? It’s not “Chinese beaches” but only Qingdao’s beaches. Further this isn’t, as the headline implies, a craze sweeping the Chinese nation, it’s one fucking city.

Still, there must be more to this story. As you read on, the writer of the herp derp piece goes on to deftly misdirect you from the sole factual paragraph of the piece into hinting again that this is a nationwide trend. The fact that it’s a local tradition is given very short shrift while the “terror of tanning” is spread all across China and even as far as Korea, again implying that this trend is nation-wide and has even reached as far as Korea.

And yet, as far as I can tell from this article, even the fact that it’s fashionable in Qingdao is questionable. Let’s look at the only other picture this article provides:

WTF?! Again we have a picture tightly cropped to show only two people. And again my bullshit alert is blaring loudly. Someone is being an asshole and, for a change, it seems it’s not only me.

Huffington Post is fair and balanced?

Yeah, I know, that scares me too, but really the Huffington Post is being fairer and more balanced than NPR. Note the difference in the headline for starters: “Face-kini, Face Mask Bathing Suit, Is Popular On Chinese Beach”.

Now it’s a Chinese beach, singular, instead of Chinese beaches, plural. That alone makes me scared: the Huffington Post, people, is being less sensationalist and more intelligent than NPR. But let’s read the article a bit more.

…the latest swimwear innovation in the Chinese resort town of Qingdao…

Oops. They’re not reporting it as a “local tradition” either, but rather a new thing. On the whole the Huffington Post article is far less sensationalized than the NPR one (I know, I know!) and, as a result, is far more accurate. They’re not making global proclamations about how the Chinese hate dark skin (”...many Chinese women who prefer a fair-skinned complexion…”). They’re not trying to imply that this involves Korea in some ill-defined way. They’re straight up reporting some facts.

Photographic evidence

The clincher, however, is in the pictures. Let’s look at a few examples from the slide show beneath:

Are you seeing what I’m seeing? How the fuck would you know?! You can’t read my mind!

I’ll tell you what I’m seeing, though, so you can see it too. I’m seeing pictures where anywhere from one to three people are portrayed wearing these face-kinis (and attendant full body suits) while in the background dozens of people are being seen who are very clearly not wearing them. Far from being the nationwide craze that’s sweeping China’s (and even Korea’s!) beaches that the NPR hack job was trying to report on, it’s pretty clear that this is a) very localized and b) not even very popular in that locale.

Don’t trust the press

Really, it’s that simple. Don’t trust the mass press. For anything. Ever. I can even prove to your satisfaction that you shouldn’t do this.

Here’s why

Take a subject you know; that you’re somewhat of an expert in. You don’t even have to be grandmaster world champion levels of expert here. Just semi-competent at it is enough. (Good news, software people! Semi-competent includes you!) For example, I could use computing as my subject, or Chinese culture, or history or even scientific fields like physics if I stretch a little.

Now take ten random newspapers and search for articles related to your area(s) of expertise. Read them. Count the egregious errors of fact. Count the over-simplifications. Count the hand-waving. Count the instances of false controversy. (What do I mean by that last one? “ONE BILLION SCIENTISTS SAY X, BUT PROFESSOR Z SAYS Y SO WE’RE GOING TO GIVE HIM EQUAL TIME FOR PURPOSES OF ‘BALANCE’!”)

Given how badly the press reports on areas you know, what makes you think they’re any better in areas you don’t know?!

Wake up, please. And, if you’re one of the people spreading these “herp derp them furrners sure is funnay!” stories (of any country at any time) please stop and start using your fucking head for a change.

Esoteric languages

In the Fantom Programming Language forums there’s a thread on building a business on an esoteric language. The opening message is the kind of “thinking” (for want of a better term) that just drives me nuts.

“Hi all, this might not be like the normal discussions around here. I myself thought of Fantom as a beautiful language, yet wouldn’t building a business on top of an “obscure” language like this would be pretty hard and disadvantageous? Mostly when hiring programmers?”

I posted an answer in the thread, but I thought I’d post (largely) the same answer here:

• C
• C++
• Java
• C#
• PHP
• Python
• Ruby
• Erlang
• Lisp
• Prolog
• Forth

Each language I’ve named above has at one point or another (some of them right now) been accused of being an “esoteric” programming language. Each of these languages has had (wannabe) pundits claiming that basing a business off of them would be a Bad Idea™ for Good Reasons™ including, but not limited to:

• Difficult to hire programmers.
• Difficult to get support.
• What happens if the vendor/creator/whatever stops working on the language?
• No surrounding ecosystem to filch code from.

Yet each language I’ve named above has multi-million to multi-billion dollar businesses based upon them.

The conclusion is simple: never base your business decisions upon the yammering of wannabe pundits. Analyze your needs, your resources and the capabilities of your possible tools. Sometimes this does, in fact, mean you won’t be using an “esoteric” programming language, but more often than not, if you do a decent analysis, you might find your “esoteric” programming language is a competitive edge.

Like a Hurricane

This is going to be a phenomenally difficult piece to write because it involves digging up some unpleasant memories and facing some unpleasant truths which have haunted me for years. It is, in essence, a confession and a condemnation combined. It is also a chronicle of what is probably the single most cowardly action I’ve ever taken in my life.

I thought long and hard about whether to name names in this or not. If I name names the story has authenticity and the condemnatory aspect of it rings stronger. On the other hand it also distracts from a reality that most developers would rather not face, to wit: it is not only these people and these companies who do the kinds of things I’m about to relate. This is my most shameful moment of my career in software (and also, not coincidentally, the beginning of the end of it), but I am by no means alone in this.

So I’ve decided to leave in the names of the companies involved (where I can recall them) and leave out specific people. That should give enough verisimilitude to ensure that the story is believed without muddying the picture with accusations of defamation (although, truth be told, it would be kind of comical watching them try to sue me).

Waiting for the Hurricane

This happened when I worked for Pronexus, a small firm that at the time primarily made a Visual Basic toolkit for IVR applications. The company had just survived what was an existential crisis (which is to say that had three people not put in superhuman effort for a few months, the company would have failed to meet payroll). Money was finally coming in, easing some of the tensions, but the company was still in the psychological “must sell at any cost” stage of things.

I had been tasked with adding support for Rhetorex (a company which no longer exists but was purchased by Lucent and then later absorbed, erasing its very existence, as Alcatel bought it and turned it into Alcatel-Lucent) T1 interface cards for the PC. The cards were actually OEM cards manufactured by Mitel and therein lay the rub. Rhetorex wasn’t making a lot of money selling the T1 interface cards. They sold them solely to value-add to sales of Rhetorex’s massive voice processing resource cards (connected via MVIP to the T1 cards, among other trunk interfaces including Rhetorex’s own POTS cards). I suspect they were selling them at break-even or possibly loss-leader prices (to compete with then mega-rival Dialogic) and, thus, any expenses attached to supporting the boards ate into their profits in a field that was increasingly short on profit. (The market for IVR cards was approaching saturation at the time.)

Now the Mitel T1 cards were fine. They worked spiffy. Even the drivers Rhetorex had for them (it’s unclear to me who actually wrote them: Mitel or Rhetorex) were fine… provided you were doing what most people do with T1 cards: taking incoming calls. There was, however, a little… problem with outbound calls. Unpredictably—it could happen on your first call or it could happen on your hundredth, but it would eventually happen—the bus would lock so tight that your whole computer froze solid. (This was an ISA bus, so there was no protection from bad hardware/driver issues.) This rendered any application involving outbound dialing impossible until such a time as the bug was found and fixed. Rhetorex, however, was not interested in fixing this bug. I eventually had to advise my boss and the sales staff that the Rhetorex T1 cards could not be used for any kind of application that may initiate calls (which included certain classes of call centre, incidentally.

Fast forward a few weeks. Imagine my surprise when I got a phone call from a customer who complained that his system was frequently freezing up when, get this, making outbound T1 calls. Curious, I enquired further about the application and found out that it was a HURRICANE WARNING SYSTEM. It was entirely an outbound calling application. The customer had some custom hardware (sort of a really primitive precursor to a modern-day blade server) with an ISA backplane in which he wanted to stick 20 cards: 5 T1 cards and 15 voice resource cards. He had a database link to the phone company to link geographical areas to phone numbers. He had a real-time feed from the National Weather Service (I think it was) for hurricane watches. The application would monitor hurricane warnings and, on certain trigger conditions, would dial people in affected areas with warnings about and instructions for dealing with the potentially oncoming hurricane.

It was, quite literally, an application in which lives were at stake. And the sales staff sold these guys cards that they knew didn’t work. Worse, the boss knew as well and specifically instructed me not to tell these customers that the problem was with the driver. Everybody laboured under the false hope that Rhetorex would come through with driver fixes and we’d make the (rather impressive for us at the time) sale.

So I was specifically instructed by my boss to lie to the customer. I was told to blame the hardware first. When that failed (they switched to a regular, stock PC for development purposes) I was told the problem must be in their connection logic in the VBVoice application. When that failed I was told… well, you get the picture. I was instructed to lie to customers who were trying to save lives. For several months I stalled and misdirected these poor bastards even though I knew damned well what the bug was, whose fault it was and that it wasn’t going to get fixed anytime soon.

Rock You Like a Hurricane

I’d like to say that this story has a happy ending. I’d like to say that I finally stood up to my boss, told the customer the truth and took the consequences. I’d like to say the customer went with another company that had functioning hardware. I’d like to but I can’t.

I was at a very vulnerable (fiscally) part of my life at the time. I was deep in debt and the local economy was such that finding another job, especially after being dismissed for basically stabbing my employer in the back, was not viable. I caved. I went along with the boss’ instructions. I torpedoed a system designed to save lives in some of the worst natural disasters humans face just so I could keep my miserable, soul-sucking job. The only good thing to come out of the fiasco was that about six weeks after this started I finally snapped and called up Rhetorex, leaving on their voice mail system an epic rant that was considered so shocking it made its way to the ears of Rhetorex’s CEO.

To be fair, Rhetorex’s CEO, upon hearing my rant, decided that there was very obviously a major problem in his company’s product support. He called up my boss and talked to him about the voice mail (thinking that my boss had sent me to do the hit). He actually committed to fixing the bug (I’m not sure if they ever did because I left my job not long after this incident for a much better job with much higher salary) and apologized for the lackadaisical approach his company had taken before. Of course after that my boss hauled me onto the carpet and essentially put me on probation because I “could have irreparably damaged our relationship with Rhetorex” with my stunt. (It was another impetus to leave, you see.)

Edited to add…

One of my readers pointed out that what happened to the project wasn’t clear. I hinted at it above, but let me spell it out. When I said this…

I torpedoed a system designed to save lives in some of the worst natural disasters humans face just so I could keep my miserable, soul-sucking job.

…I meant that the project, in the end, was cancelled. A system for saving lives of people in natural disasters was called off because I was too much of a coward to tell the truth.

Hurricane

This incident was probably the nadir of my professional software life and the beginning of my eventual departure from the field. I was (and still am) ashamed of myself for how I dealt with that situation and I’ve found since then that every company I would ever be likely to work for would do the same or worse. There’s simply no sense of professional responsibility in software at large. (Hell, there’s no sense of professionalism of any kind in most software shops!)

Some people will condemn me for my choice. Go for it. I doubt you could say anything harsher than I’ve said to myself over the years. This incident weighs heavily on my conscience a decade and a half later. I’ve done lots of soul-searching in that time and mark this as probably the lowest thing I’ve ever done. So pour your outrage over me. It’s nothing compared to what I’ve done to myself. But I salute you for trying because it means you’ve got a soul and you’ve got morals.

Others, however, are worse. They will try to defend my choice. They will try to tell me that it’s perfectly OK to worry about petty pecuniary matters over people dying when I held the key to those people not dying. These people I hold in contempt that cannot be spelled out in words. And, sadly, these people are, in my experience, the majority of those who write software. They would, under similar circumstances, make the same choices (or even worse ones) without a second’s thought nor introspection. They are, in a word, narcissists (or maybe just plain old misanthropes).

Now it’s time for you to look into your own hearts. Ask yourself (and answer honestly): were you faced with the same choices I was, how would you have reacted? WOuld you have stood up to your boss, lost your job, torpedoed your career to save abstract lives thousands of kilometres away? Or would you have taken my craven route and crumpled under your boss’ assault?

Let’s hope you choose the former. Because the industry already has enough sociopaths.

User friendlessness

Anybody who’s known me for any length of time knows I’m an advocate of the end-user in software development. I think the contempt that programmers show for end-users is a disease. I’m hardly unique in this. A quick search of the usual suspects in tech blogs finds a lot of support for this position… with one odd oversight.

Where I seem to differ from my former colleagues is in thinking that programmers are end-users too. If you take a look at programmers’ tools, however, and compared them to equivalent tools outside of the programming craft, you’d be shocked (if you’re honest with yourself) at the disparity.

Programmers are treated as second-class citizens to even the “lusers” that they themselves deride so often.

The symptoms

Programming tools include everything (I mean everything) programmers use to make software. This includes programming languages, library suites, operating system services, text editors, debuggers and even the word processors programmers should be using to write documentation. (More on that below.) When you look at these, however, you’ll see that the tools are very badly constructed.

• They’re limited in scope and difficult to integrate with other tools (gdb, the autotools suite, Vim).
• They’re massive stovepipe systems that pretty much demand you commit to them and them alone if you want to use them at all (Emacs, Eclipse).
• They’re poorly documented to the point that the lore of experts is the only way to really get ahead (Ruby on Rails, 99.44% of F/OSS libraries, Erlang’s standard library).
• They’re inconsistently constructed so that knowledge of one portion can actively harm knowledge of another (Win32, Erlang’s collections, Common Lisp’s entire library).
• They’re filled with loads of mind-numbing ceremony necessary for even the most trivial work (Java, C#).
• They’re filled with redundant information that is a regular source of error as one part is updated without another part getting properly updated as well (Java EE or pretty much any other “enterprise” software package).
• They use a model of modularity that breaks so frequently a minor change typically involves a massive rebuild (C, C++ and any other language that uses textual inclusion instead of proper modularity).
• They have poorly-defined semantics around their abstractions with a lot of harsh, sharp corners to catch on (C, C++, Java, C#, JavaScript, PHP, Ruby, Python… let’s face it, pretty much every popular programming language and most of the unpopular ones).

It’s almost as if even the developers who do understand that they are writing for their users have forgotten that programmers are end-users too whose productivity is nuked when (not if!) their tools suck.

So let’s take a look at a few common kinds of failure.

Verbosity (you know, just like this rant!)

Go take a look at Hello, world implemented in a bunch of different languages. Look at the ceremony surrounding some of the popular programming languages in this simple task:

• C (7 lines);
• C++ (5 lines);
• Java (7 lines);
• COBOL (4 lines);
• PHP (3 lines).

Look at other languages as well, the ones that aren’t popular (or even known, for that matter!). There are quite a few doozies in there. Component Pascal, for example, or Modula-2 or Dylan.NET or elastiC or Fantom or Mercury… you get the idea.

Now the “Hello, world” program isn’t quite a fair test of a language because sometimes the ceremony is there for things that really only start to apply when you deal with non-trivial code. That being said, there is room to argue that you shouldn’t need to use the ceremony until you, well, need it. And seven lines to say “Hello, world?” That smells quite a bit. (It’s amazing to me when COBOL, of all languages, is less verbose than others!)

The real check is how much of that ceremony scales. If you only have to type the ceremony once and then have the rest of the code be mostly compact and concise, then this opening ceremony is tolerable. If, however, you have to wrap everything in equivalent ceremony with every piece you touch (I’m looking at you here, Java, and even more so at Java EE!) then the code rapidly balloons out of control and into the realm of the unusable.

Documentation

This is a pet peeve of mine, to be honest, so feel free to tune me out here and skip to the next. F/OSS libraries in particular are guilty of this, but most software is to some extent or another. Documentation, if it exists at all, is purely reference documentation. There’s no tutorial documentation (which is ironic given how much pointless ceremony is attached to most libraries!). There’s not even a birds-eye overview user guide.

Now my examples here are going to come from the Erlang/OTP distribution. This is not because I hate Erlang (quite the contrary!) but rather because it’s the documentation I’m most familiar with. Here are some documentation sub-failures to look at. (There are many more and I could write a whole blog series on documentation that sucks.)

Hard to find

Erlang’s I/O system is based on a notion of “group leaders”. It’s an important concept that you actually do have to understand if you want efficient I/O in Erlang. Pity that it’s basically undocumented. Do you think you’ll find it on erlang.org? Fat chance! Here’s where you find it (and that only when trapexit.org is actually up and running). Or you can infer it, of course, by Using the Source, Luke.

This is really bad documentation for such a vital feature. Why it languishes in an external document referenced from a third-party forum is beyond me.

Addenda and Corrigenda

An astute reader has pointed out that there exists hints of documentation for the group leaders. The erlang module does mention group leaders in passing in the form of two API calls. It doesn’t explain what any of it means but you get a hint of their existence here.

There is also documentation now on the I/O protocol which goes into more detail on group leaders. It doesn’t explain the rationale at all, but it at least shows you the nuts and bolts of its operation. This is an improvement, but still fails to be good documentation because knowing how to do something doesn’t help if you don’t know when or why.

Further, the erlang BIFs module uses the term “group leader” but there’s no link between the docs and the second document uses completely different terminology. If you don’t have access to a guru to point you to Robert Virding’s “Erlang Rationale” paper you would not be able to link the concepts without using the source. If even then.

Utterly incomprehensible

Those people who know me in the Erlang community know of my deep and abiding hatred of the sofs module. I’m sure it’s a good module in terms of it being well-written and useful. It’s just that its documentation is quite possibly the worst documentation I have ever seen anywhere. Having no documentation would, in my opinion, be superior to what’s there.

The documentation begins, for some bizarre reason I can’t fathom, with a long screed on set theory. It defines terms that no programmer shouldn’t already know: empty set, equality, intersection, subsets, disjoint sets, etc. etc. etc. It then leaps straight into a description of the data types and functions in alphabetical order: pure reference documentation, in other words. And even there the documentation remains bizarrely unhelpful. Consider this gem from the first exported function:

“Creates a function. a_function(F, T) is equivalent to from_term(F, T), if the result is a function. If no type is explicitly given, [{atom, atom}] is used as type of the function.”

What’s missing from the sofs documentation? Well, quite frankly, the actual documentation. There’s nothing showing the lifecycle of sofs usage. When using sofs, for example, what function do you use first? (Hint: set. Or from_term. Or from_external. Or from_sets. I’m guessing on all of this, though.) I suspect that sofs is badly underused and that is mostly because of its documentation.

Uncivilized APIs

A perfect example of uncivilized APIs again comes from the Erlang world. (Seriously, I’m not harshing on Erlang because I hate it! I love it and just use it for examples because I know it!)

Look at the collections in Erlang. The dict module and the orddict module, for example, are identical in their interface (with a potentially nasty semantic difference buried in the Description section). Now look at the gb_sets module. Despite many operations being similar (you insert items, delete items, etc.) the function names are not. If you choose to change your data representation from a dict, say, to a gb_set, you’re rewriting all your code. This is annoying and acts as a barrier to experimenting with different data representations.

A civilized API would, in the specific case of collections, ensure that similar operations (insert and delete, say) have the same name and the same order of operands, etc. This is not the case in Erlang’s collection set.

Of course uncivilized APIs spread far beyond Erlang. Let’s look at Microsoft’s Win32 API. CreateFile is a good example. Look at the massive ceremony surrounding just creating (or opening) a file! Seven—seven—parameters, two of which are “optional” (which means 99.44% of the time they’re going to be NULL and thus merely clutter the interface for no good reason). And that’s not all, of course. There’s also CreateFile2, CreateFileTransacted and LZOpenFile to brighten up your day. And, of course, you CreateFile whether you’re opening an existing file or, you know, actually creating a new one. (Why? Because CreateFile actually means CreateFileHandle. Which brings up the other piece of stupidity. You Create a File but you Close a Handle. Because after making one of the most comically verbose APIs available for a very basic operation, of course you’re going to make things “easier” by having a single function to close everything from files to windows to threads to… well, practically anything. Except sockets.)

Stovepipe systems

Emacs is a(n in)famous tool for programmers. Many top-notch developers swear by it. Many others swear at it. What nobody can deny, however, is that if you use Emacs, you go into it all the way. There’s no such thing as a casual Emacs user because if you’re using it only casually you’re not getting its purported benefits. Using Emacs is a lifestyle choice, not a programming tool.

Emacs also, quite infamously, doesn’t play well with others. While I’m not a huge fan, necessarily, of the “standard” GUI way of doing things, the fact that it is a standard is important. I can switch from Microsoft Word (pre- that stupid “ribbon” thing, I mean) to Abiword to LibreOffice Write to pretty much any end-user oriented software product without any difficulty; I’ll be reasonably productive in it in a day and will gradually ramp up to all the shortcuts and power user things I need as I get used to the changes. This even extends to the Ctrl+C/X/V thing: in any GUI program anywhere I can Ctrl+C to copy text and Ctrl+V to paste it.

Except Emacs.

Emacs has to do everything differently. This is, in fact, why I don’t use Emacs. I use enough other programs that Emacs sets up loads of destructive interference that damages not only my text editing productivity but also my productivity in unrelated tools like spreadsheets or word processors or even web browsers. I’m not willing to do everything in Emacs (and I know you can!) so I’m not willing, as a result, to do anything in Emacs. It’s just too alien.

Now the same can be said for my editor of choice: Vim. It is, in many ways, even more alien than Emacs. It is, however, significantly simpler (being a much more limited tool!) and thus easier to partition away into “I’m doing Vim so I do this; I’m not doing Vim so I do that”-style handling.

Eclipse is almost as bad. It uses standard GUI conventions, but it pretty much demands that you do anything development-related in it. You must use it for editing, debugging, project management and pretty much everything else. If you don’t, well, lots of luck figuring out where it tosses things. (Its configuration files are a nightmare and I’ve frequently found myself changing things that aren’t reflected in the IDE itself for reasons I couldn’t fathom.) It’s not as bad as Emacs, but it’s still sufficiently bad that I don’t use it. I’d honestly rather have a window for editing (vim), a window for a REPL (if applicable) and a window for executing my builds, tests, etc.

This is going on too long…

Let’s face it, the world of software development tools is in a really bad way. As much contempt as programmers show their end-users, they show even more for their own kind. (I’m pretty sure a good Ph.D. thesis is awaiting anybody who wants to study this self-loathing.) The real tragedy here is that everybody’s productivity would be enhanced if everybody worked only a little bit more on the usability aspects of their software development products. As things stand now, however, development tools are almost the epitome of user friendlessness.

Entertaining maths

There’s loads of talk about corporations and how they’re stupid and/or incompetent. There’s then wonderment expressed at how this can come to pass in a competitive environment that should filter out incompetence quite viciously. This amuses me to no end because I know that pretty much by definition any large, hierarchical entity—corporate, governmental or otherwise—must be incompetent. Any claim that this isn’t true is risible on the face of it.

The model

We will model here a fairly typical mid-sized corporation on its way to growing into a large-sized one. In such a corporation it’s not uncommon to have about 13 layers from the lowest worker to the CEO and/or board.

Flowing uphill

Information typically flows from observations of the workers to their supervisors. Supervisors collate that information and pass it up to foremen. Foremen summarize that information up for their lower managers. The lower managers compile it into reports for their middle managers. This process of communication goes up and up through twelve transitions before it reaches the CEO in some summarized form.

Flowing downhill

Now based upon this summary of observations communicated through the chain of the company’s hierarchy the CEO makes an executive decision. This decision is sent down to the vice presidents as policy. The vice presidents translate the corporate policy for their departments and ship it down ot the directors as, say, a road map. The directors take the road map and build an action plan from this. The action plan is further subdivided and elaborated upon with greater detail all the way down the chain until such a time as it turns into specific tasks for the workers at the bottom.

Stimulus/response

This whole system, in the end, is a stimulus/response chain passed through a hierarchical control system where each person is a node in said system that acts as both a sensor (uphill) and as an actuator (downhill). When a stimulus is sensed, the signal is transmitted up, a decision is reached and the actuators are activated on the way down to respond to it.

Only…

Signal loss rears its ugly head

Let us make a flattering caricature of corporate communication. This is the kind of caricature where the flabby jowls of a chubby guy are turned into the chiseled jawline of a romance novel cover model or where the frumpy and slightly plump middle-aged woman is drawn as a Hollywood hottie.

Let’s assume, in short, that the accuracy of communication through each layer is 95%. I think anybody who’s ever spoken with another human being one-to-one knows that this is a ludicrously high percentage of signal fidelity, but I’m going to pretend that I believe that corporate communications is just that efficient. This means that each transition layer that information flows up the hierarchy has a 95% chance of being transmitted correctly. Now with a 13-layer entity, that’s 12 transitions up the chain. 1 layer has a 0.95×100=95% chance of getting the information right. 2 layers has a 0.95×0.95×100≅90% chance of getting it right. (To keep things easier this is will be written like 0.95↑2×100 from now on.) 3 layers is 0.95↑3×100≅86% and so on until we go 12 layers at which point the odds are 0.96↑12×100≅54% chance of the information being transmitted correctly to the CEO.

Read that again. The odds of the CEO getting the wrong information from the hierarchy below him are nearly half. Almost half the time the CEO is getting bad information from his employees, and this is with the Hollywood hottie/romance novel cover model version of communication! (Remember, 95% accuracy is ludicrously high for human communication!)

But it gets worse. The CEO, based on this information (which, recall, is wrong almost half the time!), makes his decision and communicates it downward. This downward communication works at that same 95% accuracy. That means the round-trip stimulus/response has a 0.95↑24×100≅29% chance of providing the right response to a given stimulus.

The scary implications.

Again, look at those numbers. Almost three quarters of the time the company will do the wrong thing. And this is with the pert-breasted/chiseled-jaw version of corporate communication!

Now let’s have the breasts sag just a bit and the jowls get just a bit flabby. Let’s assume that corporate communication is 75% accurate (which is still ridiculously optimistic, note!). The odds of the CEO getting the right information from the hierarchy under him are now 0.75↑12×100≅3% and the odds of a proper response to a given stimulus now drop to 0.75↑24×100≅0.1%.

That, my dear friends, is why large companies always come across as clowns who do the dumbest thing possible whenever they’re given a chance to. They simply cannot be any other way.

“Solutions”

Now I can already hear some of the objections. “But Michael,” some are saying, “no company forces all information to flow from the bottom to the top and back down again. Managers are given leeway and discretionary power. This offsets the problem.”

While it is true that some managers in some companies are given some measure of independent authority (it’s a lot less common than you think, however!), this doesn’t solve the problem. It just means that the incompetence is focused on precisely the most important problems. Such a corporation will, on minor issues, make what looks like smart decisions frequently. This will lull them into a false sense of security while important decisions that do travel the whole stack fall through undetected. This could be even more devastating than having all decisions done stupidly because nobody would expect that degree of stupidity. (I personally suspect that a lot of sudden corporate deaths stem from this kind of false security.)

Other objections include the objection that not all miscommunication causes things to be diametrically wrong. It could be a small mistake. While this is true, the larger the drop in signal (70% is a pretty huge one, not to mention 99.9%!) the lower the chances are that the response to a given stimulus will be even close to what’s required.

“But Michael,” others will now comment, “any mistakes can be corrected as the corporation assesses and reacts.” Well, any such assessments and reactions also travel the communication stack. They’ll be just as incompetently executed. The company is just going to lurch from crisis to crisis and getting farther and farther away from anythign resembling balance.

Also, another factor enters here that I haven’t yet addressed: communication lag. There’s not only signal loss to contend with, there’s retransmission delay. Even if by some fluke the CEO gets the right information, by the time it reaches him it’s days to weeks to even months too late.

There’s one final objection that has a patina of truth to it: CEOs don’t rely just on their hierarchy to get information. They have consultants, the press, etc. as well. That sounds good in theory, but think about it for a minute: that’s more layers and more signal loss. This is going to exacerbate the problem (especially when the CEO starts to get conflicting information!), not mitigate it.

Nope, I’m sorry, but if you’re a large, hierarchical organization you’re just doomed to be stupid.

And evil. But that’s a rant for a different time.

A tale of two Prologs

It’s conventional wisdom in Prolog circles that you should, if you’re using free tools, do your development work in SWI-Prolog (because of its wonderful environment and tooling) and then grit your teeth and port your code to YAP for performance because YAP is just faster than SWI-Prolog. Now I have no reason to doubt this conventional wisdom, but being the kind of guy I am, I always have questions like “how much faster?” and “how much teeth-gritting?” These questions are actually astonishingly hard to answer.

I’ve looked under chairs / I’ve looked under tables

The problem, you see, is that finding a decent set of benchmarks for Prolog is fiendishly difficult (for me, I mean: I’m sure there’s people out there whose search-fu is so good they can find a million detailed Prolog benchmarking suites in only a few seconds). There are two main reasons for this:

1. Prolog and portability are still not really on speaking terms after all these years. You know, despite there being an ISO standard for it.
2. Much of the Prolog space is taken up with commercial offerings whose marketing literature boasts of performance; actual hard data could be embarrassing to these claims, so there’s not a lot of incentive to make a comprehensive suite of benchmarks.

Still, I did eventually find a benchmark suite for SWI-Prolog and decided to test two things with it:

• How hard is it to port code from SWI-Prolog to YAP?
• How much faster is YAP than SWI-Prolog?

I had my own guess for that second one: I figured that YAP was likely going to be about 50% faster than SWI-Prolog on average with perhaps a range of 10% faster to 70% faster. I thought, in short, it would be along the lines of various C compiler differences.

Porting

The benchmark suite is about 7000 lines of Prolog code doing a variety of micro-benchmarks. It turns out that to add YAP support, 11 lines of code have to be changed. (The code in question deals with the guts of the implementation—peformance statistics—so it’s quite reasonable to expect these not to be compatible.) To be specific, the following code:

ntimes(M, N, T, GC):-
    statistics(gctime, GC0),
    statistics(cputime, T0).
    ntimes(M, N),
    statistics(gctime, GC1),
    statistics(cputime, T1).
    ntimes_dummy(N),
    statistics(gctime, GC2),
    statistics(cputime, T2).
    T  is (T1-T0) - (T2-T1),
    GC is (GC1-GC0) - (GC2-GC1).

…had to be changed to:

ntimes(M, N, T, GC):-
    get_performance_stats(GC0, T0),
    ntimes(M, N),
    get_performance_stats(GC1, T1),
    ntimes_dummy(N),
    get_performance_stats(GC2, T2),
    T  is (T1-T0) - (T2-T1),
    GC is (GC1-GC0) - (GC2-GC1).

get_performance_stats(GC, T):-
    current_prolog_flag(dialect, Dialect),
    get_performance_stats(Dialect, GC, T).

get_performance_stats(swi, GC, T):-
    statistics(gctime, GC),
    statistics(cputime, T).

get_performance_stats(yap, GC, T):-
    statistics(garbage_collection, [_,_,TGC]),
    statistics(cputime, [TT,_]),
    GC is TGC / 1000,
    T  is TT / 1000.

That is a rather impressively small change (a change, incidentally, that makes testing other Prolog environments that much simpler). Add one simple bash script and I’m ready to go:

#! /usr/bin/env bash
scale=1
time yap -l run -g "run($scale), halt."
time swipl -s run -g "run($scale)." -t "halt."

Note that I wrapped the whole execution up in a call to the time utility as a redundant sanity check.

Results

I found the results rather surprising, to be honest. As I said, I expected the average improvement to be perhaps 50%. Look at the real numbers:

BU2B

There’s a lesson to be learnt from all this, I think. For starters the big take-away lesson for me is that assumptions can be bad. I would never have guessed that SWI was on average 2.4 times slower than YAP with a range of 1.9 to 5.5 (!) times the speed by just switching compilers. I also have to take away the lesson that the talk of YAP and SWI working together to ensure as much compatibility as possible is not just empty talk. This porting job was miniscule.

Now I know these are microbenchmarks and thus not really worth much, but this leads to take-away lesson #3: TEST before you make any assumptions. It will save you unpleasant surprises in the future.

So it seems that conventional wisdom was right. For this set of benchmarks at least.

How to ride a bus

I’ve ranted about how to drive a bus before. Today is different. I’m going to teach you how to ride a bus. Specifically I will teach you why you need to erase the preconceptions that you have—and that insidiously poison every thought you have if you’re not careful—if you wish to truly learn (or teach!).

Back-story

A friend of mine has lived in Japan for many years. We’ve spoken (online) at length of the differences in living in “the East” vs. life in “the West”. This is always a fruitful area for exploration because things are often done differently in “the East” and those differences can often be surprising to those brought up in “the West”. In conversations with him I found the perfect illustration of these differences and, at the same time, an important lesson to all who think themselves smart. I’ll let him take over here.

Benjamin’s story

Outside of Tokyo, the buses are not yet westernized. They have the entrance at the back, the exit at the front. There is a ticket you pick up when you enter (at the back). Just like when you visit a doctor, a number is on the ticket. The number increases each stop so the ticket shows which station you entered at. Next to the bus driver at the top there is a large display that shows all the tariffs for each number and they go up as the bus continues on its journey from stop to stop. So if you leave, you can see exactly what your fare is at the time, as can the driver when you get off.

The arrogance of western visitors

The first time most westerners encounter a system like the above the reaction is “how odd”. Then the mind closes and it’s viewed as “silly” or “stupid” or whatever. The assumption is that the western system is superior in some way that is “obvious” yet… well… indefinable. And the reason that it’s indefinable is because it’s not superior in any way, shape or form.

Consider the advantages:

• it’s cheating-resistant;
• you always know what you’re going to pay;
• the driver’s handling of payments is easier;
• if you change your mind mid-journey, you pay only for what you’ve used of the system up to that point.

Ottawa’s alternative

The system I had access to in Ottawa was inferior on all counts:

• you had to pay the same fare whether you were going one stop away or one hundred;
• if you had to change buses a complicated (yet easily-gamed) system of transfers was used;
• the driver had to know every possible transfer code—and these changed daily because of the gaming possibilities—that could work on his bus.

Wuhan’s alternative

The system here in Wuhan, apparently based on some French city’s system, is bad on other grounds:

• you pay per bus, period;
• the fare is fixed, like in Ottawa;
• there is no transfer system, however, so if you have to travel ten stops but change buses twice, you’re paying three times the fare of someone who’ll be travelling a hundred stops on one bus.

At least you can pay by waving a card at a plate, though. That’s way ahead of Ottawa.

Both systems, in short, are obviously inferior to this “pre-modern” Japanese system, yet the reaction of people encountering it is to deride it reflexively as wrong and somehow “silly”. This has extended to the point that the Japanese, to be “modern”, are actually switching to an inferior system just to be more “western”!

This applies to more than buses… or to Japan

There are some lessons to be taken away from this bus system. We carry an awful lot of baggage into any situation. In most cases (like bus systems) this isn’t a problem, but when it comes to some very important circumstances (social activism and engineering both spring to mind here) it is very important to be wary of that baggage.

The lesson is to follow three simple steps.

1. Yes. It’s different. Believe it or not, things can be done differently from how you think they should be.
2. Don’t dismiss things out of hand because they’re different. Stop. Think. Analyze.
3. After your analysis, be honest: consider the very real possibility that what you’re used to—what feels “natural” to you—might, in fact, be inferior.

The lesson for software

Yes, this is another one of my software rants, albeit in muted form. This kind of baggage is especially prevalent in software, of all “engineering” disciplines (remind me to explain at some point in the future why I laugh at the notion of “software engineering”), where there’s more cargo cult nonsense than actual thought than in any other industry I’ve worked in or alongside. (Evidence: the whole curly brace blight.)

When you hit a new programming language, for example, don’t suddenly decide that begin…end means that a language is useless because it involves typing six more characters (or four more keystrokes) than {...}. Even more importantly, when you hit a language that’s confusing at first glance, say one in a paradigm you’re unfamiliar with (which, if you’re like the Chamber’s Constant of developers means any paradigm other than imperative/OOP), don’t discard it because it doesn’t use loops and explicit branching. See if maybe the approach used isn’t perhaps superior in some way in at least some problem domains.

The lesson for social activism

Social activists in particular are arrogant in their assumption that what they “know” is superior to what others may know. I see them coming to China all the time to “teach the locals”—they “teach” Christianity, they “teach” democracy, they “teach” Objectivism even (!)—and they find out that their messages fall on deaf ears because what they think of as “obviously superior” is thought of as obviously inferior to the locals. These people come to China to “teach the locals how to do it right” and leave frustrated and angry at the Chinese because the imagined mass flocking to their idology they were expecting didn’t happen; because the “intransigent” locals just won’t do things the way the visitors think they should be done.

It never seems to dawn on them that the locals don’t want the new way because the old one is, at least in their eyes, superior.

To be a successful social activist you’re going to have to remove your ideological blinders at times and try to see things from the others’ perspective. And you’ll have to, on occasion, and likely far more frequently than you’re comfortable with, confess that perhaps the other side’s approach is superior.

The take-away lesson

We all have blinders. Every last one of us. These blinders lead our thoughts in well-travelled ruts that are often so deep we can’t even see over them. In many cases—like bus systems—this doesn’t matter much. A stupid bus system isn’t the end of the world. In some important cases, however, like engineering or social activism, these blinders can be a problem that leads to failure. We can’t avoid the existence of these blinders, but we can at least try to mitigate their impact. When you encounter something different:

1. Recognize that it’s different and move on.
2. Analyze the differences instead of reacting with knee-jerk negativity.
3. If the different way is better, admit it, even if it’s only to yourself.

Brewing tea for fun and consumption of copious quantities of time

The occasional visitor to or expat in China gets called something that translates to “more Chinese than the Chinese”. Context and tone are very important when you hear this phrase. It can be used in a derisory sense as it would be directed at, say, a westerner who walks around in Hanfu and tries, too hard, to act in the way he perceives the Chinese to act. It can, however, also be used in an admiring sense in which a non-Chinese demonstrates deep understanding of and appreciation for Chinese culture.

I have twice been called this in the admiring sense. The second time was in relation to making tea.

“Kung fu” tea

The term “kung fu” in China has little to do with martial arts (except that martial arts is one area in which one can show kung fu). Any skill that takes effort to master is, upon mastering it, showing “kung fu” (or, rather more accurately, “gongfu”). Making tea the right way is a skill that takes effort to master. There is, thus, a ”功夫茶” (literally “kung fu tea”) ceremony that is used to make, in my experience, the best tea even conceivable.

I thought I’d spend a bit of time today teaching how to make this tea. Of course, as with anything worth learning, this is going to be complicated, difficult and entirely worth your time to learn.

The tools of the trade

No, that photo is not of a set of torture utensils. That is a very complete tea set. For the full experience you will need a set about that complete. You can ditch the three figurines; you can lose the three-part cup at the right rear of the tray if you’re not making green tea; you can omit the bowl at the front left (although you’ll be happy if you have it).

In addition to the tea set, you will also need:

• a supply of good-quality water; it can’t be soft water and it can’t be too hard (spring water is ideal)
• a decent quality oolong or pu’er tea¹
• a near-infinite supply of patience

The pots

There are two pots in the picture: one with a lid, one without. The one with the lid is the brewing pot. The one without a lid is the serving pot. You will need both for the full ceremony. Both pots should be made of terra cotta (Yixing’s “purple clay” is traditional, but any terra cotta should do fine). In a perfect tea-drinking world you’d have a separate pair of brewing and serving pots for every tea you brew. Realistically this isn’t likely to happen. Keeping a separate pair for each major grouping is fine: one pair for oolong, say, one pair for pu’er, etc.

The cups

There are six cups in this picture (the ones with tea in them). They’re very small. This is intentional. The kung fu tea ceremony is predicated upon frequently-flowing small doses of tea.

The cups should be made of terra cotta to help preserve some of the heat, but the best cups are terra cotta on the exterior and white ceramic on the interior. This permits people to more easily assess the colouration and quality of the ensuing liquor.

The drip tray (and optional waste bowl)

There will be drippings. You need the slotted tray to catch them. Some are just a tray and need to be frequently emptied. Others have a discretely-placed hose to redirect drippings into a larger container. If possible you’ll want one of those.

The waste bowl is an optional component. It is used at the beginning as part of the process of cleaning if present and it is used during the ceremony proper to dump tea leaves if you make a lot of tea and have to recharge.

The filter

Even the finest teas and pots will leave some unsightly grit in the liquor. The metal thing in the coiled stand is a filter. You generally pour the tea from the brewing pot into the serving pot through the filter.

The aroma cylinders

The mysterious cup-like objects that are narrower and taller than the actual cups themselves are aroma cylinders. Use of them is optional (but highly recommended for the full experience). If you choose not to use them, simply skip the steps that mention them.

Assorted tools

The brush is used before starting to dust off the equipment. The towel is used during the ceremony to help keep the workspace tidy and free of unsightly puddles. Both of these are optional.

The wooden container with the implements of destruction sticking out are the tools of the trade. You’ll find a tea scoop (looking for all the world like an elongated baker’s scoop), a handle with a flattened end, another handle attached to a needle-like shape and a pair of oversized tweezers. The two middle components are for cleaning out the teapot (the flattened end is used to shovel out the tea leaves while the spike-shaped one is used to clean the spout). The tweezers are used to handle the components during the initial stages of the ceremony when all the equipment is going to be too hot to touch.

The ceremony itself

Lay out the tools in a way that facilitates access. Make sure you have a steady supply of hot water (for oolongs you want it to be ~90°C give or take a few, and for pu’er as close to boiling as you can get without actually boiling). You then follow these simple steps:

Phase 1: cleaning

Fill the brewing pot with hot water. Put on the lid and pour hot water over the outside of the pot. Transfer the water from the brewing pot to the serving pot, swishing it a little as it drains to help dislodge any grit, etc. Again spill a little bit of the hot water over the outside. Fill the aroma cylinders from the serving pot, spilling, as before, some of the hot water over the outside. (Arranging the cylinders in a ring and pouring in a circle is a good way to do this.) Using the tweezers, pick up each aroma cylinder and pour its contents into one of the cups. Again using the tweezers, pick up each cup in turn and dump the water, into the waste bowl if you have one.

As an option, in place of transferring the water from the serving pot to the cylinders, place all of the cylinders and all of the cups into the waste bowl and douse them with the hot water from the serving cup, supplemented, perhaps, with fresh hot water if necessary. After this you would recover them from the waste bowl (using the tweezers) and place them on the drip tray after ensuring they’re empty.

Phase II: washing the tea

Using the tea scoop, fill the brewing pot roughly 1/3 full (!) of tea leaves. Pour hot water into the pot about half-way and then place the lid on top. Douse the exterior of the pot with more hot water (I told you there’d be drippings!). After about 15–30 seconds follow the procedure as for cleaning the tea set above, steps 2–5, only this time you will definitely have to transfer from the serving pot to the cylinders to the cups and out. Don’t use the waste bowl for this.

When you transfer the wash to the serving pot you will have to use the filter to catch the inevitable grit. Get in this habit because you’ll be doing it for all future steps.

Do not assume the wash is optional. Failure to wash the tea will lead to a very unpleasant brew and to future brews that will be more unpleasant as the tannins soak into the clay of the pot!

Phase III: Brewing and serving

Open the brewing pot and fill it with fresh hot water. Fill it to overflowing. Any unsightly foam or floating tea leaves should be swept aside by the lid as you put it on. You then once again douse the exterior of the pot to keep it hot (and wash off any excess leaves or unsightly foam). Once again you transfer from brewing pot to serving pot (via the filter).

Phase IIIa: First-time serving

This step is only performed if you use the (highly advised!) aroma cylinders.

The first time you serve, pour from the serving pot into the aroma cylinders. Fill each aroma cylinder about 3/4 full of tea. Place a cup on each cylinder as a lid and then turn the whole assembly upside-down so the cup is on the bottom and the cylinder, with tea inside, is face-down. These should then be given a minute to build up steam inside. When the time is ripe, the imbibers should pick up the cylinders (the tea drains quickly into the cup) and quickly transfer them face up to their noses. The concentrated aroma of the tea is then available for assessment and enjoyment.

Phase IV: Enjoying the tea

Once the tea is in the cup, the drinkers should take the cups and start with a small sip, their noses practically in the cup for the aroma. Experiment with swishing around the liquor and breathing deeply to get the aromas all over the sinuses. Any decent tea will give a large complex of flavours which will change as more sips are taken and swished around before being swallowed.

Rebrewing

While the tea still has life (a decent oolong should last at least five brews!), repeat phases III and IV, bypassing the aroma cylinders. Generally drinkers will probably find either brew #2 or brew #3 (accounts will vary) the best, but all of the brews before the flavours grow flat will be a delight. Once the flavours do grow flat, if more tea is desired simply start over again from Phase II.

But what if I’m lazy?

The kung fu tea ceremony is simply the best way to make tea that has ever been created. Every step along the way enhances the experience of drinking the tea in some way. It is, however, as should be obvious, an extremely involved process that isn’t suited to a casual cup (nor to lazy people). So what do you do if you’re in a hurry or if you’re lazy? Well, do what I do. Buy yourself one of these and make your tea with it instead. It comes a fairly close second to the full kung fu tea ceremony in terms of tea quality, but is a whole lot more convenient.

Just remember that the key to good tea is in the wash (phase II above).

¹ You can use this ceremony with some modification with black teas or green teas, but I’m trying to keep this under novella length.

War of inner spaces

It’s rare that I have a dream whose contents survive the transition to consciousness. It’s even rarer for a dream of mine to not only survive mostly intact but remain compelling hours after waking. This is one such dream.

I think this idea, at its kernel, could make a really interesting science fiction story, role-playing game or even possibly a board/card game.

It all begins…

The story begins in the past. Decades of research into neurology, sociology and psychology leads to a new discipline: socionomy. (The neologism in question is coined by way of relating it to sociology in the same way that astronomy is related to astrology.) In socionomy we don’t just study societies, we know enough that we can shape them; we can program them, in fact, like we program computers.

A new world of joy, happiness, freedom and peace is heralded.

Fast forward a hundred years

The claims of socionomers are legitimate. Socionomy can allow you to program human societies with the traits you desire. The discipline is sufficiently difficult, too, that there are very few practising socionomers out there. Of course any software developer has already spotted the flaw. Sadly the sociologists, neurologist and psychologists who created the new discipline didn’t take enough CS courses to spot it.

Writing code, for mere computers, even, is hard and bugs creep in despite the best efforts of the coders. When you have a computer you’re programming, a bug is an annoyance that you curse under your breath over, then fix. When bugs are created in social “software”, however, you’re fucking with people. You only have one computer—society—and if it crashes, you’ve really got your work cut out for you.

Society has crashed at many times since socionomy programs were launched, at least portions of it have. Socionomers have been working frantically to patch the programs of their forebears, but…

Coding styles evolve

…there’s this problem. Socionomy being a new discipline, there were no known “coding standards” in place. Initial socionomy programs, although simple and limited in scope, were written in what essentially amounts to spaghetti code. Newer, more structured styles (the most popular being “Structured Memetics”) were quickly created, but by then it was too late: damaged, buggy and undecodable social programs were already out in the wild and causing damage. Increasingly new programs were built to just patch up the vital, but buggy, older programs.

Coding wars begin

On top of this, religious wars have started to divide the formerly close-knit circles of socionomy. Structured Memetics has reached its limits of complexity, but social problems requiring code have exploded combinatorically, helped onward by undetected bugs in earlier social programs. The already-small community of socionomers is fracturing into several new, untested approaches each with its own strengths, weaknesses and, most importantly, interactions—both good and bad—with other styles.

The entry point

Whatever form this idea takes (story, game, other), this would be where it begins. The world is a nightmare of badly-designed, badly-executed, badly-patched social programs written by increasingly desperate and frantic socionomers. In an RPG it would be up to the GM and players to decide how things progress. In a story, the next step would be the focus of a protagonist or two. In a card game the game play would focus on trying to stabilize an unstable system on the brink of total existence failure.

OK, that’s out of my system. I can return to sanity.

How to drive a bus

Many, many years ago, early in my tech career, before I could afford a car, I took the bus to work every day. This being Kanata (then a separate city from Ottawa) it meant a four-block walk to the bus stop followed, typically, by waiting at least 35 minutes for the bus that came every half hour. (Yes. 35 minutes or more for a twice-hourly bus. Did I mention this was the beginning station for the route? I didn’t? Consider it mentioned now.)

The bus driver

There was one bus driver who I seemed to be cursed to always get. The bus I needed to catch to get to work on time was his bus. This bus driver hated the public as far as I can tell. If you greeted him as you got on the bus you got ignored stonily on a good day; on a bad day you got a hateful glare that made you wonder if you were going to be next in a bizarre series of ritual murders.

This guy was a real piece of work. He glared at you if you got on the bus. He glared at you if you got off the bus. He turned beet red, looking like he was just this side of an aneurysm if you dared to signal your intent to get off at the next stop. Heaven help you if you delayed him for three seconds at a stop!

That’s what I did once, you see. I showed up late (for the first time since he’d started the route) and came running to the bus stop just as he’d thrown the bus in gear. He literally had to stop for three seconds to open the door, let me on, and carry on moving. This was too much for him.

“Next time come to the bus stop on time!” he scolded me.

My back went up. I turned to him and said, in a clear, loud voice, “Listen, asshole. You’ve got a real attitude problem given that without riders you don’t have a job.” I then proceeded to my seat.

To cut a long story short (too late!) he slammed on the brakes and refused to move the bus unless I got off. I refused to get off. The other passengers supported me, however, so his peer pressure thing didn’t work out. (We’d been comparing notes on this guy for several weeks, you see, and nobody liked him one iota.) The passengers filed off the bus, several of them giving me supportive pats on the shoulder while glaring defiantly at the driver. They just waited for the next bus while I waited for what came next.

His error

The rest of what happened (I got driven to work personally by the guy’s supervisor and he was never on that route again) is irrelevant to my point. My point is that this guy had the wrong attitude as a bus driver. He thought—incorrectly—that his job was to drive the bus. It wasn’t. His job was to drive people and the bus was the means to that goal.

By confusing what his job actually was he focused on the wrong things. Delays were interfering with his incorrectly perceived job instead of being, you know, the whole point of his job. I’m sure he thought he could do his job perfectly if it weren’t for all the customers…

And here’s where I get personal

This error is typical of people in the software industry. (That’s software industry. If you’re a researcher the status is obviously different.) Indeed I’d guess that the Chambers Constant (that’s 99.44%) of software developers share this guy’s attitude. “If only the users weren’t so stupid then I could do my job!”

Don’t believe me? Ask yourself this question: when was the last time you used a term like “luser” non-ironically when talking about the people who use your software? When was the last time you sat down and listened to a user complaint (say in a bug report) without rolling your eyes at the injustice of having to support such a dunderhead? If you’re like most of the people I’ve worked with or interacted with over the years—and that number’s well into the multiple hundreds scattered across three continents (North America, Europe and Asia)—you’re probably rolling your eyes now because you know what’s coming next and you want to get your defensive eye-rolling and dismissal going before you see the punch line.

What comes next

Because, you see, just like the job of a bus driver isn’t driving a bus, the job of a software developer isn’t developing software. The job of a software developer is solving user problems using software. Those “lusers” who irritate you so? They’re the entire reason you’re in the fucking industry. If you lose sight of that you’re no better than that asshole bus driver. Indeed, since you’re presumably educated and/or intelligent, you’re likely a worse asshole than that bus driver.

If you can’t cope with this fact do yourself, your co-workers, and your customers a favour and find another job. Software development just isn’t for you.

日本人 versus 中国人 cage match

There’s a famous pair of pictures found on a blog post at The Atlantic‘s web site. The first of the pictures shows refuelling operations at a Japanese airport:

The second shows refuelling operations on the same aircraft at a Chinese airport:

Both of these approaches are also reflective, in my opinion, of philosophical approaches to software development which I will call the 日本人 approach and the 中国人 approach respectively. (I will also refer to these approaches as “Order” and “Entropy” for reasons which will become clear at the end.) It is my considered opinion that both approaches, incidentally, are utter bollocks and create shit software.

Why the 日本人 approach sucks

Look again at the picture of the Japanese refuelling operation. Every little detail is accounted for. The two workers are wearing coordinated outfits right down to the bloody shoelaces for crying out loud! Surely they must have a well-thought-out system that is a marvel of efficiency, safety and effectiveness!

Well, no, actually. I mean I don’t know about aircraft refuelling, but I do know that in software a focus on fine-grained process that monitors every little detail of the creation of software is damaging to the outcome. Brooks noticed this too over 35 years ago. The statistic he cited was that 75% of software projects are deemed failures by the people making it. (I’d wager that the percentage is closer to the Chambers Constant—99.44%—if you were to ask the users of the software.)

There’s no end of methodologies for software development out there and every single one of them (even, perhaps especially, the ones that claim to free the developers!) lose the big picture in favour of a never-ending plethora of ever-irrelevant details.

I’m sure anybody who’s ever coded in an enterprise environment has seen the software equivalent of colour-coordinated shoelaces. (Mine was being told I wasn’t allowed to use Python scripts to generate my boilerplate code because not everybody in the company knew Python. Note that this was a code generator I wrote for my own use that was not part of the build process in any way, shape or form…) I’m also sure that such people will admit, grudgingly or openly, that the software quality wasn’t appreciably improved, but the quality of the workplace was noticeably degraded.

So it’s the 中国人 approach

Not so fast. Coding anything non-trivial is hard work and it’s complicated. Human beings are limited in their cognitive capabilities and, as a result, we suck at complexity. This is why we make teams for things. And once we have teams we have a need for constancy, regularity, communication and all the other things coders suck at. I’ve worked in shops that had, for all practical purposes, no controls and the results were easily as ludicrous as—sometimes worse than—the shops that had B&D-style processes.

The problem, in the end, is that cowboy coding is like that guy in the second photo. Sure you may metaphorically get the plane fuelled, but you’re getting a metaphorical mouthful of avgas in the process. If you’re lucky. If you’re unlucky you’ve got burning fuel spilled all over you instead.

Cowboy coders produce shit. It’s often inspired shit, but it’s shit nonetheless (in terms of reliability, expandability, utility, etc.). I submit the aforementioned Chambers Constant of projects at SourceForge and Github as my evidence of this.

So the solution is…

You think if I had the solution I’d be writing a blog about it? Fuck no! I’d be making millions fixing the software world and letting computers finally live up to their potential! I have no answers, but I do have an idea what the answer would look like.

I like the works of Michael Moorcock. (Yes, this was a jarring segue. Deliberately so. Suck it up.) While he’s weak at plotting and his characterization can be charitably termed weak, his strengths outweigh these weaknesses enough that I’m a fan.

One of the concepts I’m a fan of is his cosmology of the eternal struggle between Order/Law and Entropy/Chaos. In Moorcock’s writings too much order is as bad as too much disorder. Excessive Law results in tyranny and stagnation. Too much Chaos results in continual destruction. The solution in Moorcock’s world view is a metaphysical “Cosmic Balance” that keeps the two opposing forces in check.

We need some kind of software development balance to allow us to combat the creeping control freaks behind things like the Booch method or its ilk whilst at the same time not having our software fall into the more typical anarchy that is the norm in projects (despite the increasing clamps placed on the work by management). When a business leans too much toward process, it needs a boot back to the middle. When it leans too much toward cowboy coding, someone needs to rein in the cowboys and crack the whip a little.

And we need an instance of the Eternal Champion in every software shop to boot things around until balance is finally achieved.

Profanity amuses me

Really, it does. And I don’t mean in the sniggering internal 12 year old sort of way. (OK, I don’t mean just in the sniggering internal 12 year old way.) I mean the very concept of profanity amuses me at a profound level.

Picture, if you will, a party full of ordinary, middle-class people. Boring, isn’t it? And I think that dip is from a can. But I digress…

Got that picture in your head? Good. Now picture what happens if, in one of those sudden lulls in conversation that happens on occasion, your voice is heard loudly and clearly saying, “I heard that John had vigorous sexual intercourse with Mary last night.” What would the reaction be? There might be some nervous tittering. A bit of the old “Ew! TMI!” reaction perhaps. What you wouldn’t get much of, however, is raw shock.

Now let’s rewind to that lull, but this time instead you say, “I heard that John fucked Mary last night.” What’s the reaction this time as the tape moves forward? My prediction is that you’d get a rippling ring of shock spreading through the room with you at its epicentre. The shock would be utterly visceral and it would be focused purely on the specifics of your word choice, not on what you chose to communicate. Despite, you know, you having just said exactly the same thing!

It is this which amuses me. It amuses me when others react with such stark negativity to certain key words and it amuses me even more when I catch myself reacting that way. I have a pet crackpot theory to explain this, too: I believe that the notion of profanity—the notion that words have intrinsic properties, divorced from the meaning communicated, that are somehow directly harmful—is the last vestiges of magical thinking left to a culture that claims to be rational.

Sometimes it’s just fun to believe in magic.