Here are the slides for my talk at Erlang Factory 2013:

Slides for “Writing Beautiful Code”

Transcript

Joe: Declarative programming languages have several advantages over traditional languages. For example, programs in such languages are considerably shorter than equivalent programs in imperative languages. Here for example, is a program in C… and here is the equivalent program in Erlang.

[phone rings]

Joe: Hello.

Mike: Hello Joe.

Joe: Hello Mike.

Mike: Question… does anyone care about the differences between declarative and imperative languages anymore?

Joe: Don’t be silly. The declarative/imperative debate is always a crowd pleaser!

Mike: From what I can tell, kids these days are going with what’s cool, like Node.js and Ruby on Rails. Let’s ask Robert.

[phone rings]

Mike: Hello, Robert?

Robert: Hello Mike.

Mike: Joe’s back to talking about declarative languages again. I just think Erlang’s got a problem when it’s spokesman is an old fuddy-duddy. Eh Joe?

Joe: Bite me.

Mike: Robert, what do you think?

Robert: Erlang’s the shizzah! But I can’t blame kids for picking something a little less… crusty.

Mike: Erlang’s image is stuck in the 1980s. It’s the Bananarama of languages.

Joe: Okay. Tell you what. This is bullshit. If kids these days don’t want to use Erlang, which is the real “rock star tech” then fuck it. I’ll turn this video off and go home. Take my nine nines with me!

[video turns off]

Mike: Joe, I didn’t mean anything by it.

Robert: Yeah Joe, Mike was just trying to be constructive.

Mike: Joe, turn the video back on.

Joe: I won’t. I won’t have Erlang disparaged!

[stranger’s voice]

Rex: Yo yo yo. I can totally help you geezers.

Joe: Who said that?

Rex: All Erlang needs is an image upgrade.

[video turns on]

Joe: That’s very interesting.

Rex: Lots of things from the eighties are cool. Just look at Fraggle Rock and Super Mario Brothers.

Mike: What the hell is that?

Robert: It’s one of those Ruby hipsters!

Rex: Watch it grandpa. Just because I carry my Mac in a messager bag and have great taste in nose rings doesn’t mean I use Ruby!

Joe: I for one and interested in what the little girl has to say.

Rex: Yo, whatever. First we have to fix the name. A language named after a nineteenth century Danish mathematician is about as cool as having a Facebook account.

Robert: It’s just like the olden days, when Erlang was banned. We changed the name.

Mike: That’s right. When Erlang fell out of fashion at Ericsson, we had to change the name to OTP.

Joe: Three of the trendiest names we could think of: “open”, “telecom”, “platform”.

Robert: Gives me the chills just thinking about it.

Rex: Okay, that’s bullcrap. No one care about telecom platforms. We need something fresh. Something edgy.

Robert: Like “Erlang on Rails”!

Rex: Jesus Christ. The last thing you want to do is imitate. No. Something original.

Joe: How about “Erlang++”?

Rex: I got it! We keep OTP, but with a hip, edgy twist. Check it. Your language will hereby be known as “Outlaw Techno Psychobitch”!

[Joe scowls]

Robert: It does spell “OTP”.

Mike: Don’t know what it has to do with software. What do you think Joe?

[Joe pause]

Joe: I absolutely love it! That name is fucking gangster!

Rex: Yeah Joe. You rule! Okay, next thing we need is a video that is just as cool as the original Erlang The Movie, but speaks to today’s programmer hipster. I’m thinking something like this…

[video starts]

“Outlaw Techno Psychobitch” - “OTP” - “Programming for Bad Ass Mother Fers!”

Rex: Yo, what up bitches. Rex here. This time I’m talking about a bad ass new language called “Outlaw Techno Psychobitch”, or “OTP” for short.

So, you all know I’m a huge Ruby fan. Ruby gives you 15 ways to do the same thing and freedom of choice is good. Then I realized, that’s just 15 different ways to create really slow, crappy programs.

Then I started using Node.js. I like Node because, as my hero Ryan Dahl says, “it’s like coloring with crayons and playing with Duplo blocks.” But as it turns out, it’s less like playing with Duplo blocks and more like playing with slinkies. Slinkies that get tangled together and impossible to separate.

So then I checked out Outlaw Techno Psychobitch. It’s a language out of Sweden that can be used to build web scale, asynchronous, non-blocking, sharded, event driven, message passing, NoSQL, reliable, highly available, high performance, real time, clusterable, bad ass, rock star, get the girls, get the boys, impress your mom, impress your cat applications!

Ryan Dahl, my hero, says “we should not be supporting more programming languages,” that we should be “killing them”. That “all these bullshit projects are confusing people.” But Psychobitch is just so damn cool, I couldn’t resist giving it a try.

The first thing Psychobitch kills off is curly braces. Thank God. After a half century of curly braces, we need something that’s fresh. Psychobitch uses single character block delimiters, like in sentences.

My hero Ryan Dahl says, “math is not necessary,” that “people who think it’s superior to curly braces are wrong.” But I figure, if math can get a rover to Mars, maybe it can help us write better programs.

Psychobitch gets pretty uptight about logic. If you say X is one, then turn around and say X is two, Psychobitch goes ape shit. She’s like, “what the hell, when did one start equalling two?”

Psychobitch has this great feature called “pattern matching”. You define the patterns and if there’s a match your program keeps going. If something doesn’t match it crashes. Like assertions but built in.

What happens when your program crashes? Psychobitch has you covered. It monitors your code and restart things that break. It’s like the Terminator. You can blast it in the face with a shot gun — it stops for a second, then keeps on coming!

I’ve written a chat server in Psychobitch. It’s where all my friends and me can chat about how awesome Ryan Dahl is. Because the server is written in Psychobitch, it can easily handle thousands of concurrent users. Shit, did I say thousands? Because I meant tens of thousands. And by tens of thousands, I mean hundreds of thousands.

[switch to demo]

I’m already logged in as myself at the top. I’ll log in as another user. You say something using the “tell” command. The messages are received instantly even though there are hundreds of thousands of connected users. I’ll use the “yell” command to broadcast a message to every user.

[uses yell command]

That literally sent the message to 300,000 users. About a second.

Okay, 300,000 concurrent users is impressive. But even Node.js can do that. Let’s try something hard. To make things interesting, we’ll trigger a bug on the server.

[types a message and triggers a bug]

Damn. I must have said something bad. But as you can see, Mary and Jack can still chat. Of the 300,000 users on the server, I’m the only one affected.

Let’s see if we can fix the bug. Here’s the code. Holy crap! It looks like the original programmer, in his zeal, added some forbidden utterances. Let’s clean this up because we don’t want to censor.

[fixed bug in code]

Okay, I recompiled the code. Next, I’ll reload it in real time with 300,000 concurrent chats going on.

[reloads code]

Done. In case you missed it, here it is again — in slow motion.

[reloads code, in slow motion]

For some of you, what you just saw is not self explanatory. If gracefully handling bugs with hundreds of thousands of concurrent sessions and fixing them in real time without stopping the server seems passe, you can aways use Ruby. Then, when your app doesn’t scale, move to Node.js. Then, when you can’t maintain your code because it’s a pile of tangled slinkies, move to Psychobitch. With that experience you can appreciate what you just saw. For the rest of you, yes, that just happened.

[video stops]

Rex: So, what do you guys think about Psychobitch?

Robert: She’s utterly terrifying.

Joe: It makes me wonder if perhaps Psychobitch is a real person.

Rex: You mean, is the name based on my experience with a girlfriend who’s bat shit crazy, drop dead gorgeous, brilliant, and brags to her friends about costing me tens of thousands of dollars in therapy over the last three years?

Mike: Oh shit. I know a girl like that. She scarred me for life, literally. Seriously, I’ll show you the scars!

[credits]

Erlang The Movie II: The Sequel — aka Outlaw Techno Psychobitch — was by far the most involved video I’ve ever done. This page gives what I hope is proper attribution to the inspirational sources!

Erlang The Movie

This of course is sacred text in the Erlang community. I was nervous about tampering with it. But this more a fork of the original, which remains unblemished despite my attempts!

hehasinsomnia

I wanted a classic Internet webcam monologue and hehasinsomnia is perfect. Obviously he was at my mercy from a dialog standpoint, but this guy is passionate and impressively honest. What more could you ask of a spokesman for OTP?

Industrial Dance by D_TokSIcK

The name “Outlaw Techno Psychobitch” came first, but this Internet artist inspired the piece’s darkness and edge. She is brilliant in her craft. Robert’s concluding remark, “she’s utterly terrifying” is apt. Utterly beautifully terrifying!

Death Metal “Venus”

As “the Bananarama of languages”, Erlang needed an upgrade. And this is one bad ass upgrade! The Dutch accent kills!

The Chat Server

The chat server is real. The 300K user connection demo is real. The screen shots are real. [1]

If you want help replicating the results, just ask.

Here’s the source. It’s very simple.

https://github.com/gar1t/hnblows

The name “hnblows” was from a running joke about Hacker News that turned into a running joke about Ryan Dahl.

OTP

This is the modified version of OTP, which is used in the demo. It’s full on Erlang/OTP, but with some UI changes.

https://github.com/gar1t/otp

If you want to be super bad ass, use this version in your organization — your boss will be like, “wait, what — you were serious about moving the payments system to Psychobitch??”

Ryan Dahl

The jokes about Ryan’s tweets are just that.

Ryan has publicly said some things over the years that he may regret. Whatever. People should be free to say stupid things and not be made a Pariah. I wish he hadn’t deleted his Twitter account. I have the utmost respect for Ryan’s work — his accomplishments speak for themselves.

Footnotes

[1] The screen shot of the source code fix was taken separately, after the demo, for formatting purposes. If you spot the typo of Ryan’s name and compare it to the original error, you’ll see that it couldn’t possibly be the actual bug source. Rest assured, the hot code reloading happened exactly as portrayed in the video!

Here are the slides for my talk at Tech Mesh 2012 in London.

Building an Application Platform - Lessons from CloudBees

WARNING: Contains disturbing images of Drunken Stumble.

This is a followup to Solving Embarrassingly Obvious Problems In Erlang, which provides the religious basis for what follows.

I use Python on occasion when I want to indulge my inner hacker. Python doesn’t ask you questions or give you guff — it does what it’s told. You want to do this, Python lets you do this.

It fits your brain.

And for those of you who have special brains, there’s ML.

So this is a simple program I wrote to print out the row counts for each of the tables in a database:

import sys
import MySQLdb

def count(db_name):
    db = MySQLdb.connect()
    db.select_db(db_name)
    cur = db.cursor()
    cur.execute("show tables")
    tables = [t[0] for t in cur]
    for table in tables:
        cur.execute("select count(*) from %s" % table)
        count = cur.fetchone()[0]
        print "%s %i" % (table, count)

if __name__ == "__main__":
    count(sys.argv[1])

And I can tell you it was fun to write! I typed and typed and hacked and tweaked and eventually it all just worked!

And then the guilt set in.

Just because I’m using Python — and this code looks pretty clean — I’m never justified in hiding perfectly good logic inside a long list of instructions!

“God of Clarity, forgive my sins. Give me the strength to work until my code is embarrassingly obvious…”

Looking over the code, the intent is pretty simple: I want to print a list of tables for a database, along with each table’s row count.

After a short meditation, these operations came into view:

• Connect to the database and get a cursor

• Enumerate the tables in the database

• Get the row counts for each table

• Print the tables along with their respective row counts

And after some fiddling:

def print_db_table_row_counts(db_name):
    cur = db_cursor(connect_db(db_name)
    print_row_counts(row_counts(all_tables(cur), cur))

When I stare at this code for just a few seconds, it looks really obvious to me! Each of the bullets above is neatly reflected in the function calls. It says exactly what I mean!

Next time I look at this code and ask, “what’s going on here?” I’ll squint for a few seconds and say, “ah, okay — just what the function name says it does.” No documentation, no comments — just expression-of-intent using functions!

Now the implementation.

connect_db looks like this:

def connect_db(db_name):
    return MySQLdb.connect(db=db_name)

The skeptic will say, “What a waste of a typing! The implementation is so trivial that it can be used directly!”

The abstractionists¹ will say, “Brilliant! The function hides the implementation, which lets you change the database driver without breaking the rest of the code!”

Both wrong!

To the skeptic, the point of the function is to represent an essential logical operation, which is to connect to the database. While MySQLdb.connect(db=db_name) does that, it doesn’t say that. It says, “use the MySQL Python driver to connect to a MySQL server and select a particular database.” That’s not what I mean at all! I just want to connect to a database. I don’t want to cloud the picture with a driver, a method invocation, and a particular keyword argument!

To the abstractionist, the point of the function is not to allow me to change the implementation! Hiding an implementation detail in a function doesn’t magically make the implementation details go away. The function returns a MySQL database object, which other parts of the module relies on. There’s a lot more work to even attempt such an abstraction.²

The function is there to represent the operation. That’s all.

To drive this point home, here’s the function that returns a cursor, given a database:

def db_cursor(db):
    return db.cursor()

Meditate on this and your path will become illuminated.

Now that we have our cursor we can pass it along to any operation that needs to query the database.

def all_tables(cur):
    cur.execute("show tables")
    return [row[0] for row in cur]

Using Erlang style decomposition, I’d be tempted to factored this function to this:

def all_tables(cur):
    return handle_show_tables_result(
        execute_query("show tables", cur), cur)

def execute_query(sql, cur):
    return cur.execute(sql)

def handle_show_tables_result(_, cur):
    return [row[0] for row in cur]

These functions articulate the separate logical operations: execute the query and process the results.

But I don’t like this in Python, at all. The function pipelining pattern makes more sense in Erlang, which provides pattern matching function clauses and encourages side-effect free functions. Python however, uses side effects everywhere and this pattern, to me, splits up logic that belongs together.

So I’ll stick with the first version. It’s far more Pythonic and will induce much less head-scratching.

For row_counts, we can use a straight forward list comprehension:³

def row_counts(tables, cur):
    return [(table, row_count(table, cur)) for table in tables]

In row_count, as in all_tables, we’ll run a query and handle the result in the same function:

def row_count(table, cur):
    cur.execute("select count(*) from %s" % table)
    return cur.fetchone()[0]

It still feels like we’re doing too much here, but I think it’s the right thing in Python.

Finally, our print function:

def print_row_counts(counts):
    for table, count in counts:
        print "%s %i" % (table, count)

Yes, we’re using a for statement. But this is Python — and there’s no sane alternative. Functional purists might be tempted to use a higher order function, but we don’t encourage self flagellation in our order.

There’s one more niggly problem with the original code:

if __name__ == "__main__":
    count(sys.argv[1])

This of course is is incomprehensible. This is what I want to say:

def db_name_arg():
    return sys.argv[1]

if __name__ == "__main__":
    print_db_table_row_counts(db_name_arg())

And so I present the refactored, far more obviously correct code:

import sys
import MySQLdb

def print_db_table_row_counts(db_name):
    cur = db_cursor(connect_db(db_name))
    print_row_counts(row_counts(all_tables(cur), cur))

def connect_db(db_name):
    return MySQLdb.connect(db=db_name)

def db_cursor(db):
    return db.cursor()

def all_tables(cur):
    cur.execute("show tables")
    return [row[0] for row in cur]

def row_counts(tables, cur):
    return [(table, row_count(table, cur)) for table in tables]

def row_count(table, cur):
    cur.execute("select count(*) from %s" % table)
    return cur.fetchone()[0]

def print_row_counts(counts):
    for table, count in counts:
        print "%s %i" % (table, count)

def db_name_arg():
    return sys.argv[1]

if __name__ == "__main__":
    print_db_table_row_counts(db_name_arg())

A few points jump out at me:

• This is Python, not weirdo Erlang — any language that supports functions can be used to solve embarrassingly obvious problems!

• Line count per function still seems to be a good proxy for code quality

• Looking at the Python code I’ve written over the years, I have sinned gravely

Fortunately, the sins of our past can be washed away, if we only apply ourselves to rigorous thinking — and endure typing these small, embarrassingly obvious functions!

Footnotes