Apr 6 2009

The great Twitter Ruby vs Scala war debate


The executive summary:

Twitter prefer Scala rather than Ruby  for some back-end processes.
Fun ensues

Some of the Twitter developers were recently intervied by Bill Veners on Scala: Twitter on Scala

Which seemed to raise the ire of many of the Ruby crowd in the infosphere. The fact that Twitterer(?) Alex Payne has a new Scala book smells of vested interest to many.

As Payne posted in Mending The Bitter Absence of Reasoned Technical Discussion:

the story … had gone from “Scala is a nifty language and you should think about it for your business to “Twitter engineer spits on the grave of Ruby, exalts Scala as shining new deity”

Tony Arcieri (REIA, Erlang) has a really great analysis of message queues and Ruby – the best part is some of the implicated Twitter developers address his points in the comments. The team evaulated various message queues and went with their own implementation (a Scala-based app called Kestrel).

The Twitter guys know their code, know their environment. They have tried a bunch of technologies, and have developed something that works for them.

However, it’s comments like this from the interview that really pique my interest:

I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.” It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.

Jeremy McAnally summed up my own thoughts on Twitter:

If you have to use kind_of? all over your code to mimic a “type system,” you’re doing it wrong.

Twitter: you’re doing it wrong!

I realise that I have no real right to be calling Twitter out here as my credentials with regards to developing the next big thing with a growth curve that is a straight line all the way up to world domination are currently nil.

But the secret truth of most large software projects is that the code often sucks.

See the Big Ball of Mud for more details.

So given Twitter’s code probably sucks, and your code probably sucks, and my code definitely sucks, what can we do?

I guess we assume they picked the right tool to make some of their code suck a little less.

Well done Twitter.

My takeaways:

The Rubyists are a pretty defensive group. I love Ruby and Rails. but still, I am hopefully not a member of the “cult”.

The JVM is just about the best platform there is for high-scalability. Not Java itself as such, but the JVM. I am fairly certain the future is going to be languages running on the JVM. I think Clojure is interesting for this reason – although Scala may be good middle ground as a future-proof language option. Lisps have a long history of being the best lanaguage ever that never made it to the mainstream.


One other thing I do find very curious is that given the code-compile-deploy cycle still required by Scala is how it can really be as fast to develop in as Ruby or Python? During my time as a Java Developer, it was this cycle that was the real time killer. On any non-trival project the compile cycle will start eating minutes and deployment is necessarily complicated when pushing to servers with JAR/WAR deployment systems. I can understand how this trade-off may work for infrastructure (like a message queue) where changes are slow and largely internal, but for a web-level application where change is constant, it’s going to slow you down.

Apr 4 2009

Career at the Crossroads


This week I have been mostly thinking about what I am going to do with my life.

I have been freelancing for the last couple of years.

The idea was never to actually be a full-time freelancer but to create some software and make a living from selling it.

The freelance work always seems to take up most of my available time, consuming all my energy and often the money is not that great, once you factor in the actual time it takes to perform all the tasks required – sales, marketing, project management, accounting, maintenance and once in a while actually writing some code.

I have tried a number of different ideas.

I built an SMS messaging platform that I think is actually pretty cool, but the idea itself requires some sort-of mass-marketing campaign that is really out of the realm of my knowledge and expertise. Not to mention that the advent of the iphone really puts a dint in SMS as a medium for the long term.

I created a CMS-type application for Facebook Apps (called Prefabrikator). This concept evolved into a tool for creating and managing competitions for Facebook. You know the sort of thing – “Tell us why you need an iPod in 25 words or less”.

This project has been plagued by several issues – the constant changes to the FB API has made development less than fun, and I also lost faith in investing more time (and hence money) into a platform that I have no control over. The constant and ongoing changes to Facebook have pushed apps further and further into the background, making some of what I have done rather moot. The final straw is really that because of the steps needed to setup an application inside FB (adding and configuring through the Developer Tools, then adding authentication keys to the Rails app configuration) means that there is really always going to be a manual process involved – each deployment becomes a custom installation, which ruins any hope of making the price terribly accessible.

So …

… at the moment I am working a longer-term contract for the excellent Inspire Foundation on their new ReachOut site and platform. Having a regular income is a huge bonus and working with a team on a great project is great fun. I do get to work from home, but the hours are much more regimented which is actually a benefit. On top of this I still have a couple of clients that I do very minor ongoing maintenance work for.

But the question remains …

Where to next?

The goal is still been to make great software and sell it, but I am increasingly wondering how viable this is as a lone-wolf developer.