Cal Henderson
"Why I Hate Django"

"Flickr is basically a website for sharing photos of kittens." - Cal

I am an engineering manager and I manage the engineering team at Flickr.
Flickr: 2.8 billion photos
35k photos/sec
>2m photos/min
Many thousand of pages/sec
Big Scale

I am a scale nerd.
Flickr is written in PHP, it is a proper serious language.
PHP: Yahoo, Wordpress, Mediawiki, Facebook, Digg, Delicious
Flamework is the name of the framework that Flickr uses.
Cal is taking the piss out of serious and grownup languages.

Why is Cal criticizing Django.
He has experience with Pownce.
A fucking blogging engine - 'Like everyone on the planet, I am working on a blogging engine in Django'
[Cal has the most hatred for Smug Rails Developers]

An attack on several fronts:
The Team
You can't have a good framework without a team that starts a religion in Chicago.
Frameworks: Serious business
Struts - inventor - Craig McClanahan (has a beard - so it is a serious framework)
Struts is for suits! Big boring apps without AJAX.

{Many blessings on Cal for making us laugh.}

Rails is the hot shit. (compares DHH to David Hasselhoff - 80s hair)

Django => a Boy Band (Shows photo of the Django core team looking 13)
Also, a wizard : Simon "Stinky" Willison
Can't take Django seriously due to lack of beards & 80s hair.

Python is pretty trendy. Google uses it. But not that trendy. It's no Ruby or Haskell or Erlang or Scala.

Python has significant whitespace. Either you have to do it write or not at all.
Cal then extolls the wonders of Perl.

Salmon => Homespring (greatest programming language ever)
Universe of marshy force.

No quantum superpositions.
This is UNforgiveable: "-".join(array)
Restarting the server? WTF?

"I buy my eggs at the cheese shop" - Arron Straup Cope [moon language]
A serious programming language where you buy eggs at a cheese shop. A serious programming language would make an egg factory.

Frameworks
Flamework is a framework- what does that mean?
make common tasks: faster, easier, error-free
On flickr: show a list of photos, write and aPI method, kill all humans
Front-tier frames, Act as glue (not proper grownup Java programming)

Frameworks: Great for getting started. Perhaps slows you down later. What is it doing in there?

Realistically frameworks make your life easier if you are building small to medium sites, up to large sites.

Pick 2 or 3: Fast, Easy, Flexible, Scalable
Large scale is often not a good goal.

Everything here is possible! We care about easy!
It would be nice if Django could do somethings a bit better for us:

Large Scale & Django
No multiple DB clusters
Can't do manual partitioning
No multiple hosts per DB

Where do we read from? Imagine we have a simple master-slave set up, where do we perform these from? What happens when master breaks? Read/write consistency. LAG!
Slaves are good for non-writing users. Non time-sensitive data. Feeds. Read-only APIs.
Cool kids use master/master pairs. With a pair of masters we can write to them any time. Consistent read/write.
Write to side A. Read from side A. ???? PROFIT!!
Then we can start to get fancy, where the paired masters get slaves (obviously far too technical for folks who don't use Java).

Where do we read/write?
Abstraction FTW
db_query($cluster_name, $sql)
No sharding

WTF is sharding? All about books..
Dewey decimal - a way of cataloging - but it can't do everything.
The book thing is a metaphor

Sharding up (like we do at Flickr): Makes some things 'impossible', no shard walking.
But what I really meant was database records (books) and servers (shelves)
Django knows our models - it should be possible to do this on a useful level.
If we accept constraints.

No external sequences - You worry about IDs when you are working with master/master pairs and shards. Colliding IDs.
Auto-incs can be bad in the master/master and shard situations.
No automatic denormalization in Django.
I love denormalization, it is my favorite long word that I know.
Taking normalized data and denormalizing it.
Counters would be easy. Parent and child relationship between the data. Django could be smart with this.

Who cares about high performance?
Django is slow. It is fast enough.
Tehre are limits, but they only apply when being really silly. All of your time should be spent on the database and wire.

Sessions in Django: R u on crak?
lightweight = better
sessions -> sticked into a signed cookie (as all you really need is to know who signed in, stick in their user id)
No Parallelization in Django (yeah, you have threads and all that)
Multiple cache-requests - a bit silly
What we can do is multiple multi-keyed reads (can do it in a millisecond), not such a big deal, but it does matter when Slow HTTP requests while we're doing other stuff (like loading an add).

The dreaded ORM: Creates dumb SQL (apparently now fixed)

Maintainability / Awesomeness
What is with the Verbose template syntax (Cal says Smarty is better. Then he and Simon have a give and take on this). Cal then makes Simon defend his template design choices.
Verbose template syntax makes people cry. Bytes cost money.

Django: No query debugger. Not good enough!
Pownce's query debugger: Better, but not amazing.

Flickr server's query debugger: Awesome summary of queries.
Make the queries clickable. And/or show the data.
You can do really badass stuff like MySQL Query Profiler.

The Django community is not smug enough. The Rails folks take up all the smug and there is not enough to go around.

SELECT * is fast!
Faster over the wire. Don't be afraid of SELECT * it is awesome for a reason.

Django makes me scared of Joins.
Actually, JOINs are evil at a certain scale. You should never use it anyways.

Django 1.0 was released.
I am suspicious of things with low numbers
Django is suspiciously small.
Django: 105Klocs
Flamework: 260klocs
Very long lines?

Django: Can't pluralize octopus
Commit all fields at once
Race conditions!
We should be able to use dirty fields.
Don't make Malcolm cry.
No simple way to increment or decrement counts in your data store. Turns out that they did that in SQL some time ago.
Character encoding - Input validation? I am going to guess that Django doesn't get it right.

Why does Django not have a mascot?
Simon does not count.
Echnidna?
How about one with Magical Powers? (photo of unicorn sign)
No built in deployment system.
Extending models - Tables are never big enough
Model Changes : manage.py reset appname
django-evolutions / dmigrations

What has Django done for you lately?

Thanks - now go make it better.

http://www.iamcal.com/talks/