Rapid Development with Django
How Modern Frameworks are Reshaping the Web
A quick look at the trade press finds a litany of words that have little meaning outside the web development community: “Python,” “RoR,” “Django,” “MVC.” The jargon is enough to put many off, and you might dismiss it as yet another round of Internet hype. Yet behind these words does lie significant improvements in programmer productivity, in the ability to systematically address security threats, and the ability to address performance concerns.
In this paper, I will discuss how the Django framework improves the value proposition of web application development.
CRUD
Unfortunately, I must start out by introducing yet another acronym. CRUD stands for Create, Read, Update, and Delete. It is important because it describes—in very general terms—the functions that perhaps 95 percent of business software performs.
Let’s examine a typical ecommerce website (an online store) to see how the CRUD model applies. When a customer visits the site, the software needs to show them which items are available for sale. So it must go to the database, and read a list of items from it. When a customer first adds an item to their cart, the software must create a new shopping cart for the user. Each subsequent time they add an item, it must update the cart. When they check out their order, the software must delete the shopping cart and create a new receipt record.
On the administrative side, the store manager must be able to create new items when they decide to offer them for sale, read the set of customer receipts to see what’s been ordered, update the receipts to record when the items have been shipped, and delete old items when they’re no longer offered for sale.
As stated above, nearly all business software implements CRUD. The model can be applied just as easily to newspaper websites, to accounting software, to reporting tools, to customer management software, to email clients, et cetra.
So the idea behind Django is to look at CRUD applications on the web, and figure out what they all have in common. Well, they all receive requests from a web server, and they all produce output destined to be viewed with a web browser. They all tend to store their information in a database server, and speak to it using a language called SQL.
Most have a public interface (at least, they have some sense of “normal users”) and a different interface for administrative users. While the former is typically very custom, with a unique look‐and‐feel, the latter tends to be devoted primarily to allowing administrators to browse and tweak the data in the database.
So what the Django developers looked at all these common tasks that CRUD web applications do, and came up with a suite of functions—a toolbox, if you will—that programmers can use to quickly get them out of the way, so they can focus on what is unique about the client’s specific problem.
MVC, or MVT
More acronyms. I’m so sorry. I swear that this will be the last time.
So MVC. This stands for “Model–View–Controller.” What each of those words means isn’t horribly important. The important thing to know about is that it’s a pattern that was developed some time ago for writing CRUD software.
The basic idea is that such software has to have some way to store data, which is called the model, a way to show the data to the user and allow them to modify it, called the view, and in the original design, an overseer to keep it all straight, called the controller.
The Django developers took this and thought it was a really solid model for web development. But they observed that there are a few differences from the classic design. First of all, they observed that the controller business is something they could generalize away so programmers wouldn’t have to deal with it anymore.
They also observed that web development is a little different than that of desktop applications in that you usually have a designer making things look pretty. They figured they needed a way to separate out the making things look pretty from the actual programming, so a separate non‐programmer could work on that. They called that separate part of the system the set of templates.
Since the controller isn’t something that programmers have to deal with anymore, they decided to drop that from their acronym, and since making things look pretty is so important for many websites, they decided to add template. Hence the Django model: MVT, or “Model–View–Template.”
So when a programmer approaches a new Django application, what they have to do is figure out what data they need to store in the database, and what form it should take. Once they get that straightened out, they write a file describing the models. They also figure out what actions the user must perform, and once that’s sorted out they can write the views.
Depending on the needs of the customer, there are usually a few other custom bits that must be written between those, but for the typical CRUD website, that covers the vast majority of the software that needs written. It’s then a matter of turning it over to the designer and have them create templates to make everything look nice.
So all good. The question I’m sure you’re asking is, how does this add value for the client? Why does this view and model stuff save money? It might help look at how things were done before MVT‐style frameworks.
The Old Way
So let’s go back five years or so. Django is not available, but I have a client and (of course) they need a CRUD website. I choose PHP, because it’s the best software technology available. PHP is not a framework, but rather a language that tries to make it easy to write software that talks with web servers.
In PHP, it’s somewhat easy to write views, but it doesn’t do anything about talking to the database. Instead, I had to do all that myself. When I programmed in raw PHP, I would set up the database separately, and then write a bunch of software that would generate commands using the database’s native SQL language.
The process of specifying the layout to the database server, and then writing software to generate SQL to perform all the creates and reads and updates was a tedious task. And debugging the SQL generation routines took still more time.
In Django, after I describe the models, the system automatically tells the database server what they look like, and when I have to access the data, the Django database interface generates the SQL for me.
What’s more. Remember I said that the administrators of websites generally have a set of common tasks they need to do? Well, after I describe the models, Django also auto‐generates an administrative interface that can be used to access them. Certainly, I often have to modify it to fit the client’s needs, but it’s stunning how much time this feature saves. In the old PHP systems, at least a third of my time was spent making administrative interfaces.
While talking to the web server is easy in PHP, it does leave off a lot of useful view functions. For instance, one very common task of views is to take input from the user. This may seem straightforward, but there are several pitfalls inherent in accepting user‐provided data, and properly avoiding them can be a time sink.
Suppose I need to create a view that asks the user for an email address. Well, I now have to verify that what they give me is actually an email address. After all, sometimes they mistype the ‘@’ sign as a ‘#’, or get confused and put their real name in that field. So the program I write has to look over the data they enter and make sure it is actually a valid email address.
This process is called “validation,” and in raw PHP, you have to do that all yourself. So when I write my form that takes an email address, I need to look up the email rules and then spend a lot of time telling the computer what the address should look like. In Django, it provides facilities for defining forms, and specifying what to expect in each field. So all I have to do is tell Django that I need a form that takes an email address. It will do all the validation for me produce an error if the address isn’t valid. A task that may take hours becomes one that takes a few minutes.
And then there’s templating. Recall that the Django developers separated out the facilities designers need to make the web pages look good from the views, and called them templates? In raw PHP, no such separation exists. So the software and the design is all mixed in together.
This makes it difficult to divide out the work of the designer from that of the programmer. Often people would hire a designer who knew just enough PHP to get something working, but who didn’t know how to write secure, reliable, flexible software. This failure to separate the job of designer and programmer is how a lot of very poor quality web software ended up getting written.
Security
So things like database access and form validation go from being very large tasks to things that mostly take care of themselves when moving from an older system like raw PHP to the Django MVT framework. Another advantage worth mentioning is the enhanced security benefits.
For instance, in that old PHP model, the programmer would have to write software that generates database commands himself. He would have to do this all over again for every project. Well, there was a flaw in many of those systems that hackers managed to discover, called “SQL Injection.” The way it worked was that if the programmer wasn’t very careful, he would allow the user to give him data that he wouldn’t validate before trying to put into SQL commands. Recall that SQL is the language that the database speaks.
This may seem harmless, but the trouble comes when hackers carefully fashion the data they type in to look like SQL commands. When the program sends this “data” to the database server, the server thinks its not getting data, but some new commands. For instance, hackers might type in the SQL command to print out all of the customer receipts, so they can see all the orders put in the system.
In PHP, the programmer has to be very careful to validate all data to make sure it’s not a SQL command before putting it into the commands that he is generating. But in Django, there is an SQL generator that comes with the framework, and that system has been carefully written and reviewed by many people to make sure that it always prevents the users from inserting SQL commands.
SQL Injection is an example of a threat on the web that has been the downfall of thousands of web applications, that is eliminated by proper use of the Django framework. Other common security threats are dealt with just as systematically by the system, such as Cross Site Request Forgery (CSRF) and Cross Site Scripting (CSS).
Going into detail about how those attacks work is beyond the scope of this paper, but the point that’s wroth taking away is that frameworks can address these threats in very broad strokes, and eliminate or reduce the chances of them affecting your software. They can give programmers have a safety net, as it were, when dealing with these threats so that it’s much easier to write secure software.
Performance
Generally, websites get bogged down by two problems: either the code that accesses the database is inefficient and the database gets overloaded, or it takes too long to generate the web pages and the application server gets overloaded. Django offers tools for dealing with both problems. In the former case, the database access system that Django uses uses this computer‐sciency technique called laziness.
It often happens that programmers ask the database for information that they never use. Sometimes, this is due to sloppiness, but often it is much more clean to address database access systematically even if that results in a few needless requests. The idea behind laziness is that Django doesn’t actually go to the database when the programmer asks for the information. Instead, it waits until the programmer tries to use the data.
More succinctly, it waits until the last possible minute to go get the data. If the programmer never ends up using the data, then Django never fetches it, and database performance is improved.
To address problems with the time it takes to generate pages, the Django developers first observed that most pages are viewed a whole bunch of times by different people. For instance, if you have a product page, typically hundreds of people will look at that same page over and over again.
So, they figured: why not let the programmer identify those pages, and then generated them only every once in a while, when something changes? In the meantime, the system could save the generated page somewhere, and just show that one copy to all those hundreds of people. This process is called caching, and Django has many robust facilities for web page caching.
A New Value Proposition
So while everything the Django framework does is very abstract for the non‐technical individual, all those highly abstract things cost real money when you’re buying custom software. Django takes all the common tasks that happen every time a CRUD web application is written, and takes care of them. So what is left for the programmer to do is just fill in the specific things that make your project differ from all the other websites out there.
And what about the other words at the beginning of this article? Well, Python is the language Django uses. It has some features of serious programming languages that are sadly missing from PHP, such as exceptions and namespacing. RoR stands for Ruby on Rails, which is another MVC framework which stands in friendly competition with Django. They are very similar, but I tend to recommend Django mostly because I’ve seen more professionalism in the developer community, and because of some very esoteric technical differences, although I continue to monitor RoR development closely.
