Presentation of this series
This post is going to be the first one of series, I will try to formalized some aspects of the development of a python based web project. It is tainted with django since the series is based on the note that I have taken while working on this site but also from the question that I often see coming back on IRC.
Most of them have Not been Invented Here, where I can I will add a link to original author but often this will not be the case because I have no idea is originally from. If you feel like you are the inventor of a particular technique please do not hesitate to claim it in the comments.
These are not THE ULTIMATE BEST PRACTICES but rather represent the techniques I use at a particular time. They are heavily tainted with my own personal opinion and some choices there are not main stream at least not yet...
I will tag these articles with TFG (Tip From Gwadeloop) so you can subscribes to this particular feed.
How to organize your project
The use case that support this series is the infrastructure that power this web site gwadeloop.com. This web site use a mix of public and private applications. The most notable feature are a multilingual cms (django-cms) and a blog.
First of all use the dvcs of your choice they rock and make things easier to work on multiple platform and machine. I have "standardize" myself to mercurial, hg, when I have choice but as soon as you want to do something a bit useful you will probably clone / checkout repositories from hg, bzr, svn, git, ... so it is probably a good idea to just install them all in your working environment.
Regarding the django project itself don't put everything under a single "giant" repository it will just make your life harder later down the road, use as much as repository that is necessary to logically organize your code. Use a repository for your django project and a repository per django application. This will make your life much easier when you will try to reuse some of these applications in another project.
James Bennett has written many articles about this, you can find them online with some recorded presentation from pycon or djangocon.
With regards to the structure of the django project I like to use a structure similar to the django-startproject developed by Lincoln Loop. It comes with a nice way of organizing the settings.
Some of you might be familiar with the local_settings technique where at the end of the main settings.py you add this line ::
1 | from local_settings import * |
The aim there is to overwrite you settings with the one that you have specified in your local_settings.py. The django-startproject from Lincoln Loop reverse this trick and create "conf" directory structure that you will use to defined your settings::
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | project_cms/conf/ |-- __init__.py |-- settings.py |-- urls.py |-- common | |-- __init__.py | `-- urls | |-- __init__.py | |-- admin.py | `-- static.py (...) |-- local | |-- __init__.py | |-- settings.py | `-- urls.py |-- prod | |-- __init__.py | |-- settings.py | |-- urls.py | |-- uwsgi_settings.py (...) |
There is nothing complex here but this organization gives you a structure that avoids the duplication and helps to manage your settings for all the targeted deployment configuration : local, production, staging, test, debug, ...
The idea here is easily run your project in several configurations and environments ::
1 | manage.py test --settings=project_cms.conf.prod.settings |
or ::
1 | manage.py test --settings=project_cms.conf.local.settings |
The first one could run the test suite with postgresql and the second one could run it with sqlite.
One of the drawback of this approach is the number of repositories that you have to integrate to get your project rolling. This can be an issue if you plan to pull them up manually each time you setup your project. I will tackle this issue in my next "Tip From Gwadeloop" but you Must invest on a command line that will get your site up and running from a virgin environment. In my next article I will present the fabric script that I am using to automate these tasks.
The state of python packaging and distribution have considerably improved in the recent years. For me the winners in this category are :
These 3 packages change the way I approach python installation all my projects are now isolated inside a virtualenvthat I can trow away and regenerate on demand with a single command line. You should avoid to install global python packages, a good rule is to reserve this treatment to packages that are either tricky to install locally PIL, lxml or packages that you use everywhere : pip, virtualenv, database driver, ...
Then inside your project directory add a file that will be used by pip to pull all the requirements this file does not need to have a particular name but over the time I have standardize my practice on "requirements.pip". This file should list your django project and all django applications and python library that are required. The benefits form this is that all the applications will be installed in your PYTHONPATH.
I have a startup file, for this requirements.pip that contains the tools that I use for most of my projects ::
1 2 3 4 5 6 7 | # Core packages django==1.1.1 south==0.6 docutils sphinx python-memcached -e hg+http://projects.unbit.it/hg/uwsgi#egg=uwsgi |
All this packages can be installed inside a virtualenv in a single line ::
1 | pip install -E . -r requirements.pip |
This will install all the packages inside the current virtualenv using the globally install pip.
This conclude the first episode of this series titled "Tip From Gwadeloop", in my next articles I will discuss how you can use fabric to automate the common tasks on your project.
Please do not hesitate to let me know if you think that you can improve the techniques presented in this article.