First I want to say that there are many ways of achieving this goal but this is the way I set up. So I do not guarantee that this will work for you!
Concept
A web server faces the outside world. It can serve files (HTML, images, CSS, etc) directly from the file system. However, it can’t talk directly to Django applications; it needs something that will run the application, feed it requests from web clients (such as browsers) and return responses.
A Web Server Gateway Interface (WSGI) does this job. WSGI is a Python standard.
uWSGI is a WSGI implementation. In this tutorial we will set up uWSGI so that it creates a Unix socket, and serves responses to the web server via the WSGI protocol. At the end, our complete stack of components will look like this:
web_client <=> web_server <=> socket <=> uwsgi <=> Django
Install pip
pip will make our job easier later on when we want to add other python packages. pip requires setuptools. Run following commands to install them.
yum update yum install python-setuptools curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py python get-pip.py |
Install Django
Once pip and setuptools are installed, run the following to install Django and create a new project like so:
pip install django mkdir -p /var/www/django cd /var/www/django django-admin.py startproject <new-project> |
Where new-project is the name of your project.
To find out where all the Django source files located:
# python -c "import sys; sys.path = sys.path[1:]; import django; print(django.__path__)" ['/usr/lib/python2.6/site-packages/django'] |
Check which version is installed:
# python -c "import django; print(django.get_version())" 1.5.2 |
Install South
When deploying code to an application server, it will often be necessary to perform database migrations. In this case, South is the perfect tool for the job.
# easy_install South 2>&1 > install.south.log |
The “2>&1 > install.south.log
” is to redirect all output stderr and stdout, just for record purposes. You can use the same idea for above installations.
If you’ve already got an old version of South, and want to upgrade, use:
# easy_install -U South |
Configuring Django installation
Now you’ve installed South system-wide, you’ll need to configure Django to use it. Doing so is simple; just edit your settings.py
and add ‘south
‘ to the end of INSTALLED_APPS
.
Once South is added, you’ll need to sync the database to make the South migration-tracking tables (South doesn’t use migrations for its own models, for various reasons).
python manage.py syncdb |
Example:
# python manage.py syncdb Syncing... Creating tables ... Creating table auth_permission Creating table auth_group_permissions Creating table auth_group Creating table auth_user_groups Creating table auth_user_user_permissions Creating table auth_user Creating table django_content_type Creating table django_session Creating table django_site Creating table south_migrationhistory You just installed Django's auth system, which means you don't have any superusers defined. Would you like to create one now? (yes/no): yes Username (leave blank to use 'root'): Email address: [email protected] Password: Password (again): Superuser created successfully. Installing custom SQL ... Installing indexes ... Installed 0 object(s) from 0 fixture(s) Synced: > django.contrib.auth > django.contrib.contenttypes > django.contrib.sessions > django.contrib.sites > django.contrib.messages > django.contrib.staticfiles > south Not synced (use migrations): - (use ./manage.py migrate to migrate these) |
To learn more about South check out the following tutorials:
http://south.readthedocs.org/en/latest/tutorial/part1.html
http://south.aeracode.org/wiki/Tutorial1
Install django-enumfield
django-enumfield provides an enumeration Django model field (using IntegerField) with reusable enums and transition validation. More info: https://github.com/5monkeys/django-enumfield
$ sudo pip install django-enumfield Password: Downloading/unpacking django-enumfield Downloading django-enumfield-1.0c1.tar.gz Running setup.py egg_info for package django-enumfield Installing collected packages: django-enumfield Running setup.py install for django-enumfield Successfully installed django-enumfield Cleaning up... |
Install uWSGI
Before we install uWSGI we need to first install some dependencies:
yum install python-devel libxml2-devel zlib-devel openssl-devel pcre-devel gcc autoconf automake pip install uwsgi |
Once uwsgi successfully installed, you will see the following at the end.
################# uWSGI configuration ################# pcre = True kernel = Linux malloc = libc execinfo = False ifaddrs = True ssl = True matheval = False zlib = True locking = pthread_mutex plugin_dir = . timer = timerfd yaml = True json = False filemonitor = inotify routing = True debug = False zeromq = False capabilities = False xml = libxml2 event = epoll ############## end of uWSGI configuration ############# *** uWSGI is ready, launch it with /usr/bin/uwsgi *** Successfully installed uwsgi |
Basic test
Create a file called test.py:
def application(env, start_response): start_response('200 OK', [('Content-Type','text/html')]) return "Hello World" |
Run uWSGI:
uwsgi --http :8000 --wsgi-file test.py |
The options mean:
http :8000
– use protocol http, port 8000wsgi-file test.py
– load the specified file, test.py
Visit your site at: http://example.com:8000
If you see “Hello World
“, it means the following stack of components works:
web_client <=> uWSGI <=> Python
Test your Django project
Now we want uWSGI to do the same thing, but to run a Django site instead of the test.py
module.
If you haven’t already done so, make sure that your mysite
project actually works. In your Django project’s directory run this:
python manage.py runserver 0.0.0.0:8000 |
If that works, run it using uWSGI:
uwsgi --http :8000 --module mysite.wsgi |
module mysite.wsgi
– load the specified wsgi module
Point your browser at the server; if the site appears, it means uWSGI is able serve your Django application, and this stack operates correctly:
web_client <=> uWSGI <=> Django
Now normally we won’t have the browser speaking directly to uWSGI. That’s a job for the webserver, which will act as a go-between.
Install NGINX
To add NGINX yum repository, create a file named /etc/yum.repos.d/nginx.repo
and paste the configurations below:
[nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/$releasever/$basearch/ gpgcheck=0 enabled=1 |
After that we can install nginx with yum like so:
yum install nginx chkconfig nginx on /etc/init.d/nginx start |
And now check that the nginx is serving by visiting it in a web browser on port 80 – you should get a message from nginx: “Welcome to nginx!”. That means these components of the full stack are working together:
web_client <=> web_server
If something else is already serving on port 80 and you want to use nginx there, you’ll have to reconfigure nginx to serve on a different port.
Configure NGINX for your site
You will need the uwsgi_params
file, which is available in the NGINX directory of the uWSGI distribution.
Copy it into your project directory. In a moment we will tell NGINX to refer to it.
cp /etc/nginx/uwsgi_params /var/www/mysite/ |
Now create a file called /var/www/mysite/mysite_nginx.conf
, and put this in it:
# mysite_nginx.conf # the upstream component nginx needs to connect to upstream django { # server unix:///var/www/mysite/mysite.sock; # for a file socket server 127.0.0.1:8001; # for a web port socket (we'll use this first) } # configuration of the server server { # the port your site will be served on listen 80; # the domain name it will serve for server_name 172.245.9.46; # substitute your machine's IP address or FQDN charset utf-8; # max upload size client_max_body_size 75M; # adjust to taste # Django media location /media { alias /usr/share/nginx/html/media; # your Django project's media files - amend as required } location /static { alias /usr/share/nginx/html/static; # your Django project's static files - amend as required } # Finally, send all non-media requests to the Django server. location / { uwsgi_pass django; include /var/www/mysite/uwsgi_params; # the uwsgi_params file you installed } } |
This config file tells NGINX to serve up media and static files from the filesystem, as well as handle requests that require Django’s intervention. For a large deployment it is considered good practice to let one server handle static/media files, and another handle Django applications, but for now, this will do just fine.
Symlink to this file from /etc/nginx/conf.d
so nginx can see it:
ln -s /var/www/mysite/mysite_nginx.conf /etc/nginx/conf.d |
Basic NGINX test
Reload NGINX:
/etc/init.d/nginx reload |
To check that media files are being served correctly, add an image called media.png
to the /usr/share/nginx/html/media
directory, then visit http://example.com/media/media.png
– if this works, you’ll know at least that nginx is serving files correctly.
It is worth not just restarting nginx, but actually stopping and then starting it again, which will inform you if there is a problem, and where it is.
NGINX and uWSGI and test.py
Let’s get NGINX to speak to the “Hello World” test.py
application.
uwsgi --socket :8001 --wsgi-file test.py |
This is nearly the same as before, except this time one of the options is different:
socket :8001
– use protocol uwsgi, port 8001
NGINX meanwhile has been configured to communicate with uWSGI on that port, and with the outside world on port 80. Visit: http://example.com
to check.
And this is our stack:
web_client <=> web_server <=> socket <=> uWSGI <=> Python
Meanwhile, you can try to have a look at the uswgi output at http://example.com:8001
– but quite probably, it won’t work because your browser speaks http, not uWSGI, though you should see output from uWSGI in your terminal.
Using Unix sockets instead of ports
So far we have used a TCP port socket, because it’s simpler, but in fact it’s better to use Unix sockets than ports – there’s less overhead.
Edit mysite_nginx.conf
, changing it to match:
server unix:///path/to/your/mysite/mysite.sock; # for a file socket # server 127.0.0.1:8001; # for a web port socket (we'll use this first) |
Reload NGINX:
/etc/init.d/nginx reload |
Run uWSGI again:
uwsgi --uid nginx --gid nginx --socket mysite.sock --wsgi-file test.py |
This time the socket option tells uWSGI which file to use.
Visit: http://example.com
Doesn’t work?
Check your nginx error log (/var/log/nginx/error.log
). If you see something like:
connect() to unix:///path/to/your/mysite/mysite.sock failed (13: Permission denied) |
Then probably you need to manage the permissions on the socket so that nginx is allowed to use it.
Try:
uwsgi --uid nginx --gid nginx --socket mysite.sock --wsgi-file test.py --chmod-socket=666 |
You may also have to add your user to nginx’s group, or vice-versa, so that nginx can read and write to your socket properly.
It’s worth keeping the output of the nginx log running in a terminal window so you can easily refer to it while troubleshooting.
Run Django application with uWSGI and NGINX
Let’s run our Django application:
uwsgi --uid nginx --gid nginx --socket mysite.sock --module mysite.wsgi |
Now uWSGI and NGINX should be serving up not just a “Hello World” module, but your Django project.
Configure uWSGI to run with .ini file
We can put the same options that we used with uWSGI into a file, and then ask uWSGI to run with that file. It makes it easier to manage configurations.
Create a file called mysite_uwsgi.ini
:
[uwsgi] # the base directory (full path) chdir = /var/www/mysite # project's wsgi file module = mysite.wsgi # master master = true # master process id pidfile = /tmp/mysite-master.pid # simple rule is no. of cores on machine processes = 1 # user id uid = nginx # group id gid = nginx # using unix socket (use the full path to be safe) socket = /var/www/mysite/mysite.sock # respawn processes after serving 5000 requests (avoid memory leaks) max-requests = 5000 # clear environment on exit vacuum = true enable-threads = true # background the process daemonize = /var/log/uwsgi/mysite.log |
And run uswgi using this file:
uwsgi --ini mysite_uwsgi.ini |
Once again, test that the Django site works as expected.
Reloading the server
When running with the master process mode, the uWSGI server can be gracefully restarted without closing the main sockets.
There are several ways to make uWSGI gracefully restart.
# using kill to send the signal kill -HUP `cat /tmp/mysite-master.pid` # or the convenience option --reload uwsgi --reload /tmp/mysite-master.pid # or if uwsgi was started with touch-reload=/tmp/mysite-master.pid touch /tmp/mysite-master.pid |
Or from your application, in Python:
uwsgi.reload() |
Stopping the server
If you have the uWSGI process running in the foreground for some reason, you can just hit CTRL+C to kill it off.
When dealing with background processes, you’ll need to use the master pidfile again. The SIGINT signal will kill uWSGI.
kill -INT `cat /tmp/mysite-master.pid` # or for convenience... uwsgi --stop /tmp/mysite-master.pid |
Emperor mode
uWSGI can run in ‘emperor‘ mode. In this mode it keeps an eye on a directory of uWSGI config files, and will spawn instances (‘vassals’) for each one it finds.
Whenever a config file is amended, the emperor will automatically restart the vassal.
Create a directory for the vassals.
# mkdir /etc/uwsgi # mkdir /etc/uwsgi/vassals |
Symlink from the default config directory to your config file.
ln -s /var/www/mysite/mysite_uwsgi.ini /etc/uwsgi/vassals/ |
Run the emperor:
uwsgi --emperor /etc/uwsgi/vassals --uid nginx --gid nginx |
The options mean:
emperor
– where to look for vassals (config files)uid
– the user id of the process once it’s startedgid
– the group id of the process once it’s started
Check the site; it should be running.
Start uWSGI when system boots
The last step is to make it all happen automatically at system boot/reboot.
Edit /etc/rc.local
and add:
/usr/bin/uwsgi --emperor /etc/uwsgi/vassals --uid nginx --gid nginx |
And that should be it!
Further configuration
It is important to understand that this is only a tutorial to get you started. You do need to read the NGINX and uWSGI documentation, and study the options available before deployment in a production environment.
Separating MySQL
When the traffic increases you’ll quickly run into resource contention between the different pieces of software. Database servers and Web servers love to have the entire server to themselves, so when run on the same server they often end up “fighting” over the same resources (RAM, CPU) that they’d prefer to monopolize.
This is solved easily by moving the database server to a second machine.
yum install mysql-server
chkconfig mysqld on
service mysqld start |
To set up MySQL run:
/usr/bin/mysql_secure_installation |
which will also give you the option of removing the test databases and anonymous user created by default. This is strongly recommended for production servers.
Creating Database
Login to MySQL:
[[email protected] ~]# mysql -u root -p |
Creating Database and User:
mysql> CREATE DATABASE django DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_unicode_ci; mysql> GRANT ALL PRIVILEGES ON django.* TO 'USER'@'192.168.1.100' IDENTIFIED BY 'PASSWORD'; |
iptables
If you have iptables enabled and want to connect to MySQL from another machine you’ll need to open a port in your server’s firewall (the default port is 3306). You don’t need to do this if the application using MySQL is running on the same machine.
If you do need to open a port (again, only if you’re accessing MySQL from a different machine from the one you’re installing on), you can use the following rules in iptables to open port 3306:
iptables -I INPUT -p tcp --dport 3306 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -I OUTPUT -p tcp --sport 3306 -m state --state ESTABLISHED -j ACCEPT service iptables save |
Testing MySQL remote connection
We need to make sure that our application server is able to connect to the database. On the server where you have Django, uWSGI and NGINX installed, run the following to install MySQL-python
:
[[email protected] ~]# yum install MySQL-python Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.gigenet.com * extras: yum.singlehop.com * updates: mirror.ash.fastserv.com Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package MySQL-python.x86_64 0:1.2.3-0.3.c1.1.el6 will be installed --> Processing Dependency: libmysqlclient_r.so.16(libmysqlclient_16)(64bit) for package: MySQL-python-1.2.3-0.3.c1.1.el6.x86_64 --> Processing Dependency: libmysqlclient_r.so.16()(64bit) for package: MySQL-python-1.2.3-0.3.c1.1.el6.x86_64 --> Running transaction check ---> Package mysql-libs.x86_64 0:5.1.69-1.el6_4 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================================================= Package Arch Version Repository Size ================================================================================================================================================= Installing: MySQL-python x86_64 1.2.3-0.3.c1.1.el6 base 86 k Installing for dependencies: mysql-libs x86_64 5.1.69-1.el6_4 updates 1.2 M Transaction Summary ================================================================================================================================================= Install 2 Package(s) Total download size: 1.3 M Installed size: 4.3 M Is this ok [y/N]: y Downloading Packages: (1/2): MySQL-python-1.2.3-0.3.c1.1.el6.x86_64.rpm | 86 kB 00:00 (2/2): mysql-libs-5.1.69-1.el6_4.x86_64.rpm | 1.2 MB 00:00 ------------------------------------------------------------------------------------------------------------------------------------------------- Total 2.4 MB/s | 1.3 MB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : mysql-libs-5.1.69-1.el6_4.x86_64 1/2 Installing : MySQL-python-1.2.3-0.3.c1.1.el6.x86_64 2/2 Verifying : MySQL-python-1.2.3-0.3.c1.1.el6.x86_64 1/2 Verifying : mysql-libs-5.1.69-1.el6_4.x86_64 2/2 Installed: MySQL-python.x86_64 0:1.2.3-0.3.c1.1.el6 Dependency Installed: mysql-libs.x86_64 0:5.1.69-1.el6_4 Complete! |
To test the installation, open a command prompt or shell. Enter the Python interactive interpreter and import the MySQLdb module. If this returns without error, then you are all set. If not go back and try to reinstall the MySQLdb package.
[root@web ~]# python Python 2.6.6 (r266:84292, Feb 22 2013, 00:00:18) [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import MySQLdb |
Create a file db_connection_test.py
and add the following:
#!/usr/bin/python import MySQLdb # Open database connection db = MySQL.connect("HOST","USER","PASSWPRD","DATABASE") # prepare a cursor object using cursor() method cursor = db.cursor() # execute SQL query using execute() method. cursor.execute("SELECT VERSION()") # Fetch a single row using fetchone() method. data = cursor.fetchone() print "Database version : %s " % data # disconnect from server db.close() |
When you run this script, you will see following output on your machine.
[[email protected] ~]# python db_connection_test.py Database version : 5.1.69 |
So we know that connecting to the remote MySQL is working.