Posted
by
Vláďa Macek
on
January 6, 2012.
Filed under
pro tip,
sysadmin
Without patching, the rsync utility lacks support to detect when a file was renamed/moved across multiple directories inside the synced tree. There is a --fuzzy option to save bandwidth by building upon similar files on the target side, but only in the same directory.
You may need to synchronize the large file tree over a slow connection when you’ve done a big reorganization since the last rsync run. A real world example: Joe stores multiple GiBs of family photos and videos at home and periodically backs them up to a remote server.
$ rsync -avHP --delete-after ~/family/Photos remotebox:backups
One day, Joe decides he used the wrong directory layout or the file naming scheme and shuffles these gigabytes under a totally different directory structure, a quick local operation.
Unfortunately, there is no apparent safe and quick way to mirror these changes to the remote backup disk without either ...
Posted
by
Vláďa Macek
on
November 9, 2010.
Filed under
django,
sysadmin
Recently we noticed some intermittent slow downs with our Gondola sites and wanted to track down the source of the issue. Our sites are all Django projects served by Apache/mod_wsgi behind an Nginx frontend proxy. Nginx’s upstream module makes the process of logging response times trivial with its upstream_response_time variable.
I added this block to /etc/nginx/nginx.conf to define a new log format timed_combined which captures some additional data in our logs:
log_format timed_combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'$request_time $upstream_response_time $pipe';
This is the default combined format with the three variables appended to it.
request_time This shows how long Nginx dealt with the requestupstream_response_time Gives us the time it took our upstream server (in this case Apache/mod_wsgi) to respondpipe Shows 'p’ in case the request was pipelined.
Next we modify our access_log directives to use the new format ...
Posted
by
Peter Baumgartner
on
June 24, 2010.
Filed under
pro tip,
sysadmin
I really like supervisord for long-running process management. It is Python, so it is easy to install and it is easy to script with tools like Fabric. Lately I’ve been deploying smaller Django/WSGI sites with Green Unicorn behind Nginx proxied over a Unix socket which makes for a nice little setup on shared hosting and RAM starved VPSes.
After installing supervisord with something like sudo easy_install supervisor, you’ll want to make sure it runs after you reboot your machine (and gets restarted in case it crashes). If you are on a Linux system with upstart (and have root access), you can bypass the ugly init scripts and use this simple file placed in /etc/init/supervisord.conf to manage your supervisord process:
description "supervisord"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /usr/local/bin/supervisord --nodaemon --configuration /etc/supervisord.conf
Now that it is ...
Posted
by
Peter Baumgartner
on
February 12, 2010.
Filed under
pro tip,
sysadmin
Here’s one I struggled with a bit while upgrading lincolnloop.com yesterday.
Scenario: You need to take your site offline and want to redirect all its traffic to a “down for maintenance” page. For search engine friendliness, that page should return a 503: Service Unavailable status code. Here’s the (not entirely intuitive) way to do that in Nginx:
server {
listen 80;
server_name mysite.com;
root /var/www/mysite.com/;
location / {
if (-f $document_root/maintenance.html) {
return 503;
}
... # the rest of your config goes here
}
error_page 503 @maintenance;
location @maintenance {
rewrite ^(.*)$ /maintenance.html break;
}
}
Now whenever you need to take your site offline, simply create the file maintenance.html in the $document_root (in our case, /var/www/mysite.com). If the file exists, Nginx will serve it with a 503 status code, if not, it will proceed as usual.
