PREPARE!

Learning Web Hosting Best-Practices From WebFaction

Brandon Craig Rhodes
PyCon 2010

3 slides of targeting

What this is not

  • Not an ad
  • Contains no affiliate link
  • Not by a WebFaction employee
  • Not vetted by WebFaction
  • Not about dedicated hosting

Shared hosting

  • You are a mere UNIX user
  • Many UNIX users per box
  • Useful for small sites
  • Costs a few dollars a month
  • Saves me lots of admin costs

Preview: What did I learn?

  • I can tolerate shared hosting
  • Working starting points are productive
  • Control Panel that I could live with
  • Permissions that Just Work
  • Fear no indirection

The WebFaction Landscape

What is on each web server?

  • You get a shell account
  • Postgres and MySQL are already running
  • WebFaction owns the privileged ports
  • Control panel (web or XML-RPC) controls:
    • Web sites
    • DNS entries
    • Mailboxes
    • Outgoing SMTP
landscape.png

Using the Control Panel

  • Add your domain name (“example.com”)
  • Create web apps
  • Create a site to link them
Domain       Path    Application
-----------  ------  -----------
example.com  /       “my_static_files”
             /blog   “my_wordpress”
figures/app-type.png
Lots of Python apps
CherryPy
Django release  (both mod_python and mod_wsgi)
Django trunk    (both mod_python and mod_wsgi)
mod_python
mod_wsgi
Plone
Pylons
Trac
TurboGears
Zope
plus Drupal, Joomla, Rails, WordPress

Your Account

After creating four web apps
through the Control Panel, your
account might look like this:
/home/you
└── webapps
    ├── django-test/
    ├── django-prod/
    ├── static/
    └── wordpress/
But what lies inside?
Simple static site
~/static
├── logo.png
├── banner.jpg
├── extra.css
└── ul-corner2.jpg

paster-based app (pylons, turbogears2)

~/webapps/mystore
├── MANIFEST.in
├── README.txt
├── bin ─ paster, nosetests, test ...
├── development.ini
├── docs ─ index.txt
├── lib
│   └── python2.5 ─ (Python packages)
├── mystore
│   └── (app source code)
├── setup.cfg
└── setup.py

mod_wsgi or mod_python (like django)

~/webapps/my-django-app
├── apache2
│   ├── bin ─ httpd, start, stop, ...
│   ├── conf ─ httpd.conf, mime.types
│   └── lib, logs, modules ...
├── bin
│   └── django-admin.py ...
├── lib
│   └── python2.5 ─ (Python packages)
├── myproject
│   └── manage.py, settings.py, urls.py ...
└── myproject.wsgi

The Contract

  • The Control Panel:
    • Creates and starts the app
    • Creates a crontab entry
    • Never touches the app again!
  • You:
    • Add, modify, remove files
    • Stop and start the service
    • Edit your crontab
    • WF just watches the TCP port!

Quick Start

  • They get you started
  • Configuration works immediately
  • Can use development host name
     myacct.webfactional.com
blog.myacct.webfactional.com
test.myacct.webfactional.com
 tg2.myacct.webfactional.com

“Custom” apps

Since WF only really cares
that your TCP port answers HTTP,
you can dispense with everything else:
Adding a “custom” app in the Control
Panel just allocates a port and adds
an empty directory!
figures/app-custom.png
figures/app-custom2.png
landscape.png

Permissions

$ ls -ld webapps/
drwxr-xr-x 10 root root 4096 Feb 11 11:05 webapps/
  • Only writable by the control panel
  • Assures consistency
  • Probably helps with support
  • But also provides orthogonality!

Security

When so many users share a box
(often, >100 per WebFaction machine)
how do you keep them separate?
  • This is a stability question
  • And a security question

The Dilemma

Q: How can the web server
read all of the following files
but other users not see them?
/home/you/webapps/yoursite/.htaccess
/home/you/webapps/yoursite/.htpassword
/home/you/webapps/yoursite/index.html
/home/you/webapps/yoursite/logo.png
/home/you/webapps/yoursite/portrait.jpg
  • Put the web server in your group?
  • Somehow chgrp them to "www-data"?

The Answer: ACLs

$ ls -ld ~
drwxr-x---+ 22 rhodesmill rhodesmill 4096 ...
  • What does the “+” mean?
  • That ACLs are active

The Answer: ACLs

$ getfacl ~
# file: home/rhodesmill
# owner: rhodesmill
# group: rhodesmill
user::rwx
user:apache:r-x
user:nginx:r-x
group::--x
mask::r-x
other::---

Consequences

  • Default umask can be 0002
  • Other users cannot see anything
  • Web servers can pass and see files

The Boundaries

  • File system
    • nginx can read public files
    • apache can read public files
  • Incoming TCP connections
    • Web servers connect to your ports
  • Outgoing TCP connections
    • Different username/pw for each database
    • Different password to send/receive email

Almost everything in $HOME

  • Except mailboxes and databases
  • Their logs go in ~/logs/frontend/
  • Your logs go in ~/logs/user/
  • Log analyzers are just normal apps
  • You can:
    • Compile new Apache modules
    • Build extra libraries and applications
    • (dev packages regularly present!)

Package Management

  • Their Python looks two extra places:
~/lib/pythonX.Y
~/webapps/foo/lib/pythonX.Y
  • Their easy_install uses ~/lib
  • Can override with tedious arguments
  • Unlike virtualenv, sensitive to where Python script lives rather than which Python interpreter is used

(The tedious arguments)

easy_install-X.Y \
 --install-dir=$HOME/webapps/app/lib/PythonX.Y
 --script-dir=$HOME/webapps/app/bin

Downsides

  • Idiosyncratic
  • Non-standard
  • Non-obvious

Upsides

A script in a given directory
always runs in the same environment
Libraries that link libpython
follow the same rules as scripts
run from the command line
It is concentric

Summary: What works?

  • Working starting points are productive
  • Control Panel can be useful
  • Permissions can Just Work
  • Indirection increases flexibility
  • Using standard tools
  • Clear boundaries

The End

landscape.png

(Notes on things to add to the slides above)

setfacl -R -m d:u:purple:--- $HOME/webapps/*

easy_install
easy_install-X.Y \
 --install-dir=$HOME/webapps/app/lib/PythonX.Y
 --script-dir=$HOME/webapps/app_name/bin

ack! extra webapps lib dir is confusing because only happens on 1st import of new module!

solves problem of how to get interactive prompt, scripts, etc to all follow same rules about adding new sys.paths

  • in my own hosting tried one-size-fits all vs webfaction is not afraid of adding a level of indirection
  • I was afraid of control panels vs webfaction provides control panels with real APIs