Home :: Blogs :: Thomas's blog

Patching VERP into Django

By Thomas - Posted on October 18th, 2009

Any decent sized website that sends emails needs a way to keep its email addresses up-to-date. The standard for doing this is Variable envelope return path (VERP). With VERP, we send a unique id in the Return-Path header of every email. If the mail's bounced, we pick the id out of the To: header and invalidate the associated email address.

Oddly Django doesn't seem to offer this. But what it does have is an emailconfirmation app and a mailer app, which gives us tracking logic to email addresses, and a single place to patch the outbound mailing logic.

So first let's define where we'd like our mails bounced back to in settings.py ...

EMAIL_VERP = 'verp+@verp.we20.org'

Later this will get expanded to something like verp+g7df889df@verp.we20.org

Now we need to add a custom mail sending function in mailer/engine.py . This function adds the magic id to your outbound emails. (You'll also need to replace the call to Django's core mailing function.)

from django.core.mail import EmailMessage
 
try:
    from emailconfirmation.models import EmailConfirmation, EmailAddress
except ImportError:
    EmailConfirmation = None
 
from settings import EMAIL_VERP
 
...
 
def send_an_email(subject, body, from_address, to_addresses):
    """
    Send an email message with appropriate VERP header
    """
    for to_address in to_addresses:
        email_address = EmailAddress.objects.filter(email=to_address)[:1]
        if EMAIL_VERP and EmailConfirmation and email_address:
            salt = sha.new(str(random())).hexdigest()[:5]
            confirmation_key = sha.new(salt + to_address).hexdigest()[:12]
            confirmation = EmailConfirmation(email_address=email_address[0], sent=datetime.now(), confirmation_key=confirmation_key)
            verp_response = EMAIL_VERP.replace('+', '+' + confirmation_key)
            email = EmailMessage(subject, body, from_address, [to_address], headers = {'Return-Path': verp_response})
            email.send()
            confirmation.save()
        else:
           email = EmailMessage(subject, body, from_address, [to_address])
           email.send()

Now we have a return path of verp+something@verp.we20.org on our emails, and our mail server will soon start get bounces sent to those addresses.

Then we need to get the id out of the bounced mail and feed it to some django command. How you do this will depend on your mail server. If you're using postfix as a server, you could use a procmail script looking roughly like this

:0:
* ^TO_verp+\/([A-z0-9])@we20.org
* ^Subject:.*\<(U|u)ndeliverable\>
{
  :0
  |python some/path/manage.py disable_address $MATCH
  :0:
  $DEFAULT
}

All we're interested in is the id. The exact messages to look for will depend on your mail server set-up. Here we are also delegating managing temporary failures [full mailboxes, unavailable servers, etc] to our mail delivery server - it will send us an "Undelivery" message to process after its given up.

To finally invalidate our bounced email addresses we need a Django management command that takes the id's. So for management/commands/disable_address.py we would have this code.

import logging
 
from django.conf import settings
from django.core.management.base import LabelCommand, CommandError
from django.db import models, IntegrityError
from emailconfirmation.models import EmailAddress, EmailConfirmation, EmailConfirmationManager
 
from mailer.engine import send_all
 
class Command(LabelCommand):
    help = 'Disable given email addresses from their confirmation codes'
 
    def handle_label(self, label, **options):
        try:
            confirmation = EmailConfirmation.objects.get(confirmation_key=label)
        except:
            return None
        if not confirmation.key_expired():
            email_address = confirmation.email_address
            email_address.verified = False
            email_address.set_as_primary(conditional=False)
            email_address.save()

And that's it.