Sending Django’s WSGIRequest to a celery task

There may end up being an instance or 2 where you need access to the initial request object inside a celery task, for want of forming an absolute URL for an email template that you’re generating asynchronously, amongst many other reasons. However you’ll find that the WSGIRequest object can’t be pickled because of bound methods and lambda’s that exist on the object itself. So when you try to send the task, it’ll fall over raising an Exception stating that the WSGIRequest object can’t be pickled.

No worries! Here we monkey patch WSGIRequest’s __reduce__ method to allow it to be pickled safely. You’ll need to drop this into a file that gets imported relatively early on in the project. What I usually do is have a monkey.py file sitting at the root of my project, and in the settings file simply “import monkey”.

METACOPY = ['HTTP_HOST', 'REMOTE_ADDR', 'HTTPS', 'SERVER_NAME', 'SERVER_PORT']
def __reduce__(self):
    from authentication.models import AnonymousUser
    from django.http import HttpRequest
 
    request_dict = {
        'META': dict([
            (k, self.META[k]) for k in METACOPY
            if k in self.META and
            isinstance(self.META[k], (int, str, unicode, bool))
        ]),
        'POST': self.POST,
        'GET': self.GET,
        'user': self.user if hasattr(self, 'user') else AnonymousUser(),
        'path': self.path
    }
    return HttpRequest, (), request_dict
from django.core.handlers.wsgi import WSGIRequest
WSGIRequest.__reduce__ = __reduce__

Basically all we’re doing here is creating a new dictionary that only includes the most amount of information that the WSGIRequest object needs to reconstruct itself with. The bound methods that prevent it from being pickled in the first place can be recreated when the pickled object is expanded (and for the most part you’ll probably find you don’t use them anyway). All we really care about is the META, GET, POST, user and the path info, all of which we’re including here. This makes the WSGIRequest object pickle-able, and allows access to a proper request object within a celery task.

I’ve been using this code in production for little over a year now with zero side effects, so I can vouch for its safety.

Happy pickling!