jnosal

On programming and stuff

Work on your Python style

| Comments

Goal

Coding style is important and in a way defines every programmer. Everyday we come across well tested code, code that works but is not generic, overthought code and of course crappy code :-). Personally I had a tendency to value code that is practical (does the job well and is straightforward) over generic solutions which I considered to be 'too much for now'. But, as time was passing I've learned that there are couple python tools that provide very good balance between something generic and practical and are still pythonic (that's what cool kids aim for).

The idea behind this article is to show different approaches to typical programmer struggles :-). We'll be designing a very simple metric system (basically a hit counter) for our views or business logic parts.

(Keep in mind that few lines are actually 'pseudo-code')

Slow start

from redis import StrictRedis


def my_view(request):
    # possibly database access

    r = StrictRedis()
    amount = int(r.get('my_view'))
    if not amount:
        amount = 1
    else:
        amount += 1
    r.set('my_view', amount)
    # setup context and render your response

    return AwesomeHttpResponse()

So we've decided to use redis to store our metrics - that's a plus. But, except that, overall quality is rather poor.

Improvements, improvements ...

First: we did not use INCR so we are not guarding against race conditions. Let's fix that (all imports are skipped for convenience):

def my_view(request):
    # possibly database access

    r = StrictRedis()
    r.incr('my_view')
    # setup context and render your response

    return AwesomeHttpResponse()

Shorter and better. But something does not feel right. In this case connecting to redis is simple because we rely on defaults, but it may require specifying host, port and db so our code my slightly grow. Apart from that, key name is somewhat hardcoded. So each time we'd like to 'install' this piece of code in another view we'd have to duplicate everything. It's a job for decorator!

def gimme_metric(f):
    @wraps(f)
    def _gimme_metric(*args, **kwargs):
        response = f(*args, **kwargs)
        r = StrictRedis()
        r.incr(f.__name__)
        return response
    return _gimme_metric


@gimme_metric
def my_view(request):
    return AwesomeHttpResponse()


@gimme_metric
def my_another_view(request):
    return YetAnotherHttpResponse()

That feels great. Not only it's reusable, allows us to switch everything on and off - it looks pythonic (it means we're getting there). But still, something is missing. What if I'd like to use this code in 'regular' class or function, not as a decorator. We need something better:

class metrics(object):

    def __init__(self, key=None, auto=True):
        self.key = key
        self.auto = auto

    def setup_backend(self):
        return StrictRedis()

    def bump_metric(self, key, amount=1):
        backend = self.setup_backend()
        backend.incr(name=key, amount=amount)

    def __enter__(self):
        if self.auto and not self.key:
            raise MetricException(u"Gimme key!")

        elif self.auto:
            self.bump_metric(key=self.key)

        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_val:
            logging.error(exc_val)

    def __call__(self, f):
        @wraps(f)
        def wrapped(*args, **kwargs):
            response = f(*args, **kwargs)
            self.bump_metric(f.__name__)
            return response
        return wrapped

Uff, that's pretty long. But boy, it was worth it. As I believe Raymond Hettinger said - context managers are one of the most underused features of Python, which is strange since they're a good choice when it comes to implementing acquire & release pattern (here it's not that obvious, but we could move redis connection to enter, and possibly provide better exception handing, nevertheless context manager is a protocol which we can freely use).

We can decorate our regular views (function names will be used as keys):

@metrics()
def my_view(request):
    return AwesomeHttpResponse()


@metrics()
def my_another_view(request):
    return YetAnotherHttpResponse()

We can use custom metric around one block of code:

with metrics(key='super-metric'):
    # do something

Or let ourselves bump couple of metrics manually:

with metrics(auto=False) as m:
    m.bump_metric('Uno')
    m.bump_metric('Dos')

Last, but no least - since it's a class we can support other backends:

class memcached_metrics(metrics):

def setup_backend(self):
    return MemcachedSuperDriver()

That was a long road - and there are still places for improvements. As You can see with some tweaking we can improve quality of working code and bring it to another level. Here, we fixed race condition issue, improved reusability by rewriting component as decorator and allowed another, more raw usage from any class/function.

Cheers :)

Comments

comments powered by Disqus