Kyle Jensen

Solo IP stimulus

Thank you Solo for your visionary false marking and the ensuing litigation stimulus plan.  No firm left behind despite the bum economy.

Filed under  //   curiousities   ip  
Posted August 8, 2010
// 0 Comments

A decorator for detecting memory leaks in python

I thought I had a memory leak in a program I was writing for my wife today and found this awesome answer on StackOverflow. I turned it into a decorator to I could detect memory leaks more easily in the future. The code for the decorator is below.

def report_memory_leaked(f):
       """ Counts objects allocated in memory before and after the
           decorated function and reports the results.  If you decorate
           a function like
           @report_memory_leaked
           def myfunc():
              pass # do stuff

           It will report objects by type that were leaked or created.
       """
       from collections import defaultdict
       from gc import get_objects
       import logging

       def obj_counting_wrapper(*args, **kwargs):

           before=defaultdict(int)
           for i in get_objects():before[type(i)]+=1

           retval = f(*args, **kwargs)

           after=defaultdict(int)
           for i in get_objects():after[type(i)]+=1

           leaked_or_returned = [(k,after[k]-before[k]) for k in after if after[k]-before[k]]
           if leaked_or_returned:
               logging.debug('Leaked or returned:\n%r' % leaked_or_returned)

           return retval
       return obj_counting_wrapper

Filed under  //   python   software  
Posted July 29, 2010
// 0 Comments

Send email via Postmark from Appengine

I’m trying out Postmarkapp in order to improve my email deliverability for a business I own. The API is dead simple and it only took about an hour to get integrated into my existing code. Below is a simple Python class that is a subclass of GAE’s EmailMessage class and allows you to send mail via Postmark instead of GAE’s built-in mail service.

Obviously, you need your own API key from Postmark. But, using this class you could easily just make a few changes to your existing appengine code and call, e.g. msg.pmsend() instead of msg.send() assuming you’re already using the EmailMessage class to build your outgoing emails. Oh…there’s no error handling yet obviously. Caveat emptor.

from google.appengine.api import mail
from google.appengine.api import urlfetch

try:
    import json
except ImportError:
    try:
        import simplejson as json
    except ImportError:
        from django.utils import simplejson as json


class PMEmailMessage(mail.EmailMessage):
    """ PMEmailMessage subclass that enables
                    you to call the pmsend()
        method in order to send via Postmark
                    instead of using GAE's
        built-in mail service.
    """

    PM_API_KEY = 'PUTYOURKEYHERE'
    PM_URL = 'http://api.postmarkapp.com/email'

    @classmethod
    def get_pm_headers(cls, test=False):
        """docstring for get_pm_headers"""
        d = {
            'Accept' : 'application/json',
            "Content-Type" : "application/json",
        }
        if test:
            k = 'POSTMARK_API_TEST'
        else:
            k = cls.PM_API_KEY

        d['X-Postmark-Server-Token'] = k
        return d

    @property
    def postmark_dict(self):
        """docstring for postmark_dict"""

        param_mapping = {
            'Headers' : 'headers',
            'ReplyTo' : 'reply_to',
            'Tag' : 'tag',
            'To' : 'to',
            'From' : 'sender',
            'Cc' : 'cc',
            'Bcc' : 'bcc',
            'Subject' : 'subject',
            'HtmlBody' : 'html',
            'TextBody' : 'body',
        }
        d = {}
        for k,v in param_mapping.iteritems():
            val = getattr(self, v, None)
            if val:
                d[k] = val

        return d

    @property
    def postmark_json(self):
        """docstring for postmark_json"""
        return json.dumps(self.postmark_dict)

    def pmsend(self):
        """docstring for pmsend"""
        response = urlfetch.fetch(self.PM_URL,
                               method=urlfetch.POST,
                               payload=self.postmark_json,
                               headers=self.get_pm_headers())
        return response

Filed under  //   appengine   email   python   software  
Posted July 28, 2010
// 2 Comments

Multiple field full-text search using elasticsearch

Elasticsearch is a Java application and set of shell scripts that makes it nearly trivial to add full-text search to your web application. It's powered by Apache Lucene, but its got a significantly lower barrier-to-entry as you can see from the intro slides.

I could extoll numerous virtues of Elasticsearch, but there's one I think is particularly cool: you get a full query syntax language out of the box. So, for example, you can allow your users to do searches of your documents like "+monkey +ape -gorilla" or "cheese AND crackers OR pickles" without having to parse the query yourself. However, I could not figure out how to do multiple field full-text search using elasticsearch until some helpful soul in the #elasticsearch IRC channel told me: you need to wrap it in a boolean query. So, if I have documents with "abstract" and "title" fields and a corresponding web form into which a user submits 'Kyle AND "body odor"' and 'heat OR wave' (respectively) I'd translate that into the elasticsearch JSON search syntax as follows:

That's it. Super cool. Super easy. Using elasticsearch's boolean query you could designed monstrous "advanced search" forms in a flash.

Filed under  //   search   software  
Posted July 26, 2010
// 2 Comments

Increase email deliverability on Appengine by setting the SPF record

Email deliverability is a a huge problem. I'm a big fan of Google Appengine and send a fair amount of email from it. I noticed recently that some of it was being caught, particularly by the Postini spam filter on the most aggressive setting. After investigating, I decided to set up the SPF DNS record for the domains sending email from Appengine. But, there's a catch, answered by a clever fellow in this thread. The correct DNS TXT entry is

v=spf1 include:aspmx.googlemail.com ~all

not

v=spf1 include:_spf.google.com ~all

which is suggested in this google answer. However, this other answer on configuring SPF with GoDaddy is quite useful.

To verify your SPF record is set up correctly, use the following command to inspect your DNS TXT entries:

dig TXT your-domain-here.com

You can check if your changes worked using these SPF checkers.

Filed under  //   appengine  
Posted July 23, 2010
// 7 Comments