Search Twitter Public Timeline with Python

Thursday, April 15 2010 10:13 p.m.

This Python function takes a search string as a parameter and returns a dictionary representing the JSON returned by Twitter with your search results from the public timeline. Code requires simplejson. It supports all the parameters as defined by Twitter here.

def search_public_timeline(q, refresh_url=None, **kwargs):
    '''
    Searches the public timeline for the q string. There is no sanity checking
    of the parameters. It's all passed straight to the API. Spec defined here:
    
    http://apiwiki.twitter.com/Twitter-Search-API-Method:+search
    
    If you send refresh_url then all parameters are ignored. Example usage:
    
    search_public_timeline('menendez')
    search_public_timeline('menendez', since='2010-04-15')
    
    Ed Menendez - ed@menendez.com
    
    >>> len(search_public_timeline('the')['results']) > 1
    True
    '''
    if not refresh_url:
        parms = {}
        parms['q'] = q
        parms.update(kwargs)
        query_str = '?%s' % urllib.urlencode(parms)
    else:
        query_str = refresh_url
    
    u = 'http://search.twitter.com/search.json%s' % query_str
    
    #user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
    #headers = {'User-Agent' : user_agent}
    
    req = urllib2.Request(u) #, None, headers
    
    return simplejson.load(urllib2.urlopen(req))

Python Syntax Highlighting

Sunday, April 4 2010 11:30 p.m.

Simple web utility that colors your python code. Useful for putting in blogs, HTML emails, etc.

See: Python Color Utility

Using Django as a Pass Through Image Proxy

Sunday, March 21 2010 8:39 p.m.

An earlier post shows you how to setup Nginx as a pass through image proxy. This post shows you how to do it with just Django and nothing else.

We've solving the same problem as the earlier post. However, I will repeat it here for clarity as there's been some confusion.

You have a production DB with lots of images uploaded by users. For example, NationalGeographic.com has over 11gb of user uploaded images. When you download a data dump of the production database, it has links to all these images which you don't have. You either have to download and sync all the images locally every time you copy the database, live with broken images or point your static images to the prod server.

Maintain Constants.py through Data Migration

Sunday, March 14 2010 7:49 p.m.

This post will show you how to create a local_contants.py file using Django South. If you're not familiar with South, it's a data migration tool. Stated differently, it's a tool that helps keep your database changes current across different sandboxes and server environments.

Efficiently Test Emails in Django

Sunday, July 19 2009 6:08 p.m.

Testing emails can be an inefficient process. The email needs to leave your machine, arrive at the SMTP server, get forwarded to your test mail server and then downloaded by your email client. Do you want those 30 seconds of your life back?

Using an SMTP sink, you can receive the email to your screen instantly. Setup is easy.

•  ♦  Download smtp_sink.py
•  ♦  Put it in whatever your project directory is. You will want to create a "inbox" directory. Besides outputting to the screen, this will automatically copy the email to the inbox directory in case you need to reference it later
•  ♦  Change your settings.py to have: EMAIL_HOST = '127.0.0.1', EMAIL_HOST_USER = '', EMAIL_PORT = 25

If you have a script that loads your sandbox environment automatically, you will want to load smtp_sink.py automatically. It will send a test signal to see if it's already running and not start twice.

If you run automated test scripts, the output of the "inbox" can be tested.

AvaTint Launched!

Tuesday, June 30 2009 11 p.m.

The site very simply takes your avatar and turns it into any color you select. The online community is starting to do this to support and protest many things. This is the 2.0 way of making your page background black for a cause.

We created this site in 90 minutes using Django on June 19th.

Easily Code Templates for iPhone in Django

Monday, May 4 2009 9:49 p.m.

Sometimes, you want to create a custom page for an iPhone or other small phone browser but don't want a custom URL or view for it. This can easily be handled by using the mini_render_to_response function.

Celebrity Fantasy Game in Django

Monday, August 18 2008 9:53 p.m.

If you've played fantasy football or any fantasy sport you know what I'm takling about. If you're new to the concept, you pick a roster of celebrities and if they show up in the news, you get points.

The site was written using Django (trunk as of the 13th!) and Postgres 8.2. We've built a fantasy games package on top of the Django core which we would probably open-source if the community showed some interest in it.

This is our 2nd major site in Django and we really love it. To begin with, Python is a great language. When we built our fantasy engine, it was originally coded on MySQL. We encountered quite a few problems porting it over to Postgres. There are lots of little things where MySQL does type conversions for you automatically and Postgres (correctly in my opinion) does not. I definitely feel the conversion was worth it so that now we're on an enterprise level database.

MySQL Connection Pooling with Django and SQLAlchemy

Friday, July 25 2008 12:41 p.m.

Here's a quick and dirty recipe to get connection caching from SQLAlchemy. This is really not connection pooling as that would require a separate process to only handle connections. All this does is prevent the connection from closing after you finish a query. We're also not replacing the Django ORM.

To give a little more background, normally when you use Django to get to your database, Django will automatically disconnect from the database when the thread is done with that query.

Launching a High Performance Django Site

Sunday, July 6 2008 3:29 p.m.

Are the brakes on your Django app?

When building an application using an application framework like Django... the priority is often to get the application working first and optimize it later. The trade off is between getting it done and getting it done for 1 million users. Here's a check list of things you can do to make sure your application can be optimized quickly when you put on your optimization hat. Note, most applications don't need all of this since most applications do not get anywhere near enough traffic to justify even bothering. But if you're lucky enough to need to optimize your Django app, I hope this post can help you.

Python Timezone Conversion Example Using pytz

Tuesday, May 13 2008 12:11 a.m.

Here's an example showing how to convert from one timezone to another in Python. Note that you won't be able to save the full date/time in MySQL without stripping out the timezone info.