Using Twitter API with Python

My brain never rests. One of the ways I relax is finding solutions to problems. Sometimes thats with crosswords or old episodes of Poirot on ITV3, sometimes thats with writing code. I particularly like using Python.

I use Twitter all the time, for finding out all the latest news and political gossip in astronomy. So, I thought I'd have a bash at using Twitter through python. You used to be able to access Twitter just by using your account username and password, with their basic auth in Twitter API v.1.0. Not any longer. Version 1.0 is now defunct and to do interesting things with Twitter API you have to use version 1.1 which requires OAuth.

This can be a whole world of pain, so I'm writing my findings here to help anyone experiencing similar discomfort.

1. Find a Twitter library


Various people have written libraries to access twitter (I tried writing my own and it was a thankless task). I would recommend using the simply named twitter package. You'll also need a python way of logging in via OAuth, so I would recommend using OAuth2.

I've changed my mind on which library to use. I'm now using TwitterAPI, which seems more flexible for my needs, has good documentation, examples and regular updates.

It is on PyPi so you should be able use:

pip install twitterapi

2. Get developer credentials


You'll need a Twitter account to go much further with this tutorial. Once you have that:

  1. Log into the Twitter developer site,
  2. Create a new app by clicking on your icon in the top left
  3. Click "My applications" in the menu,
  4. Click the button "Create a new application"
  5. Fill out all the details, sign your life away and click "Create you Twitter application"

You should now be redirected to a page with all the information about your new app, including "OAuth settings". This page has the 4 horsemen of the OAuth apocalypse:

3. Put it together


You are now ready to have a play with Twitter from python. Open up a python shell:

➜  ~ ipython
Python 2.7.1 |CUSTOM| (r271:86832, Dec 3 2010, 15:56:20)
Type "copyright", "credits" or "license" for more information. IPython 0.13.2 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details. In [1]: from TwitterAPI import TwitterAPI
In [2]: t = TwitterAPI(
CONSUMER_KEY, CONSUMER_SECRET, OAUTH_TOKEN, OAUTH_SECRET
)

Substitute the 4 horsemen into those capitalised variables and hopefully that won't give you any trouble.

Now you can take Twitter for a spin. There are loads of interesting things you can do listed on the Twitter REST API v1.1 help pages.

I really wanted to have a look at various #hashtags without being rate limited. The way you do this is by using the Twitter Stream and not the Twitter Search. This lets you enter hashtags, words or phrases and then sit on the stream of tweets happening which contain the words you are tracking.

from TwitterAPI import TwitterAPI
api = TwitterAPI(TWITTER_CONSUMER_KEY, TWITTER_CONSUMER_SECRET,TWITTER_ACCESS_TOKEN_KEY, TWITTER_ACCESS_TOKEN_SECRET)
ts = api.request('statuses/filter', {'track': 'holiday'})
stop = datetime(2014,11,1,9,50,0)
for item in ts:
print item['user']['screen_name'], datetime.strptime(item['created_at'],'%a %b %d %H:%M:%S +0000 %Y'), item['text']
if datetime.now() > stop:
print datetime.now().isoformat()
break

Once you open a stream, tracking certain words, it will carry on forever. For this reason I stop the for loop which checks the stream with a timeout, in the variable stop (a datetime object).

Tip: You can't look backwards in time in a Twitter stream, only forwards in real-time. If you want to look back in the Twitter feeds you'll have to use search.

Thats my recent findings. There are so many more things you can do with the Twitter API, but that is just a taste.