Finally! A Dev Tracker Twitter Feed

Have you ever wanted to follow the Dev Tracker on the LOTRO Forums straight from your Twitter feed? Well, if you haven’t already seen it yet, I made just that: A twitter account (@LOTRODevTracker) that tweets out whenever a new post gets added to the Dev Tracker page.

The LOTRO Dev Tracker Twitter autotweets with the help of twitterfeed.com from a feed that I created using Yahoo! Pipes which in turn extracts new information from the Dev Tracker page. The feed is publicly available so you can do what you want with it including using it in your own RSS reader, having it post to your Facebook or building upon it yourself, mashup style.

If you want to know how I did it, read on, but if reading technical information gives you headaches or is against your religion, turn away now.

The idea for this came long, long ago, but I had no clue how to approach it. One day months later, I set out to tackle this problem, so I started by searching for a direct feed of the dev tracker for hours, but the best I found was a site feed for the entire forums. I was planning on filtering this feed by community members based on the author, but the feed only tracks new threads created. While that could be useful if I wanted to read new posts on specific topics, it didn’t capture any new replies so it wouldn’t work for this project.

Not until I found Yahoo! Pipes did it really come together for me. Pipes is a powerful tool for creating custom feeds and data from a multitude of sources. I played around with it for a while trying to mimic what I found in other peoples’ pipes. I even found previous attempts at dev trackers for LOTRO that were no longer functioning because of a deprecated module. Instead, users have to use the XPATH fetch module which calls a page and separates it into a structure based on page elements (e.g., tables, div tags). All I needed to do was find the path to each row in the table on the dev tracker page then the module would separate each entry as an item in the feed. After I excluded the header, I could easily extract the URL and title of each thread.

The big sticking point I ran into was trying to capture the author name. There was something about the handling of multiline elements that I just could not figure out. If anyone else knows how to do it, go for it, but be nice enough to explain to me how you did it. I settled on an alternate format without the author name and built the title with just the thread title and the URL.

Each item includes two parts, the title and the link to the dev post, but if someone really wants to get creative, they could pull the source from each of these posts and include it directly in the feed. That’s too advanced for me at the moment; I’ve got lots of sleep to catch up on.

All that was left was to create a new twitter account and use a service to autotweet items from this new feed. Upon multiple recommendations on the ‘Net, I settled on twitterfeed.com which has a simple interface and does everything I needed it to do. I set it up to check for new dev posts every half hour and tweet out new items. But if there are more than five new items in a thirty-minute window, the account will only tweet out the latest five. That’s just a limitation with twitterfeed.com. However, this is good news for anyone worrying about a strenuous load on the LOTRO Forums (or more correctly, fora) as the script runs only twice an hour.

Now I just have to see if it continues working; there’s quite a bit that could go wrong. The feed relies on the LOTRO Forums, Yahoo! Pipes, twitterfeed.com and Twitter to work. If any of those breaks, so does the feed. Nevertheless, it shouldn’t take much more than an adjustment here and there to fix. Here’s hoping for sustained service stability and more open community relations for the future!

If you want to see exactly how I built the feed, check out the feed source.