Tuesday, October 12, 2010


Here is a paper that details the back ground material, the program architecture and the methods employed in the project.


The author will demonstrate that text data from an Online Social Network can be tokenized, filtered, expanded with synonyms and rated with an approximated Vector State Model cosine Θ function.  The resulting value is used by an agent to rate and possibly filter subscribed content.  Additionally, the agent can monitor a public content source and identify and display  content that may be potentially interesting to the consumer.


There are several ideas mentioned at the end of the paper for possible future development.  If you are interested in using SkimmerAgent as a starting point for a project or in contributing to the application please contact me.

Monday, December 7, 2009

Pre-Alpha Code


Dropped a quick video out on YouTube showing how to run/use the current version of SkimmerAgent.  If the expand query document setting is checked, when you tab out of the query terms field a message box will display with the query terms.

Here’s some very pre-alpha bits in a zip file.  You will need to have the .Net Framework 3.5 installed on your machine.

Expand the zip file and run the SkimmerAgent.exe from directly in the folder.

If you download and give SkimmerAgent a run around the block be sure to leave a comment here or via twitter.

Sunday, November 8, 2009

Content Rating Progress

I got a lot of work done on SkimmerAgent today.  Overall, there were 33 files created and 11 altered for interface specifications, code classes, test classes and user interface functionality. 

Today’s focus was the content rating feature.  To reach the end goal a number of filters were created.  Under the hood there are filters for removing @usernames and URLs from tweets.  The tweet itself is also tokenized.  “Stop words” are now removed as part of the content rating.  I’m using the MySQL stop words list.

I implemented a simple heuristic for content rating using a hard coded goal word list.  The heuristic is fairly simple yet has proven effective.

There was also a UI concurrency issue that had to be dealt with.  I did change up the main screen to display the skimmed public timeline all the time in a splitter panel.  A toggle was added to turn off public time line skimming.  There is much that needs to be done with the UI.  However, for now the primary focus is the content rating.  I plan to address some of the UI needs in December.

Next up I will implement a content rater that takes a vector space model (VSM) cosine θ approach.

With some UI enhancements I’m thinking about a December alpha release.

screen shot showing SkimmerAgent split grid UI

Friday, October 16, 2009

Architectural Proof of Concept

Working out the POC stuff.  Do have the application using OAuth and pulling recent messages down from Twitter.  Can also tweet!

Mocking in architecture for Friendship evaluation, content rating, and overall utility calculation.

Here’s a screen capture.


Friday, October 2, 2009

What is Skimmer Agent

Skimmer Agent is a Twitter client that I am creating which will feature AI techniques to filter/highlight content and suggest friends or contacts to drop.

The motivation for this project is a way to manage the high volume of information that you must process when you follow a number of people.

So, just like in other aspects of life, what can we not bring in? What is really important? What is just junk?

The long term goal is that Skimmer Agent serves as a personal assistant in the twitterverse.