This preview brings TweetSharp within striking distance of the V1 roadmap we’ve been working on for nearly a year. Since TweetSharp is designed to make application developers lives easier, we’re happy to announce that we’ve completed work on rate limiting features that will allow you to maintain control over your user’s experience when consuming their API allotment.
In addition, you can now take advantage of Twitter’s native streaming to achieve the speed of push-style updates in your client. Last but not least, for those who aren’t comfortable with fluent interfaces (or perhaps just ours), we are introducing the TwitterService class, which is an optimized API layer that is powered by TweetSharp but hides most of the details from you: no more deserialization responses or guessing when to use GZIP, TwitterService will use the most efficient configuration. If you’re currently using a library like Twitterizer and want to make the switch to TweetSharp without picking up a fluent style, now you can. Right off the bat, TwitterService covers 100% of the Twitter API.
Breaking Changes
Changing class name
Because local trends are now supported, and they differ from previous trends associated with search, the AsTrends() extension method is now called AsSearchTrends() which returns the usual TwitterSearchTrends object, and a new AsLocalTrends() will return the TwitterLocalTrends object, respectively.
Changing property types
Previously, some values we had in our model that are actually nullable due to responses received from Twitter, we treated as non-nullable. To bring our model classes closer to fidelity with Twitter’s response elements we have made the following class properties nullable: TwitterStatus (InReplyToUserId, InReplyToStatusId), and TwitterUser (IsProtected).
New Features
Parsing Helpers
We added new read-only properties TextHtml, TextMentions, TextLinks, and TextHashtags on model classes TwitterStatus, TwitterDirectMessage, and TwitterSearchStatus to make it easy to parse Twitter artifacts for application customization. This means that you don’t have to write code to parse @mentions, #hashtags, or URLs in tweets. TextHtml will produce a HTMLified text with links back to Twitter, and if you need more control, TextMentions, TextHashtags, and TextLinks will return strings or Uris, respectively, to allow you to present them how you like.
Twitter Streaming API
If you are interested in using Twitter’s streaming API, your first stop should be Twitter’s API wiki, so that you fully understand the permissions and limitations of each streaming endpoint. In TweetSharp, streaming is supported through the asynchronous API, and is a little more strict with input parameters in order to do its best to obey the rules of the streaming API and therefore not get you banned. While we intend to put out more documentation this month on streaming as part of the V1 roadmap, you can get started with streaming now by following this example that targets the Filter stream, the most feature-packed endpoint.
var twitter = FluentTwitter.CreateRequest()
.AuthenticateAs(TWITTER_USERNAME, TWITTER_PASSWORD)
.Configuration.TimeoutAfter(1.Minute())
.Configuration.UseAutomaticRetries(RetryOn.ConnectionClosed, 2)
.Stream().FromFilter()
.For(10.Seconds()).Take(100)
.Tracking("Twitter") // Add other filter options here...
.CallbackTo((sender, result) =>
{
var statuses = result.AsStatuses();
foreach (var status in statuses)
{
Console.WriteLine("{0}: {1}", status.User.ScreenName, status.Text);
}
});
twitter.RequestAsync();
A few key considerations for the example above:
- You must use RequestAsync() with streaming or TweetSharp will throw an exception
- You need to specify a CallbackTo expression or TweetSharp will throw an exception
- Connections close often, so using the existing retry policy set to ConnectionClosed is helpful
- You need to specify a timeout which will function as the stream reader’s ReadWrite timeout
- The optional For() extension method will close the stream after a specified time span; if it’s unspecified the stream will continue until it fails
- The optional Take() extension method will bundle streamed tweets into groups by the given number before raising a response event
- Be sure to check the API wiki for parameter ranges; TweetSharp will auto-correct these to make sure you are sending valid values
Twitter Local Trends
We now support Twitter’s recent local trends API. This API uses Yahoo’s Where On Earth (WOE) specification to enumerate and identify locations where trending data is desired. The following code example retrieves a list of all localities Twitter has trending data on.
var query = FluentTwitter.CreateRequest()
.AuthenticateAs(TWITTER_USERNAME, TWITTER_PASSWORD)
.Trends().GetAvailable();
TwitterResult result = query.Request();
IEnumerable<WhereOnEarthLocation> locations = result.AsWhereOnEarthLocations();
Once you obtain a WOE location, you can pass its ID to retrieve the trending topics for that location.
var sanFrancisco = 2487956;
var query = FluentTwitter.CreateRequest()
.AuthenticateAs(TWITTER_USERNAME, TWITTER_PASSWORD)
.Trends().ByLocation(sanFrancisco);
TwitterResult result = query.Request();
TwitterLocalTrends trends = result.AsLocalTrends();
foreach (var trend in trends.Trends)
{
Console.WriteLine(trend.Query);
}
TwitterService Optimized API
The optimized API layer is a one-method-per-API-call style interface that should be intuitive to use. Method prefixes for Intellisense discovery include List, Send, Create, Delete, Search, and Verify. Authentication, caching, retry and timeout policies all function similar to the Fluent style but exist on methods on the TwitterService class itself. Here are a few examples of using the API.
var service = new TwitterService();
service.AuthenticateAs("username", "password");
IEnumerable<TwitterStatus> mentions = service.ListTweetsMentioningMe();
IEnumerable<TwitterDirectMessage> directMessages = service.ListDirectMessagesReceived();
TwitterStatus tweet = service.SendTweet("The new optimized API rocks!");
There are many overloads for methods to allow passing additional options like paging, cursors, and geo locations. You’ll notice there’s no deserialization to work out here, all the results are provided in the method return. If you want to use the streaming API, you can hook up an event handler like this:
var service = new TwitterService();
service.AuthenticateAs("username", "password");
service.StreamResult += service_StreamResult; // Your callback
service.StreamSample(10.Minutes()); // This is asynchronous
We really hope this API is useful for those who prefer a verbose method-style interface. Let us know if you have any suggestions for this new API and if you run into any issues with it, and stay tuned for more examples.
Rate Limiting
We now have two ways for you to dynamically throttle your recurring calls to avoid hitting the Twitter rate limit. The first is a simple percentage-based metric that will adjust the repeat rate of a recurring query so that it will always use a set percentage of the remaining calls. It’s a pretty simple algorithm, but it’s powerful in that it is constantly updating itself to take into account the current conditions of the user’s rate limit.
For example, if you set up a recurring task to pull down “mentions” and set it to use 10% of the user’s rate limit, it will initially calculate:
Rate Limit : 150 calls /hr
x Desired percent : 10%
= 15 calls / hr or 1 call every 4 minutes.
Each time the query runs, it looks at the rate limit status that it gets back from Twitter and re-applies that formula, taking into account the remaining rate limit, and the time until the next limit reset.
Rate Limit remaining: 50 calls
x Desired percent : 10%
Time until reset : 10 minutes
= 5 calls / 10 minutes or 1 call every 2 minutes.
TweetSharp does not track the total percentage across instances, so it’s possible to specify much more or much less than 100%. If you specify a total that’s over 100%, it will have the effect of slowing down and possibly eventually stopping your queries until the next reset as the rate limit is used up. Specifying less than 100% will result in your queries iterating faster near the end of the cycle as there will be more limit available to be used up in a shorter time. Since you will likely use a mixture of recurring and ad-hoc queries, you’ll likely want to experiment a bit to find the combination that’s right for you.
Here’s a code example for setting up a recurring task to check for mentions (aka ‘replies’) with a throttling value of 20% of the total rate limit.
var twitter = FluentTwitter.CreateRequest()
.AuthenticateAs(USER, PASSWORD)
.Statuses().Mentions()
.Configuration.UseRateLimiting(20.Percent())
.RepeatEvery(5.Seconds()); // initial value, will be adjusted automatically
var count = 0;
twitter.CallbackTo((s, e) =>
{
// do stuff with the response
});
twitter.RequestAsync(); //recurring tasks must be async
For those of you who want even more control, we have included advanced rate limiting, whereby you pass TweetSharp some delegates to call to determine if a query should run. Unlike the previous example, where the rate limiting is adjusted following each call, this method calls a predicate delegate just before the query is scheduled to run (as specified by the RepeatEvery(timespan) call) which determines whether or not to proceed. The predicate is passed a RateLimitStatus object that it can use to evaluate whether or not the query should run. If the predicate returns false, the query is skipped until the next iteration, if it returns true, the query is run as scheduled.
Because the predicate is evaluated before the query is run, it cannot rely on the rate limit status that Twitter returned from the previous iteration, as many subsequent calls may have occurred in the interim. To deal with that, you must also specify another delegate to the query that returns a TwitterRateLimitStatus object. That object is then passed to the evaluation predicate.
Here’s a code example that explains it – again, we want to periodically get mentions:
// function to get current rate limit
private TwitterRateLimitStatus getRateLimit()
{
var rateLimitQuery = FluentTwitter.CreateRequest()
.AuthenticateAs(user,password)
.Account().GetRateLimitStatus().AsJson();
var response = rateLimitQuery.Request();
var limit = response.AsRateLimitStatus();
Assert.IsNotNull(limit);
return limit;
}
// create a recurring task that is skipped when
// there are fewer than 20 remaining API calls
var twitter = FluentTwitter.CreateRequest()
.AuthenticateAs(user,password)
.Statuses().Mentions()
.Configuration.UseRateLimiting(rateLimit => rateLimit.RemainingHits >= 20, getRateLimit)
.RepeatEvery(30.Seconds()); // predicate is evaluated this often
twitter.CallbackTo((s, r) =>
{
if (!r.SkippedDueToRateLimiting)
{
//do something meaningful
}
});
twitter.RequestAsync();
The above example makes a call to Twitter to get the current rate limit before each iteration of the repeating tasks. Obviously, if you have lots of recurring tasks, or even just a few that iterate frequently, you probably want to provide a better mechanism for getting the rate limit that doesn’t require a network trip every time. Remember that recurring tasks run on background threads, so if you’re sharing rate limit state amongst multiple instances, you’ll need to take thread safety and concurrency into account.
