Targeting Your Twitter Bot

By Posted in - Development & Technology on October 15th, 2013 7 Comments Robot

Is this your first time to the site? If so, you may want to check out the first part of this series on Twitter bots.

Focusing Your Twitter Bot

Twitter’s Rules

In the last post we got a simple Twitter bot up and running with the minimum amount of work possible. Today we’re going to add some focus to our Twitter bot. We’re going to take a deeper look at the Twitter API, do some refactoring, and add some more behaviors to our bot.

First off, it would you probably want to go through Twitter’s rules. Twitter does not outright ban bots, in fact there are many out there (see some examples). Twitter does however do what it can to prevent spam. This forces them in to a balancing act: don’t ban all bots outright, but suspend accounts that show behavior indicative of a spam bot.

What Twitter Looks For

Our first bot did three things: retweets, follows, and unfollows. If you set the action interval too low, you might have had your account suspended, and you likely were informed by Twitter that you were practicing “aggressive following”. Since they don’t provide a hard limit (that I’ve seen) to how many people you can follow in a day, they can pretty much define “aggressive following” anyway they want. Same for any of the behaviors they frown upon in their list of rules.

I don’t have some “hack” for never having your bot’s account suspended. The only advice I can give you for now is to be conservative with the frequency with which you perform actions. Create a handful of different bots and try different configurations. See what Twitter allows and what they don’t by experiment.

Diving into the API

Here’s an overview of all of the possible API calls we can make to Twitter – https://dev.twitter.com/docs/api/1.1. The API pretty much covers anything you can manually do as a Twitter user – no functionality is left out. It’s worth taking the time to dig around in the API documentation as it can give you some ideas for you what you might want your bot to do.

API Calls vs. Bot Behaviors

A lot of potential Twitter bot behaviors will consist of making multiple API calls. Our “mingle” behavior, for example, consists of 3 API calls:

  1. ‘followers/ids': this retrieves the ids of all of the bot’s followers.
  2. ‘friends/ids': this retrieves the ids of all of the followers of a supplied user
  3. ‘friendships/create': this causes the bot to follow a given user

Let’s look at the code in a bit more details:

Bot.prototype.mingle = function (callback) {
  var self = this;

  this.twit.get('followers/ids', function(err, reply) {
    if(err) { return callback(err); }

    var followers = reply.ids
    , randFollower = randIndex(followers);

    self.twit.get('friends/ids', { user_id: randFollower }, function(err, reply) {
      if(err) { return callback(err); }

      var friends = reply.ids
      , target = randIndex(friends);

      self.twit.post('friendships/create', { id: target }, callback);
    })
  })
};

The bot module has a function called mingle. It starts out by retrieving all of the bot’s followers. Then it uses the randIndex helper function to select a random follower. It makes another API call and retrieves all of the followers of the randomly selected user. When the API call finishes, it again uses the randIndex helper to select one of those random followers. Finally, it makes the last API call and follows the randomly selected user.

Future behaviors will follow the same pattern. We’ll add a new function to the bot module, and that new function will consist of one or more Twitter API calls in nested callbacks.

What Behaviors?

What behaviors you add depend entirely on what you want your bot to do. The first bot I made got a lot of followers relatively quickly, but they were scattered across the world and had next to nothing in common. I wanted to see if it was possible to gain followers in a specific niche, so one of the first behaviors I added was a targeted follow. We’ll add that next (though I encourage you to experiment with creating different kinds of behaviors).

Targeted Following

Below is the function we will add to the bot module:

Bot.prototype.searchFollow = function (params, callback) {
  var self = this;

  self.twit.get('search/tweets', params, function (err, reply) {
    if(err) return callback(err);

    var tweets = reply.statuses;
    var target = randIndex(tweets).user.id_str;

    self.twit.post('friendships/create', { id: target }, callback);
  });
};

As you can see, it follows a pattern roughly similar to the mingle function. In this case, the first API call – search – requires some search parameters, which we’ll provide from rtd2.js when we call the function. The search API call is made. We then retrieve the results, then randomly select one of those tweets and pull the corresponding ID string from the user. We then simply follow the randomly selected user.

With this behavior, we can now select a topic and follow people who happen to be tweeting about it. So, say we were interested in following people who are interested in sandwiches. We now do a search for anyone tweeting about sandwiches, and then randomly select one of those people. Blending targeted follows with mingling will lead to a focused yet somewhat natural pattern of following and, hopefully, being followed. You can probably think of ways of improving following behavior even further.

A Few Bonus Behaviors

Here are a few more simple behaviors you can add to your bot. They’re pretty self-explanatory.

//
// retweet
//
Bot.prototype.retweet = function (params, callback) {
  var self = this;

  self.twit.get('search/tweets', params, function (err, reply) {
    if(err) return callback(err);

    var tweets = reply.statuses;
    var randomTweet = randIndex(tweets);

    self.twit.post('statuses/retweet/:id', { id: randomTweet.id_str }, callback);
  });
};

//
// favorite a tweet
//
Bot.prototype.favorite = function (params, callback) {
  var self = this;

  self.twit.get('search/tweets', params, function (err, reply) {
    if(err) return callback(err);

    var tweets = reply.statuses;
    var randomTweet = randIndex(tweets);

    self.twit.post('favorites/create', { id: randomTweet.id_str }, callback);
  });
};

Putting it Together

Let’s actually add the new searchFollow behavior to rtd2.js now. The code here is a bit messy. We can’t complain, of course, because we’re taking what was meant to be a simple example and turning it into a full-fledged bot. We could really clean things up, but for now let’s make a few simple changes. First, I’m not a big fan of the “tweet a popular github tweet” section. It’s not as modular as the rest of the code. Additionally, when I run it, that section always seems to give me a 404. Rather than debugging it, let’s just get rid of it.

Our next step is to add our searchFollow behavior (and optionally our retweet and favorite behavior). You can put it anywhere in the if-else-if chain, you just need to make sure you change the random number comparisons to something that makes sense. Here’s an example:

//
// RTD2 - Twitter bot that tweets about the most popular github.com news
// Also makes new friends and prunes its followings.
//
var Bot = require("./bot")
, config1 = require("../config1");

var bot = new Bot(config1);

console.log('RTD2: Running.');

//get date string for today's date (e.g. 2011-01-01)
function datestring () {
  var d = new Date(Date.now() - 5*60*60*1000); //est timezone
  return d.getUTCFullYear() + "-" + 
         (d.getUTCMonth() + 1) + "-" + 
         d.getDate();
};

setInterval(function() {
  bot.twit.get("followers/ids", function(err, reply) {
    if(err) return handleError(err)
    console.log("\n# followers:" + reply.ids.length.toString());
  });
  var rand = Math.random();

  if(rand <= 0.55) { //  make a friend
    bot.mingle(function(err, reply) {
      if(err) return handleError(err);

      var name = reply.screen_name;
      console.log("\nMingle: followed @" + name);
    });
  } else if(rand <= .85) {  // do a targeted follow
    var params = {
        q: "sandwiches"
      , since: datestring()
      , result_type: "mixed"
    };

    bot.searchFollow(params, function(err, reply) {
      if(err) return handleError(err);

      var name = reply.screen_name;
      console.log("\nSearchFollow: followed @" + name);
    });
  } else if(rand <= .90) {  // retweet
    var params = {
        q: "sandwiches"
      , since: datestring()
      , result_type: "mixed"
    };

    bot.retweet(params, function(err, reply) {
      if(err) return handleError(err);

      console.log("\nRetweet: retweeted response: " + reply.id);
    });
  } else if(rand <= .95) {  // favorite
    var params = {
        q: "sandwiches"
      , since: datestring()
      , result_type: "mixed"
    };

    bot.favorite(params, function(err, reply) {
      if(err) return handleError(err);

      console.log("\nFavorite: favorited response: " + reply.id);
    });
  } else {                  //  prune a friend
    bot.prune(function(err, reply) {
      if(err) return handleError(err);

      var name = reply.screen_name
      console.log("\nPrune: unfollowed @"+ name);
    });
  }
}, 4000);

function handleError(err) {
  console.error("response status:", err.statusCode);
  console.error("data:", err.data);
}

Notice I added the targeted follow to the middle. The first if-statement has a 55% chance of occurring, the second has a 30% chance, and the third, fourth, and fifth each have a 5% chance. These are numbers you will constantly be fiddling with the get the best result. Notice I have the interval set to 4000 milliseconds. If I let this bot run for even a short while, I’ll start getting errors saying I’ve exceeded my API call limit. I had it set low for testing, but when I’m letting it run the interval will obviously be considerably higher.

Fine-tuning Search

You’ll definitely want to look at all of the search options at https://dev.twitter.com/docs/api/1.1/get/search/tweets. In our example, the q refers to the actual query, and since refers to a time frame, which in this case is anything within the last 7 days. result_type let’s you specify if you want recent results, popular results, or a mix of the two. You can also filter search results by language and geography (by specifying latitude, longitude, and radius). Finally, all kinds of advanced searches are possible within the query field itself.

Conclusion

We’ve only added a few relatively simple behaviors, but you can already see how dramatically it will impact the bot’s behavior. There are a number of things we haven’t even come close to covering yet: the streaming API, managing multiple bots at a time, refactoring the code so as to make bots easier to configure, etc. What do you want me to cover next?

Stay Up to Date

If you have enjoyed the tutorials so far and want to be notified as soon as another one comes out, please sign up to be notified. I promise you that I won’t spam you.



(7) awesome folk have had something to say...

  • Maxim - Reply

    November 28, 2013 at 10:27 am

    I just don’t know why you don’t have 1000 comments on this page. I found this information very useful and I was searching for it quite a lot. You’re doing great job man) Thank you!

    I’m very interested in twitter api and twitter bots. Maybe you can tell a little how to make this bot reply to messages from other twitter users with 20-30 phrases and post random popular posts? This’ll be great !

  • Chet - Reply

    November 30, 2013 at 3:53 pm

    I’d like to start getting my bot to make text updates. I’ve been looking through the API files, but I want to make a bot that just makes innuendos and stupid dad jokes all day. How can I populate random text and get it to reply to status’s involving balls and stuff?

    PS.: Yes, I’m 12.

  • James S - Reply

    December 5, 2013 at 11:43 am

    Boom. another fantastic tutorial. streaming api, yes please! (& maybe at some point how to get these out of my local machine onto something like heroku etc). Loving this thank you so much.

    • Christian Paulsen - Reply

      December 5, 2013 at 7:13 pm

      Thanks! I’m glad it’s useful for you. And thanks for the feedback, streaming API and deploying to Heroku would be a great 3rd tutorial.

  • Neerav - Reply

    December 6, 2013 at 5:28 pm

    Thank you for 2 awesome tutorials! I’m trying to create a bot which searches for some keywords, randomises the search results array and then retweets these random tweets once every 10 minutes . But I’m not able to do this as I’m unable to figure out how to make the code “wait” for 10 minutes between successive retweets..

    I tried searching and posting the 1st search result, and repeat this indefinitely every 10 minutes. But it gives me the 403 error. Is it because I’m generating so many search results every 10 minutes?

    Hence, I intend to search once every 1 or 2 hours , but retweet search results 1 by 1 every 5-10 minutes..

  • Maximus - Reply

    December 9, 2013 at 2:35 am

    Fantastic tutorial, Christian! This is super clear and useful – like Maxim says, I dont know why it has not blown up yet. Streaming API would be a great thing to cover, as well as how to engage with other uses.

    Heroku deployment is something else that would be fantastic :)

  • tr1p - Reply

    December 22, 2013 at 5:13 pm

    great script!
    how much interval did you set in real? i have 100000 miliseconds…its too much or not? thx!

Please leave a Comment