Introduction

This guides looks at alternative API solutions for developers impacted by the recently announced request limit changes to /statuses/mentions_timeline and /statuses/user_timeline.

What’s changing?

On June 19, 2019, we will begin limiting total GET requests to the v1.1 /statuses/mentions_timeline and /statuses/user_timeline endpoints to 100,000 requests per day. This is a total request limit (per endpoint) applied across both user-auth and app-auth requests. This means that in a 24-hour period, a single app can make up to 100,000 requests to /statuses/mentions_timeline and/or 100,000 requests to /statuses/user_timeline (with either app or user auth) before hitting this new app-level rate limit. The existing default user-auth and app-auth rate limits will not change.

Am I impacted?

Before diving into this guide, it’s worth confirming that you will indeed be impacted by the forthcoming changes. Please review the criteria below to check if you will be impacted by this change:

• A single client app exceeds 100,000 requests per day (24-hour period) to the /mentions_timeline endpoint.
• A single client app exceeds 100,000 requests per day (24-hour period) to the /user_timeline endpoint.

If either condition above is true, and you want to migrate your usage of the mentions / user timeline endpoints to an enterprise API alternative, please continue reading through this guide.
 

Alternative APIs

The following enterprise API solutions can be functional replacements for the most common use cases of the /mentions_timeline and /user_timeline endpoints endpoints. While offering methods for getting “mentions” and Tweets from users, they do differ in terms of authentication, delivery mechanism (streaming, webhooks), and data availability depending on the API. Follow the links below to explore the API documentation directly.

1. Account Activity API (webhooks) – delivers both mentions and Tweets posted by the authenticating user
2. PowerTrack API (streaming) – supports the ability to stream mentions (@ operator) and/or Tweets posted by any public user (from: operator) as well as many other filter operators.
3. Enterprise Search API (REST) – supports the ability to query for mentions (@ operator) and/or Tweets posted by any public user (from: operator) as well as several other filter operators. Depending on your historical data needs, there are two options to choose from:
   • 30-Day Search (past 30 days)
   • Full-Archive Search (complete access back to March 2006)

Choosing an API

To help evaluate which API is right for you, please see the comparison table below that outlines the technical characteristics, unique features, and limitations of each API.

*The /mentions_timeline and /user_timeline endpoints are listed for easy comparison.

Which API is right for me?

Based on the comparison table above, you should have a better idea of which API(s) will better serve your needs and use case. If you’re still not sure, below are some additional aspects to consider:

Enterprise Search API

The enterprise Search APIs offer low-latency, reliable, query-based access to publicly available Tweets ~30 seconds after they are published on the platform. We offer two different enterprise Search options:

1. 30-Day Search: provides historical Tweets from the past 30 days.
2. Full-Archive Search: provides complete access to the full corpus of historical Tweets dating back to the first Tweet in March 2006.

These REST APIs support complex boolean queries of up to 2,048 characters using the 30 different filter operators available – including the @ operator to capture mentions and the “from:” operator to capture Tweets from specific users.

Use cases: Retrieve historical brand mentions, perform analysis (for example, how many Tweets mentioned a brand this year vs last year?), or get recent Tweets related to an event, topic, or trend.
 

Key characteristics

• Historical access: retrieve past mentions, Tweets from users, or search on keywords, hashtags, and other attributes within the past 30 days or the full archive.
• Equivalent functionality:
  • /mentions_timeline – “@” operator retrieves all mentions of an account.
  • /user_timeline – “from:” operator retrieves all Tweets from the specified user.
• Infrastructure: minimal infrastructure changes will be required to migrate to the Search APIs as they provide a similar REST interface. The request structure will change slightly and the response payload may be larger as the ability to trim response objects goes away.
• Limitations:
  • Date range: The mentions / user timeline endpoints were limited by a static (max) number of Tweets returned; whereas, the Search API will return all Tweets matching your query from within the specified time period (minute granularity available).
  • Public Tweets: the Search API only returns Tweets published by public users. The mentions / user timeline endpoints can return Tweets from protected accounts (user token must be provided) and Tweets that were created when an account was protected, but has since switched to public (unprotected).
     

Migration considerations

Below are some key technical considerations you’ll need to account for when planning your migration:

I. Authentication

Change authentication mechanism to HTTP basic auth (see docs).

II. Request structure

Adjust code to make requests to a new endpoint with a different JSON request body and required parameters. Both GET and POST requests are supported. A new query parameter is required.

III. Response structure

The Tweet payload is largely the same, but you will need to account for the full response payload (no ability to trim user/entities) and additional metadata enrichments. You may need to adjust your parser to account for, or simply ignore, new objects in the payload.

IV. Pagination

You will need to implement new pagination logic (see docs). When there is more data than can be returned in a single response, a “next” token will be present in the response that is needed to make the next request. This pattern should continue until there is no “next” token in the subsequent response(s).

Additional references:

• Learn more about enterprise search
• Quick start guides
  • 30-Day Search API
  • Full-Archive Search API
• Available operators (search filters)
• API reference
• See an example client library (python)
   

PowerTrack API

PowerTrack provides filtered access to the full stream of publicly available Tweets in realtime. With over 50 different operators—including the @ mentions and from: operators—customers can ingest a targeted stream of only the data they care about.

Use cases: listen for brand mentions in realtime, monitor topics with complex, boolean filters, or detect emerging trends and anomalies to keep your customers informed.

Key characteristics

• Realtime: delivers all Tweets to your stream that match your rules (filters) in realtime. Tweets are delivered seconds after they are published.
  • This is a significant change in the data delivery mechanism to an always-on, streaming service, rather than a pull-based, paginating endpoint.
• Equivalent functionality:
  • /mentions_timeline – “@” operator retrieves all mentions of an account.
  • /user_timeline – “from:” operator retrieves all Tweets from the specified user.
• Infrastructure: You will need to create a streaming client/infrastructure that can handle potentially high volumes of inbound data. A common architecture will include a “stream reader” that reads inbound data off of the stream and loads it into a queue for further processing (parsing, storing, etc). For more details, please see the Consuming streaming data tutorial.
• Limitations: PowerTrack will only deliver Tweets that match your specified rules. It is up to you to create boolean rules using the available operators to match the Tweets that you want to receive through the stream.
  • Rate of inbound Tweets: the API does not offer a setting to control or limit the rate at which matching Tweets are delivered to your stream. In the event that your rule(s) matches a large volume of Tweets, they will be sent down the stream accordingly. It is important that your client code can handle this.
  • Public Tweets: PowerTrack only returns Tweets published by public users. The mentions / user timeline endpoints can return Tweets from protected accounts (user token must be provided) and Tweets that were created when an account was protected, but has since switched to public (unprotected).

Migration considerations

Below are some key technical considerations you’ll need to account for when planning your migration:

I. Authentication

Change authentication mechanism to HTTP basic auth (see docs).

II. Streaming infrastructure

You will need to create a streaming client/infrastructure to work with PowerTrack. This client will need to connect to and sustain an http streaming connection so that data can flow properly. Tweets that match your rule(s) are delivered in realtime, so the client will need to read potentially high volumes of inbound data and be flexible to bursts of data. It is recommended to have code that reads data off of the stream and loads it into a queueing system to separate this from the parsing, processing, and storing of data.

III. Rule management

PowerTrack provides a separate “rules” endpoint where you add and remove filters for your stream. This is managed independent of the stream endpoint, so that you can add / remove filters while maintaining your streaming connection.

IV. Request structure

To receive data, you must first create a rule (POST request to /rules endpoint) on your stream (for example, from:twitterdev). Then, connect to the stream (GET request to /stream endpoint) and you will start seeing Tweet payload flow through as they are published in realtime.

V. Response structure

The Tweet payload is largely the same, but you will need to account for the full response payload (no ability to trim user/entities) and additional metadata enrichments. You may need to adjust your parser to account for, or simply ignore, new objects in the payload.

Additional resources:

• Available operators
• Rules and filtering
• Consuming streaming data
• API reference

Account Activity API

The Account Activity API delivers activities related to an account in real-time, including “mentions”. Tweets from authenticated users, and all activity related to an account such as Tweets, replies, likes, follows, direct messages, and more.

Use cases: power customer service interactions, build chatbot experiences, or monitor account activity for brands that you manage on Twitter.

Key characteristics

Realtime: delivers all activity related to a subscribed account, including “mentions”. Activities will be delivered (“posted”) to your registered webhook almost instantaneously.

• It will substantially change the request pattern to a push-based service, rather than a pull-based, paginating endpoint.

Equivalent functionality:

• /mentions_timeline – “mention” events of subscribed user (pushed to webhook in realtime)
• /user_timeline – “Tweet” events from subscribed user (pushed to webhook in realtime)

Infrastructure: you will need to create a web app to serve as your “webhook” for Twitter to post events to. You will only receive events related to accounts (users) that are subscribed to your webhook.

Limitations: as is the case with /statuses/mentions_timeline, you must have a user token (user context) to retrieve mentions and other account activity.

• Important: Account Activity API delivers DMs, so your Twitter app permissions must be set to “Read, write, direct messages”. If this is not the case, you will need to change your app permissions to this setting and regenerate existing access tokens to alter permissions levels.

Migration considerations

Below are some key technical considerations you’ll need to account for when planning your migration:

I. Authentication

Unchanged for new endpoints, but requires implementation of OAuth 2.0 for others.

II. Webhook infrastructure

Developers will need to build and host their own endpoints (webhooks). This requires creating a web app to serve as the “webhook” for Twitter to deliver (POST) events to. In more detail, you will need to:

1. Create a webhook consumer app
2. Secure your webhook (CRC)
3. Register your webhook URL
4. Make sure your web app supports POST requests for incoming events

III. App permissions

Check your Twitter app permissions to ensure the “read, write, and direct messages” is set. If this is not set already, you will need to change it at the app-level, then regenerate existing access tokens to alter the permissions levels (Note: this will likely involve re-authenticating users at the new permissions level)

IV. Subscribing accounts

You will need to make a POST request (3-legged OAuth - registered consumer key and subscribing user's access token) to the Twitter subscriptions endpoint (see docs).

V. Data objects (response)

Data objects are JSON formatted “events”. There are 19 event types that flow through the API. Your HTTP service (webhook) must respond to inbound “events” within 3 seconds.

Additional resources:

• Getting started with webhooks
• Securing webhooks
• Authenticating users
• Managing webhooks and subscriptions
• Account activity data objects
• Enterprise API reference