The endpoints we release in Labs will be previews and are likely to change before they are released broadly, so we encourage you to take that into consideration as you build. Before getting started, we encourage you to read more about Twitter Developer Labs.
How many times can I connect, and how many simultaneous connections can I make?
Each app is allowed one connected client at a time. You can attempt reconnecting up to 50 times per 15 minute period.
How can I check my usage and rate limits?
To check connection limits response will return three headers: x-rate-limit-limit, x-rate-limit-remaining and x-rate-limit-reset headers. This is useful to understand how many reconnections attempts are allowed for the streaming endpoint.
- x-rate-limit-limit indicates the number of allotted requests your client is allowed to make during the 15-minute window
- x-rate-limit-remaining indicates the number of requests made so far in the 15-minute window
- x-rate-limit-reset is a UNIX timestamp indicating when the 15-minute window will restart, resetting x-rate-limit-remaining to 0.
This endpoint does not currently report usage data. To check how many Tweets have been delivered, your code can implement a metering logic, so that consumption can be measured and paused if needed.
How do I consume a stream?
Realtime streams are similar to any other REST API request, except they do not close the connection when done, or when no response is available. The code samples on the “Quick start” page provide an example of how to consume a stream.
Given the potential of high volumes of Tweets delivered in a stream, it is highly recommended that incoming content is processed in an asynchronous fashion. Your code that hosts the client side of the stream simply inserts incoming Tweets into a first in, first out (FIFO) queue, or a similar memory structure; a separate process/thread should consume Tweets from that queue to parse and prepare content for storage. With this design, you can implement a service that can scale efficiently in case incoming Tweet volumes changes dramatically.
I am having problems authenticating. How can I avoid authorization errors?
The sampled stream endpoint is authenticated using OAuth 2.0 Bearer token (also known as app-only authentication). Make sure your OAuth token is correct and valid. In order to use this preview, you need to activate from the Labs dashboard. Make sure “Sampled Stream” is listed under “Your active previews”.
My stream disconnected. Why did it happen, and how can I avoid disconnections?
Your stream can disconnect for a number of reasons. Inspect the error message returned by the stream to understand the reason for the failure. Common reasons for disconnections are:
- an authentication error (such as a wrong token or a wrong authentication method being used)
- a temporary server issue, scheduled maintenance and updates
- your client is not keeping up with the volume of Tweets the stream is delivering
- your account exceeded your monthly quota of Tweets
I’m receiving an “HTTP 429 Too Many Requests” error, but I have not exceeded my rate limit yet. What does it mean and how do I avoid getting this error?
If you’re receiving this error on your stream, it may mean one of the following:
- You tried to establish more connections than allowed during the 15-minute rate limit window.
Solution: wait until the rate limit window resets. You can inspect the x-rate-limit reset to understand how much time is left in the rate limit window.
- You are already connected through another client, and the stream will not accept any additional connections.
Solution: disconnect other clients, and ensure you are connecting with only one client at the time
- The stream did not disconnect in a clean way, which means it may still look connected to Twitter. This can happen when your connection experiences packet loss, or when your service can’t handle a sudden data burst. When this happens, you may notice the client resetting a TCP/IP connection.
Solution: these disconnections should timeout automatically. If this error persists, switch network connections if possible, or contact your ISP.
I’m trying to reconnect from a single client but I’m still getting an “HTTP 429 Too Many Requests” error. How do I connect?
If your stream is receiving a low volume of Tweets, it may take up to 20 seconds to detect a disconnection. This can happen for example when your client hasn’t received new Tweets, and it disconnects before attempting an immediate reconnection.
What if I get disconnected from the stream? How can I collect any Tweets that was missed while disconnected?
When streaming Tweets, the goal is to stay connected for as long as possible, recognizing that disconnects may occur. Streaming endpoints in Labs do not include a way to recover Tweets that were missed while disconnected. Instead, the endpoint provides a 20-second keep alive heartbeat (it will look like a new line character). Use this signal to detect if you’re being disconnected.
- Your code should detect when fresh content and the heartbeat stop arriving.
- If that happens, your code should trigger a reconnection logic. Some clients and languages allow you to specify a read timeout, which you can set to 20 seconds.
- Your service should detect these disconnections and reconnect as soon as possible.
The code samples on the “Quick start” page provide an example of this reconnect logic.