Filtered stream
Building queries for the filtered stream endpoint
Operator syntax
With the filtered stream endpoints, up to 25 queries (also referred to as a 'rule' or 'filter') can be submitted to a stream using the POST /tweets/search/stream/rules endpoint, which will then match on Tweets in real-time and deliver those matching Tweets through a persistent streaming connection. Queries are made up of operators that are used to match on a variety of Tweet attributes.
To create a query, a boolean syntax is used to group multiple operators. Here are the building blocks of the filtered stream query syntax:
- Successive operators with a space between them will result in AND logic, meaning that Tweets will match only if both conditions are met. For example, snow day #NoSchool will match Tweets containing the keywords snow and day and the hashtag #NoSchool.
- Successive operators with OR between them will result in OR logic, meaning that Tweets will match if either condition is met. For example, specifying snow OR day OR #NoSchool will match any Tweets containing at least one of snow, day, or the hashtag #NoSchool.
- You can use parentheses to group operators together. For example, (snow day) OR (#NoSchool has:images) will return either Tweets containing the terms snow and day, or Tweets with images containing the hashtag #NoSchool. Note that ANDs are applied first, then ORs are applied.
- Prepend a dash to a keyword (or any operator) to negate it (NOT). For example, snow #NoSchool -day will match Tweets containing the hashtag #NoSchool and the term snow, but only if they do not contain the term day. You can also negate operators grouped using parenthesis. One common query clause is -is:retweet, which will not match on Retweets, thus matching only on original Tweets.
Please note the following important details about building queries:
- Queries can be 512 characters long.
- All operators can be negated except for sample. Negated operators cannot be used alone.
- Do not negate a set of operators grouped together in a set of parentheses. Instead, negate each individual operator.
For example, Instead of using -(snow OR day OR noschool), we suggest that you use -snow -day -noschool.
Twitter API v2 introduces new features that expand ways to match on Tweets of interest:
- Twitter annotations: Filtered stream endpoints provide entity and context operators to match on contextual information about the Tweet itself.
- Conversation IDs: To help make it easier to thread together conversations on Twitter, a new conversation_id operator is available. When a Tweet is part of a conversation thread, the conversation ID for that Tweet will match the parent Tweet ID.
Standalone operators
These operators can be used alone or together with any other operators (including those from the Entity operators list).
| Operator | Description |
|---|---|
| keyword | Matches a keyword within the body of a Tweet. This is a tokenized match, meaning that your keyword string will be matched against the tokenized text of the Tweet body. Tokenization splits words based on punctuation, symbols, and Unicode basic plane separator characters. For example, a Tweet with the text “I like coca-cola” would be split into the following tokens: I, like, coca, cola. These tokens would then be compared to the keyword string used in your rule. To match strings containing punctuation (e.g. coca-cola), symbol, or separator characters, you must wrap your keyword in double-quotes. |
| emoji | Matches an emoji within the body of a Tweet. Similar to a keyword, emojis are a tokenized match, meaning that your emoji will be matched against the tokenized text of the Tweet body. Note that if an emoji has a variant, you must wrap it in double quotes to add to a rule. |
| "exact phrase match" | Matches the exact phrase within the body of a Tweet. |
| # | Matches any Tweet containing a recognized hashtag, if the hashtag is a recognized entity in a Tweet. This operator performs an exact match, NOT a tokenized match, meaning the rule #thanku will match posts with the exact hashtag #thanku, but not those with the hashtag #thankunext. |
| @ | Matches any Tweet that mentions the given username, if the username is a recognized entity (including the @ character). |
| from: | Matches any Tweet from a specific user. The value can be either the username (excluding the @ character) or the user’s numeric user ID. |
| to: | Matches any Tweet that is in reply to a particular user. The value can be either the username (excluding the @ character) or the user’s numeric user ID. |
| url: | Performs a tokenized match on any validly-formatted URL of a Tweet.
Tokens and phrases containing punctuation or special characters should be double-quoted (for example, url:"/developer"). Similarly, to match on a specific protocol, enclose in double-quotes (for example, url:"https://developer.twitter.com"). |
| retweets_of: | Matches Tweets that are Retweets of the specified user. The value can be either the username (excluding the @ character) or the user’s numeric user ID. |
| context: | NEW Matches Tweets with a specific domain id and/or domain id, enitity id pair where * represents a wildcard. To learn more about this operator, please visit our page on annotations.
|
| entity: | NEW Matches Tweets with a specific entity string value. To learn more about this operator, please visit our page on annotations.
|
| conversation_id: | NEW Matches Tweets that share a common conversation ID. A conversation ID is set to the Tweet ID of a Tweet that started a conversation. As Replies to a Tweet are posted, even Replies to Replies, the conversation_id is added to its JSON payload. |
Non-standalone operators
The following operators cannot be used as an independent clause in a query; they can only be used together with at least one operator from the Standalone operators list.
For example, the following query is not supported since it contains only Entity operators, would be far too general, and would likely match on an extreme volume of Tweets.
has:mentions (has:media OR has:links)
If we add in a standalone operator, such as the keyword snow, the following query is supported.
snow has:mentions (has:media OR has:links)
| Operator | Description | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| is:retweet | Matches on Retweets that match the rest of the specified rule. This operator looks only for true Retweets (for example, those generated using the Retweet button). Quote Tweets will not be matched by this operator. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| is:reply | Deliver only explicit replies that match a rule. Can also be negated to exclude replies that match a rule from delivery (see example request below). When used with the filtered stream, this operator matches on replies to an original Tweet, replies in quoted Tweets and replies in Retweets. Example request: from:twitterdev -is:reply. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| is:quote | Returns all Quote Tweets, also known as Tweets with comments. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| is:verified | Deliver only Tweets whose authors are verified by Twitter. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| has:hashtag | Matches Tweets that contain at least one hashtag. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| has:links | This operator matches Tweets which contain links in the Tweet body. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| has:mentions | Matches Tweets that mention another Twitter user. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| has:media | Matches Tweets that contain a media URL recognized by Twitter. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| has:images | Matches Tweets that contain a recognized URL to an image. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| has:videos | Matches Tweets that contain native Twitter videos, uploaded directly to Twitter. This will not match on videos created with Periscope, or Tweets with links to other video hosting sites. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
sample: |
Returns a random percent sample of Tweets that match a rule rather than the entire set of Tweets. The percent value must be represented by an integer between 1 and 100 (for example, sample:10 will return a random 10% sample). This operator first reduces the scope of the stream to the percentage you specified, then the rule/filter is applied to that sampled subset. In other words, if you are using, for example, sample:10, each Tweet will have a 10% chance of being in the sample. This operator applies to the entire rule and requires all OR'd terms to be grouped. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| lang: | Matches Tweets that have been classified by Twitter as being of a particular language (if, and only if, the tweet has been classified). It is important to note that each Tweet is currently only classified as being of one language, so AND’ing together multiple languages will yield no results. Note: if no language classification can be made the provided result is ‘und’ (for undefined).
|
