PowerTrack Rules API

Methods 

Method Description
POST /rules Add filtering rules to the stream
GET /rules Retrieve rules currently in place on the stream
POST /rules _method=get Get specific rules from the stream by ID
POST /rules _method=delete Delete filtering rules from the stream
POST /validation Validate PowerTrack rules

Authentication 

All requests to the PowerTrack rules API must use HTTP Basic Authentication, constructed from a valid email address and password combination used to log into your account at console.gnip.com. Credentials must be passed as the Authorization header for each request. Make sure your client is adding the "Authentication: Basic" HTTP header (with encoded credentials over HTTPS) to all API requests.

Important note on rule management: Rule sets are indexed by the value or ruleID, not the tag; therefore, all rule additions must reference the rule value or ruleID. In order to to make a tag update to an existing rule, you must first delete it and then add it back with the new tag value.

Rules must be unique per stream based on rule value, see below for a rule management example scenario:

CREATE RULE Action: POST rule {"value":"#TwitterData","tag":"tagtextA"} {"summary":{"created":1,"not_created":0},"detail":[{"rule":{"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"},"created":true}],"sent":"2018-02-08T18:14:23.691Z"} System: {"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"}

FAILED ATTEMPT TO UPDATE TAG Action: POST rule {"value":"#TwitterData","tag":"tagtextB"} **Rule tags cannot be "updated" - This request is ignored because rule value #TwitterData already exists. {"summary":{"created":0,"not_created":1},"detail":[{"rule":{"value":"#TwitterData","tag":"tagtextB","id":961664522481119232,"id_str":"961664522481119232"},"created":false,"message":"A rule with this value already exists"} System: {"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"}

FAILED ATTEMPT TO DELETE BY TAG Action: POST method=delete rule {"tag":"tagtextA"} **Rules cannot be deleted by tag. {"summary":{"deleted":0,"not_deleted":1},"detail":[{"rule":{"value":"","tag":null},"deleted":false,"message":"Rule does not exist"}],"sent":"2018-02-08T18:42:37.004Z"} System: {"value":"#TwitterData","tag":"tagtextA","id":961664522481119232,"id_str":"961664522481119232"}

DELETE BY ID Action: POST method=delete rule {"rule_ids":[961664522481119232]} {"summary":{"deleted":1,"not_deleted":0},"detail":[],"sent":"2018-02-08T18:53:54.185Z"}

DELETE BY VALUE Action: POST method=delete rule {"value":"#TwitterData"} {"summary":{"deleted":1,"not_deleted":0},"detail":[],"sent":"2018-02-08T18:53:54.185Z"}

RECREATE RULE- NOW WITH NEW ID Action: POST rule {"value":"#TwitterData","tag":"tagtextB"} {"summary":{"created":1,"not_created":0},"detail":[{"rule":{"value":"#TwitterData","tag":"tagtextB","id":961675641140609025,"id_str":"961675641140609025"},"created":true}],"sent":"2018-02-08T18:58:34.586Z"} System: {"value":"#TwitterData","tag":"tagtextB","id":961675641140609025,"id_str":"961675641140609025"}

POST /rules 

Adds one or many rules to your PowerTrack stream's ruleset.

Request Specifications

Request Method HTTP POST
Content Type "application/json". The request should specify this as the "Content-type".
URL Found on the API Help page, and uses the following structure:

https://data-api.twitter.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json
Character Encoding UTF-8
Request Body Format JSON
Request Body Size Limit 5 MB
Rate Limit 60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET). Rule addition requests will be processed serially and will be rejected if you have more than one rule request happening at the same time.

Request Body Content

Your request should provide rule data in the following format:

Content-type: "application/json"
{
    "rules":
    [
        {"value":"rule1","tag":"tag1"},
        {"value":"rule2"}
    ]
}

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line, using JSON rules.

curl -v -X POST -uexample@customer.com "https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json" -d '{"rules":[{"value":"rule1","tag":"tag1"},{"value":"rule2","tag":"tag2"},{"value":"rule2"}]}'

Responses

The following responses may be returned by the API for these requests. Non-200 responses should be retried after making any necessary modifications in the rules.

Status Text Description
201 Created The rule or rules were successfully added to your PowerTrack ruleset.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
422 Unprocessable Entity Generally occurs due to an invalid rule, based on the PowerTrack rule restrictions. Requests fail or succeed as a batch. For these errors, each invalid rule and the reason for rejection is included in a JSON message in the response. Catch the associated exception to expose this message.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, email support@gnip.com.

Example Responses

This response indicates that all rules (two in this case) were successfully created.

HTTP/1.1 201 Created
{
    "summary": {
        "created": 2,
        "not_created": 0
    },
    "detail": [{
        "rule": {
            "value": "(#snow OR snowday OR \"snow day\" OR from:noaa) lang:en",
            "tag": "4581245",
            "id": 734938587381145604
        },
        "created": true
    }, {
        "rule": {
            "value": "(skiing OR boarding OR \"hitting the slopes\" OR from:OnTheSnow) lang:en",
            "tag": "4581267",
            "id": 734938587381174273
        },
        "created": true
    }],
    "sent": "2016-05-24T02:46:28.402Z"
}

This response indicates that one rule was successfully created, and two were not created because they already exist. Rules are indexed by rule value (syntax). For rules not created there is a 'message' field explaining why the rule could not be created.

HTTP/1.1 201 Created
{
    "summary": {
        "created": 1,
        "not_created": 2
    },
    "detail": [{
        "rule": {
            "value": "eggplant 🍆",
            "tag": null,
            "id": 734861971116285952
        },
        "created": true
    }, {
        "rule": {
            "value": "fish 🐟",
            "tag": null
        },
        "created": false,
        "message": "A rule with this value already exists"
    }, {
        "rule": {
            "value": "Pizza 🍕",
            "tag": null
        },
        "created": false,
        "message": "A rule with this value already exists"
    }],
    "sent": "2016-05-23T21:42:01.661Z"
}

The following responses indicate that no rules were created. In each case there is a 'message' field explaining why the rule could not be created. Note that when one or more rules are invalid, no rules are added (even rules with valid syntax).


HTTP/1.1 422 Unprocessable Entity
{
    "summary": {
        "created": 0,
        "not_created": 2
    },
    "detail": [{
        "rule": {
            "value": "streaming gnip contains:$twtr",
            "tag": null
        },
        "created": false,
        "message": "no viable alternative at input '$twtr' (at position 25)\n"
    }, {
        "rule": {
            "value": "streaming gnip contains:\"$twtr\"",
            "tag": null
        },
        "created": false
    }],
    "sent": "2016-06-01T15:41:27.451Z"
}

HTTP/1.1 422 Unprocessable Entity
{
    "summary": {
        "created": 0,
        "not_created": 1
    },
    "detail": [{
        "rule": {
            "value": "-follow",
            "tag": null
        },
        "created": false,
        "message": "Rules must contain a non-negation term (at position 1)\nRules must contain at least one positive, non-stopword clause (at position 1)\n"
    }],
    "sent": "2016-05-23T18:34:25.218Z"
}

HTTP/1.1 422 Unprocessable Entity
{
    "summary": {
        "created": 0,
        "not_created": 1
    },
    "detail": [{
        "rule": {
            "value": "streaming AND lang:en",
            "tag": null
        },
        "created": false,
        "message": "Ambiguous use of and as a keyword. Use a space to logically join two clauses, or \"and\" to find occurrences of and in text (at position 11)\n"
    }],
    "sent": "2016-05-23T21:39:49.612Z"
}

POST /rules _method=delete 

Removes the specified rules from the stream.

Request Specifications

Request Method HTTP POST
Authentication Basic Authentication. Your login credentials must be included in the header of the request.
Content Type "application/json". The request should specify this as the "Content-type".
URL Found on the API Help page, and uses the following structure:

https://data-api.twitter.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json?_method=delete
Character Encoding UTF-8
Request Body Format JSON
Request Body Size Limit 5 MB
Rate Limit 60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).

Note: You cannot delete rules by referencing the rule tag. You must reference the rule value(s) or rule ID(s) when deleting a rule.

Action: POST rule { “value”:”#123”, “tag”:”A” } System: #123,A

Action: Delete (POST request with _method=delete) rule { "tag":"A" } **this will return an error, rules cannot be deleted by tag System: #123,A

Action: Delete (POST request with _method=delete) rule { "value":"#123", "tag":"A" }
System: 0 rules

Action: POST rule { "value":"#123", "tag":"B" } System: #123,B

Request Body Content

Your request should provide rule data in the following formats:

Content-type: "application/json"
{
    "rules":
    [
        {"value":"rule1"},
        {"value":"rule2"}
    ]
}
Content-type: "application/json"
{
    "rule_ids":
    [
        734938587381145604,
        734938587381174273
    ]
}

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl -v -X POST -uexample@customer.com \
"https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json?_method=delete" \
-d '{"rules":[{"value":"testrule"}]}"
curl -v -X POST -uexample@customer.com \
"https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json?_method=delete" \
-d '{"rule_ids":[734938587381145604,734938587381174273]}"

Responses

The following responses may be returned by the API for these requests. Non-200 responses should be retried following any necessary modifications to the rules being deleted.

Status Text Description
200 OK Indicates that the rule data supplied with the request consisted of valid JSON. However, note that if no rules are found in the ruleset for the PowerTrack stream based on a case-sensitive search, no rules will be deleted.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, email support@gnip.com.

Example Responses

HTTP/1.1 200 OK
{
    "summary": {
        "deleted": 3,
        "not_deleted": 0
    },
    "detail": [],
    "sent": "2016-06-01T15:46:48.654Z"
}

HTTP/1.1 200 OK
{
    "summary": {
        "deleted": 0,
        "not_deleted": 3
    },
    "detail": [{
        "rule": {
            "value": "Pizza",
            "tag": null
        },
        "deleted": false,
        "message": "Rule does not exist"
    }, {
        "rule": {
            "value": "eggplant",
            "tag": null
        },
        "deleted": false,
        "message": "Rule does not exist"
    }, {
        "rule": {
            "value": "fish",
            "tag": null
        },
        "deleted": false,
        "message": "Rule does not exist"
    }],
    "sent": "2016-06-01T15:49:15.951Z"
}

HTTP/1.1 400 Bad Request
{
    "error": {
        "message": "Invalid JSON. The body must be in the format {\"rules\":[{\"value\":\"rule1\", \"tag\":\"tag1\"}, {\"value\":\"rule2\"}]} or {\"rule_ids\": [rule_id1, rule_id2, rule_id3, rule_id4, rule_id5]}",
        "sent": "2013-08-16T00:50:00+00:00"
    }
}

POST /rules _method=get 

Retrieves requested existing rules for a stream.

Request Specifications

Request Method HTTP POST
URL Found on the API Help page, and uses the following structure:

https://data-api.twitter.com/rules/powertrack/accounts/{gnip_account_name}/publishers/twitter/{stream_label}.json?_method=get
Character Encoding UTF-8
Request Body Format JSON
Request Body Size Limit 5 MB
Rate Limit 60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).
Compression Gzip compression is supported, but not required for these requests.

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl -v -X POST -uexample@customer.com \
"https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json?_method=get" \
-d '{"rule_ids":[734938587381145604,734938587381174273]}"

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status Text Description
200 OK The request was successful, and the current ruleset is returned in JSON format.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, email support@gnip.com.

Example Response

HTTP/1.1 200 OK
{
    "rules": [{
        "value": "from:furiouscamper",
        "tag": null,
        "id": 734938587381145604
    }, {
        "value": "fish 🐟",
        "tag": null,
        "id": 734938587381174273
    }],
    "sent": "2016-06-01T15:52:37.341Z"
}

    {
    "error": {
        "message": "Invalid JSON. The body must be in the format {\"rules\":[{\"value\":\"rule1\", \"tag\":\"tag1\"}, {\"value\":\"rule2\"}]} or {\"rule_ids\": [rule_id1, rule_id2, rule_id3, rule_id4, rule_id5]}",
        "sent": "2013-08-16T00:50:00+00:00"
    }
}

GET /rules 

Retrieves all existing rules for a stream.

Request Specifications

Request Method HTTP GET
URL Found on the API Help page, and uses the following structure:

https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label.json
Request Body Format N/A
Request Body Size Limit N/A
Rate Limit 60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).
Compression Gzip compression is supported, but not required for these requests.

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status Text Description
200 OK The request was successful, and the current ruleset is returned in JSON format.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, email support@gnip.com.

Example Response

HTTP/1.1 200 OK
{
    "rules": [{
        "value": "from:jondee_test",
        "tag": null,
        "id": 735163830813134848
    }, {
        "value": "fish 🐟",
        "tag": null,
        "id": 738034522583769088
    }, {
        "value": "Pizza 🍕",
        "tag": null,
        "id": 738034522579554304
    }, {
        "value": "eggplant 🍆",
        "tag": null,
        "id": 738034522579570689
    }],
    "sent": "2016-06-01T15:52:37.341Z"
}

POST /validation 

Validates PowerTrack rules.

Request Specifications

Request Method HTTP POST
URL Found on the API Help page in console, and uses the following structure:

https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label/validation.json
Character Encoding UTF-8
Request Body Format JSON
Request Body Size Limit 5 MB
Rate Limit 60 requests per minute, aggregated across all requests to /rules endpoint for the specific stream's API (POST and GET).
Compression Gzip compression is supported, but not required for these requests.

Example curl Request

The following example request demonstrates how to add rules using cURL on the command line.

curl --compressed -v -uexample@customer.com \
"https://data-api.twitter.com/rules/powertrack/accounts/:account_name/publishers/twitter/:stream_label/validation.json" \
-d '{
    "rules": [{
        "value": "Pizza OR 🍕 OR \"🍕\" sample:100"
    }, {
        "value": "from:contains:heart"
    }, {
        "value": "fish AND bird"
    }, {
        "value": "(((\"#quotedhashtag\""
    }, {
        "value": "bounding_box:[-71.199636,42.230046,-70.979909,42.398619]"
    }, {
        "value": "from:jack"
    }]
}'

Response

The following responses may be returned by the API for these requests. Non-200 responses should be retried, utilizing an exponential backoff for subsequent requests.

Status Text Description
200 OK The request was successful, and the rule validation result is returned.
400 Bad Request Generally relates to poorly formatted JSON, and includes an "Invalid JSON" message in the response.
401 Unauthorized HTTP authentication failed due to invalid credentials. Log in to console.gnip.com with your credentials to ensure you are using them correctly with your request.
429 Rate Limited Your app has exceeded the limit on requests to add, delete, or list rules for this stream.
503 Service Unavailable Gnip server issue. Reconnect using an exponential backoff pattern. If no notice about this issue has been posted on the Twitter API Status Page, email support@gnip.com.

Example Response

HTTP/1.1 200 OK
{
    "summary": {
        "valid": 1,
        "not_valid": 5
    },
    "detail": [{
        "rule": {
            "value": "from:jack",
            "tag": null
        },
        "valid": true
    }, {
        "rule": {
            "value": "Pizza OR 🍕 OR \"🍕\" sample:100",
            "tag": null
        },
        "valid": false,
        "message": "The sample operator cannot be used with an OR. To use the sample operator with an OR in the rule, the ORed clauses must be grouped together with parenthesis.  For example, to get 10% of activites that have term1 or term2, the rule should be (excluding the single quotes) '(term1 OR term2) sample:10' (at position 21)\n"
    }, {
        "rule": {
            "value": "from:contains:heart",
            "tag": null
        },
        "valid": false,
        "message": "Cannot parse rule at ':' (position 14)\n"
    }, {
        "rule": {
            "value": "fish AND bird",
            "tag": null
        },
        "valid": false,
        "message": "Ambiguous use of and as a keyword. Use a space to logically join two clauses, or \"and\" to find occurrences of and in text (at position 6)\n"
    }, {
        "rule": {
            "value": "(((\"#quotedhashtag\"",
            "tag": null
        },
        "valid": false,
        "message": "mismatched input 'EOF' expecting ')' (at position 20)\n\n"
    }, {
        "rule": {
            "value": "bounding_box:[-71.199636,42.230046,-70.979909,42.398619]",
            "tag": null
        },
        "valid": false,
        "message": "Cannot parse rule at '71.199636,42.230046,-70.979909,42.398619' (position 16)\n"
    }],
    "sent": "2017-03-16T02:33:01.827Z"
}