Historical PowerTrack Data Format
Data dictionary
All Twitter APIs that return Tweets provide that data encoded using JavaScript Object Notation (JSON). JSON is based on key-value pairs, with named attributes and associated values. In addition to the text content itself, a Tweet can have over 150 attributes associated with it.
To learn more about these attributes and the organization of the Tweet payload, please read through our Data Dictionary documentation. Note that Tweets provided by Historical PowerTrack have User objects that can contain an additional "updated" attribute, as described below.
User object specification in Historical PowerTrack
User profile information reflects the values of the referenced user at the time of the Historical PowerTrack job processing. When the updated User objects are delivered, there will be a "updated" string array in the User/Actor object. There are up to three possible values: name, description, location.
"updated": ["name", "description", "location"]
"name" is included when the display name has changed since the 'to_date' of the requested time period.
"description" is included when the Profile bio has changed since the 'to_date' of the requested time period..
"location" is included when the Profile Location has changed since the 'to_date' of the requested time period..
See below for "original behavior" and "updated behavior" User objects for pulling a Tweet posted last year. The original payload contains the Twitter User Object as it was when the Tweet was posted. The after example reflects that the display name, the Profile bio/description (added an hashtag at end), and Profile Location all changed, and the values here are from the Profile as of when the historical job ran ("time of query").
Note: while these examples are in the "original" (or "enriched native") format, this new "updated" attribute is exactly the same in the Activity Streams format.
User object at the time of Tweet creation
User profile attributes at the time the Tweet was posted:
- Name: "the SnowBot"
- Description: "Boulder, CO"
- Profile location: "Home of the SnowBot, a demo of the Twitter Account Activity and Direct Message APIs. Also a repository of snow-related Tweets. @TwitterDev"
{
"created_at": "Sat Feb 24 02:17:37 +0000 2018",
"id": 967221948471115776,
"id_str": "967221948471115776",
"text": "A good week of snow https:\/\/t.co\/b8DbfQ9Igp",
"user": {
"id": 906948460078698496,
"id_str": "906948460078698496",
"name": "the SnowBot",
"screen_name": "SnowBotDev",
"location": "Boulder, CO",
"url": "https://github.com/TwitterDev/SnowBotDev",
"description": "Home of the SnowBot, a demo of the Twitter Account Activity and Direct Message APIs. Also a repository of snow related Tweets. @TwitterDev",
"translator_type": "none",
"derived": {
"locations": [{
"country": "United States",
"country_code": "US",
"locality": "Boulder",
"region": "Colorado",
"sub_region": "Boulder County",
"full_name": "Boulder, Colorado, United States",
"geo": {
"coordinates": [-105.27055, 40.01499],
"type": "point"
}
}]
},
"protected": false,
"verified": false,
"followers_count": 104,
"friends_count": 32,
"listed_count": 4,
"favourites_count": 39,
"statuses_count": 134,
"created_at": "Sun Sep 10 18:32:17 +0000 2017",
"utc_offset": null,
"time_zone": null,
"geo_enabled": null,
"lang": null
}
}
User object returned for matched Tweet at the time of Historical PowerTrack request
Since the Tweet was originally posted, these user Profile attributes have changed. Note that the JSON payload (user.updated) reflects this.
- Name: "the SnowBot!"
- Description: "Colorado, USA"
- Profile location: "Home of the SnowBot, a demo of the Twitter Account Activity and Direct Message APIs. Also a repository of snow-related Tweets and photos. @TwitterDev #ThinkSnow"
{
"created_at": "Sat Feb 24 02:17:37 +0000 2018",
"id": 967221948471115776,
"id_str": "967221948471115776",
"text": "A good week of snow https:\/\/t.co\/b8DbfQ9Igp",
"user": {
"id": 906948460078698496,
"id_str": "906948460078698496",
"name": "the SnowBot!",
"screen_name": "SnowBotDev",
"location": "Colorado, USA",
"url": "https://github.com/TwitterDev/SnowBotDev",
"description": "Home of the SnowBot, a demo of the Twitter Account Activity and Direct Message APIs. Also a repository of snow related Tweets and photos. @TwitterDev #ThinkSnow",
"translator_type": "none",
"derived": {
"locations": [{
"country": "United States",
"country_code": "US",
"region": "Colorado",
"full_name": "Colorado, United States",
"geo": {
"coordinates": [-105.50083, 39.00027],
"type": "point"
}
}]
},
"protected": false,
"verified": false,
"followers_count": 104,
"friends_count": 32,
"listed_count": 4,
"favourites_count": 39,
"statuses_count": 134,
"created_at": "Sun Sep 10 18:32:17 +0000 2017",
"utc_offset": null,
"time_zone": null,
"geo_enabled": true,
"lang": null,
"updated": ["name", "description", "location"]
}
}
Matching rules enrichment
When an activity is delivered, metadata is added in the “matching rules” portion of the activity to indicate which rule or rules caused that specific activity to be delivered. If multiple rules match a single activity, the activity is delivered a single time with each of the matching rules included in this metadata. The matching rules provide an easy way to associate a specific activity with specific rules and customers in your product, even when you have many customers with lots of distinct rules.
Here is an example of the matching rules object:
"matching_rules": [{
"tag": snowflake,
"id": 907728589029646336,
"id_str": "907728589029646336"
}]
Rule tags
At the time they are created, each filtering rule may be created with a tag. Rule tags have no special meaning, they are simply treated as opaque strings carried along with the rule. They will be included in the “matching rules” metadata in activities returned. Tags provide an easy way to create logical groupings of PowerTrack rules. For example, you may generate a unique ID for a specific rule as its tag, and allow your app to reference that ID within activities it processes to associate a result with specific customers, campaigns, categories, or other related groups.
In the above ‘Matching Rules’ example, you can see that the rule that the tweet matched was tagged with the ID, ‘snowflake.’
Note that tags cannot be updated on an existing rule, but can only be included when a rule is created. In order to “update” a tag, you need to first remove the rule, then add it again with the updated tag. The best solution is to simply use a unique identifier as your tag, which your system can associate with various other data points within your own app, all without having to change anything in the rule set.