Getting started with premium Search Tweets: Full-Archive API

⏱ 10 min read

The premium Search Tweets: Full-Archive API provides you with Tweets since the first one posted in 2006. Tweets are matched and sent back to you based on the query you specify in your request. A query is a rule in which you define what the Tweet you get back should contain. In this tutorial, we will search for Tweets originating from the Twitter account @TwitterDev in English.

The Tweets you get back in your payload can be in a data format, which provides you with the full Tweet payload, or it can be in a counts format which gives you numerical count data of matched Tweets. We will be using cURL and Twurl to make requests to the data and counts endpoints, but feel free to use your preferred method.

You will need the following:

 

Accessing the data endpoint


The data endpoint will provide us with the full Tweet payload of matched Tweets. We will use the from: and lang: operators to find Tweets originating from @TwitterDev in English. For more operators click here.

Select a method to run the API call:

  • cURL
  • cURL example
  • Twurl
  • Twurl example

cURL is a command-line tool for getting or sending files using the URL syntax.

 

Copy the following cURL request into your command line after making changes to the following:

 

  • Environment name <ENV> e.g. Prod
  • Bearer token <BEARER_TOKEN>e.g. AAAAAAAAAAAA0%2EUifi76ZC9Ub0wn...
  • fromDate and toDate e.g. "fromDate":"201802010000", "toDate":"201802282359"

  curl --request POST \
  --url https://api.twitter.com/1.1/tweets/search/fullarchive/<ENV>.json \
  --header 'authorization: Bearer <BEARER_TOKEN>' \
  --header 'content-type: application/json' \
  --data '{
                "query":"from:TwitterDev lang:en",
                "maxResults": "100",
                "fromDate":"<YYYYMMDDHHmm>", 
                "toDate":"<YYYYMMDDHHmm>"
                }'

This is an example cURL request. If you try to run this it will not work. 

  curl --request POST \
  --url https://api.twitter.com/1.1/tweets/search/fullarchive/prod.json \
  --header 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAAMLheAAAAAAA0%2BuSeid%2BULvsea4JtiGRiSDSJSI%3DEUifiRBkKG5E2XzMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F' \
  --header 'content-type: application/json' \
  --data '{
                "query":"from:TwitterDev lang:en",
                "maxResults": "100",
                "fromDate":"201802010000", 
                "toDate":"201802282359"
                }'

Twurl is a cURL-like application, tailored specifically for the Twitter API.

 

Copy the following Twurl request into your command line after making changes to the following:

 

  • Environment name <ENV> e.g. Prod
  • fromDate and toDate e.g. "fromDate":"201802010000", "toDate":"201802282359"

  twurl "/1.1/tweets/search/fullarchive/<ENV>.json" -A "Content-Type: application/json" -d '{"query":"from:TwitterDev lang:en","maxResults":"100","fromDate":"<YYYYMMDDHHmm>","toDate":"<YYYYMMDDHHmm>"}' 

This is an example Twurl request. If you try to run this it will not work. 

  twurl "/1.1/tweets/search/fullarchive/prod.json" -A "Content-Type: application/json" -d '{"query":"from:TwitterDev lang:en","maxResults":"100","fromDate":"201802010000","toDate":"201802282359"}'

Data endpoint response payload

The payload you get back from your API request will appear in JSON format, as shown below.

{
  "results": 
     {
      "created_at": "Thu Feb 01 16:40:07 +0000 2018",
      "id": 959104084845453300,
      "id_str": "959104084845453312",
      "text": "RT @TwitterAPI: New year, new access for our developer community! 🎉\n\nToday, we’re launching our premium Search Tweets: Full-archive endpoin…",
      "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
      "truncated": false,
      "in_reply_to_status_id": null,
      "in_reply_to_status_id_str": null,
      "in_reply_to_user_id": null,
      "in_reply_to_user_id_str": null,
      "in_reply_to_screen_name": null,
      "user": {
        "id": 2244994945,
        "id_str": "2244994945",
        "name": "Twitter Dev",
        "screen_name": "TwitterDev",
        "location": "Internet",
        "url": "https://developer.twitter.com/",
        "description": "Your official source for Twitter Platform news, updates & events. Need technical help? Visit https://twittercommunity.com/ ⌨️ #TapIntoTwitter",
        "translator_type": "regular",
        "protected": false,
        "verified": true,
        "followers_count": 503283,
        "friends_count": 1478,
        "listed_count": 1449,
        "favourites_count": 2198,
        "statuses_count": 3381,
        "created_at": "Sat Dec 14 04:35:55 +0000 2013",
        "utc_offset": null,
        "time_zone": null,
        "geo_enabled": true,
        "lang": "en",
        "contributors_enabled": false,
        "is_translator": false,
        "profile_background_color": "FFFFFF",
        "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png",
        "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png",
        "profile_background_tile": false,
        "profile_link_color": "0084B4",
        "profile_sidebar_border_color": "FFFFFF",
        "profile_sidebar_fill_color": "DDEEF6",
        "profile_text_color": "333333",
        "profile_use_background_image": false,
        "profile_image_url": "http://pbs.twimg.com/profile_images/880136122604507136/xHrnqf1T_normal.jpg",
        "profile_image_url_https": "https://pbs.twimg.com/profile_images/880136122604507136/xHrnqf1T_normal.jpg",
        "profile_banner_url": "https://pbs.twimg.com/profile_banners/2244994945/1498675817",
        "default_profile": false,
        "default_profile_image": false,
        "following": null,
        "follow_request_sent": null,
        "notifications": null
      },
      "geo": null,
      "coordinates": null,
      "place": null,
      "contributors": null,
      "retweeted_status": {
        "created_at": "Thu Feb 01 16:37:43 +0000 2018",
        "id": 959103481599742000,
        "id_str": "959103481599741953",
        "text": "New year, new access for our developer community! 🎉\n\nToday, we’re launching our premium Search Tweets: Full-archive… https://t.co/eiEirrFQWy",
        "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
        "truncated": true,
        "in_reply_to_status_id": null,
        "in_reply_to_status_id_str": null,
        "in_reply_to_user_id": null,
        "in_reply_to_user_id_str": null,
        "in_reply_to_screen_name": null,
        "user": {
          "id": 6253282,
          "id_str": "6253282",
          "name": "Twitter API",
          "screen_name": "TwitterAPI",
          "location": "San Francisco, CA",
          "url": "https://developer.twitter.com",
          "description": "The Real Twitter API. Tweets about API changes, service issues and our Developer Platform. Don't get an answer? It's on my website.",
          "translator_type": "regular",
          "protected": false,
          "verified": true,
          "followers_count": 6151067,
          "friends_count": 12,
          "listed_count": 12992,
          "favourites_count": 31,
          "statuses_count": 3652,
          "created_at": "Wed May 23 06:01:13 +0000 2007",
          "utc_offset": null,
          "time_zone": null,
          "geo_enabled": false,
          "lang": "en",
          "contributors_enabled": false,
          "is_translator": false,
          "profile_background_color": "C0DEED",
          "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png",
          "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png",
          "profile_background_tile": true,
          "profile_link_color": "0084B4",
          "profile_sidebar_border_color": "C0DEED",
          "profile_sidebar_fill_color": "DDEEF6",
          "profile_text_color": "333333",
          "profile_use_background_image": true,
          "profile_image_url": "http://pbs.twimg.com/profile_images/942858479592554497/BbazLO9L_normal.jpg",
          "profile_image_url_https": "https://pbs.twimg.com/profile_images/942858479592554497/BbazLO9L_normal.jpg",
          "profile_banner_url": "https://pbs.twimg.com/profile_banners/6253282/1497491515",
          "default_profile": false,
          "default_profile_image": false,
          "following": null,
          "follow_request_sent": null,
          "notifications": null
        },
        "geo": null,
        "coordinates": null,
        "place": null,
        "contributors": null,
        "is_quote_status": false,
        "extended_tweet": {
          "full_text": "New year, new access for our developer community! 🎉\n\nToday, we’re launching our premium Search Tweets: Full-archive endpoint — providing access all the way back to @jack’s first Tweet in 2006. https://t.co/emgyLhnUob",
          "display_text_range": [
            0,
            216
          ],
          "entities": {
            "hashtags": [],
            "urls": [
              {
                "url": "https://t.co/emgyLhnUob",
                "expanded_url": "https://cards.twitter.com/cards/3q12a/5aju9",
                "display_url": "cards.twitter.com/cards/3q12a/5a…",
                "unwound": {
                  "url": "https://cards.twitter.com/cards/3q12a/5aju9",
                  "status": 200,
                  "title": "Introducing the premium Search Tweets: Full-archive endpoint",
                  "description": null
                },
                "indices": [
                  193,
                  216
                ]
              }
            ],
            "user_mentions": [
              {
                "screen_name": "jack",
                "name": "jack",
                "id": 12,
                "id_str": "12",
                "indices": [
                  164,
                  169
                ]
              }
            ],
            "symbols": []
          }
        },
        "quote_count": 0,
        "reply_count": 24,
        "retweet_count": 250,
        "favorite_count": 537,
        "entities": {
          "hashtags": [],
          "urls": [
            {
              "url": "https://t.co/eiEirrFQWy",
              "expanded_url": "https://twitter.com/i/web/status/959103481599741953",
              "display_url": "twitter.com/i/web/status/9…",
              "indices": [
                117,
                140
              ]
            }
          ],
          "user_mentions": [],
          "symbols": []
        },
        "favorited": false,
        "retweeted": false,
        "possibly_sensitive": false,
        "filter_level": "low",
        "lang": "en"
      },
      "is_quote_status": false,
      "quote_count": 0,
      "reply_count": 0,
      "retweet_count": 0,
      "favorite_count": 0,
      "entities": {
        "hashtags": [],
        "urls": [],
        "user_mentions": [
          {
            "screen_name": "TwitterAPI",
            "name": "Twitter API",
            "id": 6253282,
            "id_str": "6253282",
            "indices": [
              3,
              14
            ]
          }
        ],
        "symbols": []
      },
      "favorited": false,
      "retweeted": false,
      "filter_level": "low",
      "lang": "en",
      "matching_rules": [
        {
          "tag": null
        }
      ]
    }

Accessing the counts endpoint

With the counts endpoint, we’ll retrieve the number of Tweets originating from the @TwitterDev account in English grouped by day.

Please note that the counts endpoint is currently unavailable to premium sandbox users. 

Select a method to run the API call:

  • cURL
  • cURL example
  • Twurl
  • Twurl example

cURL is a command-line tool for getting or sending files using the URL syntax.

 

Copy the following request into your command line after making the appropriate adjustments to the following:

 

  • Environment name <ENV> e.g. Prod
  • Bearer token <BEARER_TOKEN> e.g. AAAAAAAAAAAA0%2EUifi76ZC9Ub0wn...
  • fromDate and toDate e.g. "fromDate":"201802010000", "toDate":"201802282359"

  curl --request POST \
  --url https://api.twitter.com/1.1/tweets/search/fullarchive/<ENV>/counts.json \
  --header 'authorization: Bearer <BEARER_TOKEN>' \
  --header 'content-type: application/json' \
  --data '{
                "query":"from:TwitterDev lang:en",
                "fromDate":"<YYYYMMDDHHmm>", 
                "toDate":"<YYYYMMDDHHmm>",
                "bucket": "day"
                }'

This is an example cURL request. If you try to run this it will not work.

  curl --request POST \
  --url https://api.twitter.com/1.1/tweets/search/fullarchive/prod/counts.json \
  --header 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAAMLheAAAAAAA0%2BuSeid%2BULvsea4JtiGRiSDSJSI%3DEUifiRBkKG5E2XzMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F' \
  --header 'content-type: application/json' \
  --data '{
                "query":"from:TwitterDev lang:en",
                "fromDate":"201802010000", 
                "toDate":"201802282359"
                "bucket": "day"
                }'

Twurl is a cURL-like application, tailored specifically for the Twitter API.

 

Copy the following Twurl request into your command line after making changes to the following:

 

  • Environment name <ENV> e.g. Prod
  • fromDate and toDate e.g. "fromDate":"201811010000", "toDate":"201811122359"

  twurl "/1.1/tweets/search/fullarchive/<ENV>/counts.json" -A "Content-Type: application/json" -d '{"query":"from:TwitterDev lang:en","fromDate":"<YYYYMMDDHHmm>","toDate":"<YYYYMMDDHHmm>","bucket": "day"}' 

This is an example Twurl request. If you try to run this it will not work. 

  twurl "/1.1/tweets/search/fullarchive/prod/counts.json" -A "Content-Type: application/json" -d '{"query":"from:TwitterDev lang:en","fromDate":"201802010000","toDate":"201802282359","bucket": "day"}'

Counts endpoint response payload

The payload you get back from your API request will appear in JSON format, as shown below.

{
  "results": [
    {
      "timePeriod": "201802010000",
      "count": 1
    },
    {
      "timePeriod": "201802020000",
      "count": 1
    },
    {
      "timePeriod": "201802030000",
      "count": 0
    },
    {
      "timePeriod": "201802040000",
      "count": 0
    },
    {
      "timePeriod": "201802050000",
      "count": 0
    },
    {
      "timePeriod": "201802060000",
      "count": 0
    },
    {
      "timePeriod": "201802070000",
      "count": 0
    },
    {
      "timePeriod": "201802080000",
      "count": 1
    },
    {
      "timePeriod": "201802090000",
      "count": 0
    },
    {
      "timePeriod": "201802100000",
      "count": 0
    },
    {
      "timePeriod": "201802110000",
      "count": 0
    },
    {
      "timePeriod": "201802120000",
      "count": 0
    },
    {
      "timePeriod": "201802130000",
      "count": 0
    },
    {
      "timePeriod": "201802140000",
      "count": 0
    },
    {
      "timePeriod": "201802150000",
      "count": 1
    },
    {
      "timePeriod": "201802160000",
      "count": 0
    },
    {
      "timePeriod": "201802170000",
      "count": 2
    },
    {
      "timePeriod": "201802180000",
      "count": 0
    },
    {
      "timePeriod": "201802190000",
      "count": 1
    },
    {
      "timePeriod": "201802200000",
      "count": 3
    },
    {
      "timePeriod": "201802210000",
      "count": 1
    },
    {
      "timePeriod": "201802220000",
      "count": 1
    },
    {
      "timePeriod": "201802230000",
      "count": 1
    },
    {
      "timePeriod": "201802240000",
      "count": 0
    },
    {
      "timePeriod": "201802250000",
      "count": 0
    },
    {
      "timePeriod": "201802260000",
      "count": 0
    },
    {
      "timePeriod": "201802270000",
      "count": 0
    },
    {
      "timePeriod": "201802280000",
      "count": 1
    }
  ],
  "totalCount": 14,
  "requestParameters": {
    "bucket": "day",
    "fromDate": "201802010000",
    "toDate": "201802282359"
  }
}

Great job! Now you've successfully accessed the premium Search Tweets: Full-Archive API.

 

Referenced articles

 

Next Steps