Twitter API Documentation

From wiki
Jump to navigation Jump to search

There are two ways to authenticate a request with the Twitter API.

  • App-only authentication - requests don't have to be on the behalf of a user.
  • Oauth - requests can be made on behalf of a user.

App-only auth - good for if you want your application to process public tweets that can come from a user or a search query.

Oauth is required for requests that are on behalf of a user, eg reading friends-only tweets and posting tweets (in addition to reading public tweets).

App-only authentication

https://dev.twitter.com/docs/auth/application-only-auth

Lets start with App-only authentication as that is the easier on to start with. In registering an app with twitter, each app is assigned a Consumer Key and a Consumer Secret.

Consumer key: xvz1evFS4wEEPTGEFPHBog
Consumer secret: L8qq9PZyRg6ieKGEKhZolGC0vJWLw8iEJ88DRdyOg

(examples as provided by twitter)

These are then encoded individually according to RFC 1738 and then the encoded key is concatenated with a colon (:) and then the encoded secret.

This entire string is then encoded using base64.

This is serves as a basic authentication header for a request that then retrieves an access token.

encoded_consumer_key.Text = Uri.EscapeDataString(consumer_key_2.Text);
encoded_consumer_secret.Text = Uri.EscapeDataString(consumer_secret_2.Text;
bearer_token_credentials.Text = encoded_consumer_key.Text + ":" + encoded_consumer_secret.Text;
byte[] byteArray = Encoding.UTF8.GetBytes(bearer_token_credentials.Text);
encoded_bearer_token_credentials.Text = System.Convert.ToBase64String(byteArray);

Constructing the request (example headers)

POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic eHZ6MWV2RlM0d0VFUFRHRUZQSEJvZzpMOHFxOVBaeVJnNmllS0dFS2hab2xHQzB2SldMdzhpRUo4OERSZHlPZw==
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
grant_type=client_credentials

Example request

Code fragments are C#

request_uri.Text = "https://api.twitter.com/oauth2/token";
request_query.Text = "";
request_method.Text = "POST";
request_host.Text = "api.twitter.com";
request_user_agent.Text = "Alan's Test Application";
request_body.Text = "grant_type=client_credentials";
authorization_header_textbox.Text = "Basic " + encoded_bearer_token_credentials.Text;
HttpWebRequest http_request = null;
Stream dataStream = null;
HttpWebResponse http_response = null;
StreamReader reader = null;
http_request = (HttpWebRequest)WebRequest.Create(request_uri.Text);
http_request.Method = request_method.Text;
http_request.Host = request_host.Text;
http_request.UserAgent = request_user_agent.Text;
http_request.Headers.Add("Authorization", authorization_header_textbox.Text);
string body = request_body.Text;
byte[] bodydata = Encoding.UTF8.GetBytes(body); 
http_request.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
http_request.ContentLength = bodydata.Length;
dataStream = http_request.GetRequestStream();
dataStream.Write(bodydata, 0, bodydata.Length);
dataStream.Close();
string responseFromServer = "";
response.Text = "";
try{
  http_response = (HttpWebResponse)http_request.GetResponse();
  dataStream = http_response.GetResponseStream();
  reader = new StreamReader(dataStream);
  responseFromServer = reader.ReadToEnd();
  response.Text = responseFromServer;
}
catch (Exception e2)
{
  response.Text = e2.ToString();
}
finally
{
  if (reader != null) { reader.Close(); }
  if (http_response != null) { http_response.Close(); }
  if (dataStream != null) { dataStream.Close(); }
}

The request parameters are set. Then the http request is constructed. Because this is a POST request (as opposed to a GET request) the data is embedded in the body of the request rather than as parameters in the URL. A response is constructed and the result of that is stored somewhere.

A response looks like this:

HTTP/1.1 200 OK
Status: 200 OK
Content-Type: application/json; charset=utf-8
...
Content-Encoding: gzip
Content-Length: 140 

{"token_type":"bearer","access_token":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%2FAAAAAAAAAAAAAAAAAAAA%3DAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}

Responses are in the form of json. Here is more information on how to handle json

Now that we've gotten the app-only auth access token we can use this to make authorized GET requests to read public user's timeline.

GET statuses/user_timeline

https://dev.twitter.com/docs/api/1.1

The url is:

https://api.twitter.com/1.1/statuses/user_timeline.json

and two parameters we can use are screen_name and count. screen_name is the user whose timeline we want to look at, and count is the number of tweets to retrieve starting from the latest one. We set the parameters and because this is a GET request we string the parameters with the url like so:

https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=billgates&count=2

Then we create a httpwebrequest on this url. Compare this to a POST request where the parameters make up the mode. No body here means we can skip that part.

With a public account we just can just use app-only auth to do the request. So lets use that access_token from before.

http_request.Headers.Add("Authorization", "bearer AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%2FAAAAAAAAAAAAAAAAAAAA%3DAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
http_request.Method = request_method.Text;
http_request.Host = request_host.Text;
http_request.UserAgent = request_user_agent.Text;
http_request.Headers.Add("Authorization", authorization_header_textbox.Text);
HttpWebRequest http_request = null;
Stream dataStream = null;
HttpWebResponse http_response = null;
StreamReader reader = null;
http_request = (HttpWebRequest)WebRequest.Create(request_uri.Text + "?" + request_query.Text);
string responseFromServer = "";
response.Text = "";
try
{
  http_response = (HttpWebResponse)http_request.GetResponse();
  dataStream = http_response.GetResponseStream();
  reader = new StreamReader(dataStream);
  responseFromServer = reader.ReadToEnd();
  response.Text = responseFromServer;
}
catch (Exception e2)
{ 
  response.Text = e2.ToString();
}
finally
{
  if (reader != null) { reader.Close(); }
  if (http_response != null) { http_response.Close(); }
  if (dataStream != null) { dataStream.Close(); }
}


POST statuses/update

https://dev.twitter.com/docs/api/1.1/post/statuses/update

In updating a status to twitter (in other words, tweeting a tweet) status is a required parameter otherwise whats the point in making a POST request to update? So we start like so.

request_method.Text = "POST";
request_uri.Text = "https://api.twitter.com/1.1/statuses/update.json";
request_query.Text = "status=" + PercentEncode(status_textbox.Text);
request_host.Text = "api.twitter.com";
request_user_agent.Text = "Alan's Test Application";
request_body.Text = "status=" + PercentEncode(status_textbox.Text, true);

Lets take a look at how PercentEncode works. Note in the query the status is encoded slightly different to the body. Also it is important to note that because this is a POST request the query part won't part of the actual request. This is just here to generate the oauth sig which will be covered later.

https://en.wikipedia.org/wiki/Percent_encoding#Percent-encoding_arbitrary_data

in header Reserved Characters are encoded as usual.

in body additional chars are also encoded. table under character data. unauth request if body wasn't encoded like that

Unreserved Characters never need encoding

looking at wiki article, there are reserved characters that have to be encoded. Encoding is used to facilitate the transmission of data. It makes a distinction between characters which have special meaning (instructions) and characters which should be treated as data.

Actually, a lot of exploits (eg SQL injections) are the result of systems treating data as instructions.

sidetracked

 https://en.wikipedia.org/wiki/Sql_injections
eg from wikipedia
 statement = "SELECT * FROM users WHERE name ='" + userName + "';"
suppose userName stored the value "john". 
 Then this statement would be executed
 SELECT * FROM users WHERE name = 'john';
 now imagine userName stored the value "Robert'); DROP TABLE Students;--".
 Then this statement would be executed
 SELECT * FROM users WHERE name = 'Robert'); DROP TABLE Students;--';
 What is happening here? Two queries are now being executed. 
 SELECT * FROM users WHERE name = 'Robert'); 
 DROP TABLE Students;--';
  

Robert becomes that name that is searched for. This statement is ended with a closing bracket (this assumes there is an corresponding opening bracket) and a semi-colon. Ending a statement with a semi colon allows a new statement to be executed. In this case it is the DROP TABLE statement which deletes the table 'Students'. -- starts a comments which means the rest of the query is ignored.

the trick to fix this is to parametize queries so that when the system expects data it will be treated as data rather than instructions.


because in urls & has a special meaning and is used to separate parameters. imagine you want to search for the query "truth&lies" compare the urls and what you get from them.

http://nz.bing.com/search?q=truth%26lies
http://nz.bing.com/search?q=truth&lies


Encoding Data

https://en.wikipedia.org/wiki/Percent_encoding

public static string PercentEncode(string s, bool is_query_string = false)
{
	//https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters

	string result = "";
	foreach (char c in s)
	{
		switch (c)
		{
			case ' ':
				if (is_query_string) { result += "+"; } else { result += "%20"; }
				break;
			case '!':
				result += "%21";
				break;
			case '#':
				result += "%23";
				break;
			case '$':
				result += "%24";
				break;
			case '%':
				result += "%25";
				break;
			case '&':
				result += "%26";
				break;
			case '\'':
				result += "%27";
				break;
			case '(':
				result += "%28";
				break;
			case ')':
				result += "%29";
				break;
			case '*':
				result += "%2A";
				break;
			case '+':
				result += "%2B";
				break;
			case ',':
				result += "%2C";
				break;
			case '/':
				result += "%2F";
				break;
			case ':':
				result += "%3A";
				break;
			case ';':
				result += "%3B";
				break;
			case '=':
				result += "%3D";
				break;
			case '?':
				result += "%3F";
				break;
			case '@':
				result += "%40";
				break;
			case '[':
				result += "%5B";
				break;
			case ']':
				result += "%5D";
				break;
			// these must be done for arb data
			case '"':
				if (is_query_string) { result += "%22"; } else { result += c; }
				break;
			case '-':
				if (is_query_string) { result += "%2D"; } else { result += c; }
				break;
			case '.':
				if (is_query_string) { result += "%2E"; } else { result += c; }
				break;
			case '<':
				if (is_query_string) { result += "%3C"; } else { result += c; }
				break;
			case '>':
				if (is_query_string) { result += "%3E"; } else { result += c; }
				break;
			case '\\':
				if (is_query_string) { result += "%5C"; } else { result += c; }
				break;
			case '^':
				if (is_query_string) { result += "%5E"; } else { result += c; }
				break;
			case '_':
				if (is_query_string) { result += "%5F"; } else { result += c; }
				break;
			case '`':
				if (is_query_string) { result += "%60"; } else { result += c; }
				break;
			case '{':
				if (is_query_string) { result += "%7B"; } else { result += c; }
				break;
			case '|':
				if (is_query_string) { result += "%7C"; } else { result += c; }
				break;
			case '}':
				if (is_query_string) { result += "%7D"; } else { result += c; }
				break;
			case '~':
				if (is_query_string) { result += "%7E"; } else { result += c; }
				break;
			default:
				result += c;
				break;
		}
	}
	return result;
}

post status

encoding status in query and body (differently) query -> encoded one way to generate oauth signature body -> will be what is posted.

Generating oauth headers and stuff

Sign a request so that we are sure of

  • Which application is making the request
  • Which user the request is posting on behalf of
  • Whether the user has granted the application authorization to post on the user's behalf
  • Whether the request has been tampered by a third party while in transit

oauth headers added as Authorization HTTP header

consist of 7 key/value pairs, keys start with "oauth_"

oauth_consumer_key - identifies application

oauth_nonce - random string with each request - https://en.wikipedia.org/wiki/Cryptographic_nonce

private static string validChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
private static Random random = new Random();
public void set_oauth_nonce()
{
  int length = 42;
  var nonceString = new StringBuilder();
  for (int i = 0; i < length; i++)
  {
    nonceString.Append(validChars[random.Next(0, validChars.Length - 1)]);
  }
  oauth_nonce_textbox.Text = nonceString.ToString();
}


oauth_signature_method - set to "HMAC-SHA1"

oauth_timestamp - unix timestamp

oauth_timestamp_textbox.Text = "" + (Int32)(DateTime.Now.Subtract(new DateTime(1970, 1, 1))).TotalSeconds;

oauth_token - user specific token

oauth_version - set to "1.0"

oauth_signature - generated based on all request parameters. See https://dev.twitter.com/docs/auth/creating-signature

Generating oauth_signature

https://dev.twitter.com/docs/auth/creating-signature

Collecting parameters

  • Percent encode every key and value that will be signed.
  • Sort the list of parameters alphabetically[1] by encoded key[2].
  • For each key/value pair:
  • Append the encoded key to the output string.
  • Append the '=' character to the output string.
  • Append the encoded value to the output string.
  • If there are more key/value pairs remaining, append a '&' character to the output string.

for status parameter I've already encoded it before hand and then concat with other param in header, else things may get lost

ArrayList parameters = new ArrayList();
string parameter_string = "";
foreach (string s in query.Split(new Char[] { '&' }))
{
	if (s.Equals(""))
	{
		continue;
	}
	string query_key = s.Split(new Char[] { '=' })[0];
	string query_value = s.Split(new Char[] { '=' })[1];
	if (query_key.Equals("status"))
	{
		parameters.Add(query_key + "=" + (query_value));
	}
	else
	{
		parameters.Add(query_key + "=" + PercentEncode(query_value));
	}
}
parameters.Add("oauth_consumer_key=" + PercentEncode(oauth_consumer_key));
parameters.Add("oauth_nonce=" + PercentEncode(oauth_nonce));
parameters.Add("oauth_signature_method=" + PercentEncode(oauth_signature_method));
parameters.Add("oauth_timestamp=" + PercentEncode(oauth_timestamp));
parameters.Add("oauth_token=" + PercentEncode(oauth_token));
parameters.Add("oauth_version=" + PercentEncode(oauth_version));
parameters.Sort();
foreach (String s in parameters)
{
	parameter_string += s + "&";
}
parameter_string = parameter_string.Substring(0, parameter_string.Length - 1);

Creating the signature base string

To encode the HTTP method, base URL, and parameter string into a single string:

  • Convert the HTTP Method to uppercase and set the output string equal to this value.
  • Append the '&' character to the output string.
  • Percent encode the URL and append it to the output string.
  • Append the '&' character to the output string.
  • Percent encode the parameter string and append it to the output string.
string signature_base_string = "";
signature_base_string = method.ToUpper();
signature_base_string += "&";
signature_base_string += PercentEncode(uri);
signature_base_string += "&";
signature_base_string += PercentEncode(parameter_string);
signature_base_string_textbox.Text = signature_base_string;

Getting a signing key

Consumer secret - app specific eg kAcSOqF21Fu85e7zjz7ZN2U4ZRhfV3WpwPAoE3Z7kBw
OAuth token secret - user specific eg LswwdoUaIvS8ltyTt5jkRh4J50vUPVVHtR2YPi5kE
Signing key - concat with & eg kAcSOqF21Fu85e7zjz7ZN2U4ZRhfV3WpwPAoE3Z7kBw&LswwdoUaIvS8ltyTt5jkRh4J50vUPVVHtR2YPi5kE

Calculating the signature

Finally, the signature is calculated by passing the signature base string and signing key to the HMAC-SHA1 hashing algorithm. The details of the algorithm are explained in depth here, but thankfully there are implementations of HMAC-SHA1 available for every popular language. For example, PHP has the hash_hmac function.

The output of the HMAC signing function is a binary string. This needs to be base64 encoded to produce the signature string. For example, the output given the base string and signing key given on this page is B6 79 C0 AF 18 F4 E9 C5 87 AB 8E 20 0A CD 4E 48 A9 3F 8C B6. That value, when converted to base64, is the OAuth signature for this request:

OAuth signature	tnnArxj06cWHq44gCs1OSKk/jLY=
byte[] key = Encoding.UTF8.GetBytes(PercentEncode(oauth_consumer_secret) + "&" + PercentEncode(oauth_token_secret));
string input = signature_base_string;
oauth_signature = Encode(input, key);
oauth_signature_textbox.Text = oauth_signature;


finishing off the request - https://dev.twitter.com/docs/auth/authorizing-request

Building the header string

To build the header string, imagine writing to a string named DST.

  • Append the string "OAuth " (including the space at the end) to DST.
  • For each key/value pair of the 7 parameters listed above:
  • Percent encode the key and append it to DST.
  • Append the equals character '=' to DST.
  • Append a double quote '"' to DST.
  • Percent encode the value and append it to DST.
  • Append a double quote '"' to DST.
  • If there are key/value pairs remaining, append a comma ',' and a space ' ' to DST.
string DST = "OAuth ";
DST += "oauth_consumer_key=\"" + PercentEncode(consumer_key_2.Text) + "\", "; 
DST += "oauth_nonce=\"" + PercentEncode(oauth_nonce_textbox.Text) + "\", "; 
DST += "oauth_signature=\"" + PercentEncode(oauth_signature_textbox.Text) + "\", "; 
DST += "oauth_signature_method=\"" + PercentEncode(oauth_signature_method_textbox.Text) + "\", "; 
DST += "oauth_timestamp=\"" + PercentEncode(oauth_timestamp_textbox.Text) + "\", "; 
DST += "oauth_token=\"" + PercentEncode(access_token.Text) + "\", "; 
DST += "oauth_version=\"" + PercentEncode(oauth_version_textbox.Text) + "\", "; 

authorization_header_textbox.Text = DST;
OAuth oauth_consumer_key="xvz1evFS4wEEPTGEFPHBog", oauth_nonce="kYjzVBB8Y0ZFabxSWbWovY3uYSQ2pTgmZeNu2VS4cg", oauth_signature="tnnArxj06cWHq44gCs1OSKk%2FjLY%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1318622958", oauth_token="370773112-GmHxMAgYyLbNEtIKZeRNFsMKPR9EyMZeS9weJAEb", oauth_version="1.0"