Build a Twitter Search App: Fetch & Parse the Twitter JSON Feed

This two-part tutorial series will introduce you to the fundamentals of working with RESTful web services using the Android SDK. Along the way, you’ll learn how to perform searches against the public Twitter API!

In this tutorial we will use AsyncTask to fetch the JSON tweets, parse them, then display them within the user interface. Although the code in this tutorial is specific to Twitter, the principles involved apply to fetching many other Web feeds and APIs using a RESTful architecture.

This tutorial series on Android Twitter Search is in two parts:

• Part 1: Build the Search Request
• Part 2: Fetch & Parse the Twitter JSON Feed

Step 1: Create an Inner AsyncTask Class

We are going to use an AsyncTask to carry out the fetching, parsing and displaying of the tweet feed on the user search query. By using AsyncTask, we can shift processing into a background thread, with the results still accessible to the app’s UI for display. In your Activity class, add an inner AsyncTask class after the searchTwitter method:

private class GetTweets extends AsyncTask<String, Void, String> {
}

The AsyncTask class is abstract, which means you have to create a subclass in order to use it. Such subclasses must implement one particular method which we will add next. In the meantime, Eclipse will display error messages (just ignore them for now).

The AsyncTask uses three generic types as you can see from the opening line of the class declaration. The first one specifies parameter type – in our case this is a string as we are going to pass the URL string we created. The middle type is for progress units, which we will not be using in this tutorial so we simply specify void. Progress units can be used to indicate the progress of background tasks. The third type is another string, which the background process returns. In our case this is going to be the retrieved JSON feed text.

Step 2: Perform Background Processing

Inside your AsyncTask class, add the following method, which we override from the parent class:

@Override
protected String doInBackground(String... twitterURL) {
}

The parameter type is string, matching the first type we indicated in the class declaration opening line. This string is going to be the Twitter Search API URL we built last time. Inside this method, we specify what background process we want the AsyncTask to carry out. The method also returns a value of type string, to match what we specified as the third parameter in the class declaration opening line. This returned string will be received as parameter to a second method, which we will implement later.

Tip: If you are unfamiliar with abstract classes or generic types, don’t worry – what we are doing here is really just following a pattern. As with much of the Android platform, you can learn how to use these resources from the examples in the Developer Guide and, in this case, the API Reference.

Step 3: Send the Request

The Twitter search result is going to be a string, which we will build from the response we receive. Inside the doInBackground method, start by creating a String Builder:

StringBuilder tweetFeedBuilder = new StringBuilder();

Although we are using a single URL string, the method is going to receive an array of objects of the type specified (string), so let’s loop through it:

for (String searchURL : twitterURL) {
}

In reality, the loop will only iterate once, but you could extend your code to fetch multiple search feeds, so we will use a loop for now. Inside the loop, create an HTTP Client for executing the request:

HttpClient tweetClient = new DefaultHttpClient();

Now we need to catch I/O exceptions, as is invariably the case when you attempt to fetch data from anywhere outside your application. Add try and catch blocks:

try {
}
catch(Exception e) {
	tweetDisplay.setText("Whoops - something went wrong!");
	e.printStackTrace();
}

If there is an Input/Output error, we simply write a message to the user interface. Inside the try block, create an HTTP Get object to issue the request, passing the search URL:

HttpGet tweetGet = new HttpGet(searchURL);

Now we can go ahead and execute the request, storing the results in an HTTP Response object:

HttpResponse tweetResponse = tweetClient.execute(tweetGet);

We will be able to use the response object to access the content and status of the response message received from Twitter.

Step 4: Process the Response

Before we attempt to process the response message, let’s check the status. Still inside the try block:

StatusLine searchStatus = tweetResponse.getStatusLine();

If the response is OK we can carry on and attempt to parse the tweets, otherwise we will output an error message to the user:

if (searchStatus.getStatusCode() == 200) {
}
else
	tweetDisplay.setText("Whoops - something went wrong!");

Inside the if statement block, we can now retrieve the HTTP Entity and message content as an Input Stream:

HttpEntity tweetEntity = tweetResponse.getEntity();
InputStream tweetContent = tweetEntity.getContent();

Now we can begin bringing the message content into the program, using an Input Stream Reader rather than a Buffered Reader for managing the incoming data:

InputStreamReader tweetInput = new InputStreamReader(tweetContent);
BufferedReader tweetReader = new BufferedReader(tweetInput);

Let’s read in the data a line at a time, appending each line to the String Builder we created:

String lineIn;
while ((lineIn = tweetReader.readLine()) != null) {
	tweetFeedBuilder.append(lineIn);
}

This will keep adding to the String Builder until all of the received data has been processed. Now move to the bottom of the doInBackground method, after the catch block – return the string we’ve built the results into:

return tweetFeedBuilder.toString();

That’s the doInBackground method complete – it carries out the fetching of the tweet feed and importing it into the app, so now we can parse and display it.

Step 5: Parse the JSON Tweet Feed

Now that the AsyncTask has retrieved the tweet feed, we can parse it and display it in our application UI. To do this we can implement another method of the AsyncTask class, added after doInBackground, still inside the AsyncTask class declaration:

protected void onPostExecute(String result) {
}

Notice that this method receives a string parameter, which is what we specified as the third type in the opening class declaration line and what we returned from doInBackground in the last step. The string is the JSON text representing the recent tweets on the user search term retrieved from Twitter. We will need to parse this text to access the content of the tweets and display them in the UI.

We will be building the tweet text into a string for display, so start the onPostExecute method by creating another String Builder:

StringBuilder tweetResultBuilder = new StringBuilder();

The JSON processing methods can throw exceptions, so add try and catch blocks next:

try {
}
catch (Exception e) {
	tweetDisplay.setText("Whoops - something went wrong!");
	e.printStackTrace();
}

You should be noticing a pattern here – as with any program in which you attempt to retrieve and process external data, you must pay careful attention to handling potential errors.

Inside the try block, create a JSON Object, passing the JSON text string:

JSONObject resultObject = new JSONObject(result);

Inside the JSON text string is an array containing the tweets, plus some other data – carry out a Twitter query by pasting the search URL into your browser address bar as we did last time, for example:

http://search.twitter.com/search.json?q=android

You may need to copy and paste the resulting text into a text editor to read it effectively. If you look through the JSON text, you will see the following section, which marks the beginning of the tweet array:

"results":[

We need to retrieve this tweet array, using its name: “results”. Get the “results” array from the JSON Object:

JSONArray tweetArray = resultObject.getJSONArray("results");

Now we can loop through the tweets and prepare them for display.

Step 6: Iterate the Tweet Array

If you look again at the JSON text retrieved through the browser (or, optionally, within Eclipse if you write it to the Android Log while running your app), you can see the content returned for each tweet in the array. In this tutorial we will only write out the username of the account tweeting and the content of the tweet itself. You can extend the app to include any of the information you want. In the text returned from Twitter, the username is represented as “from_user” and the tweet content is named “text” – look at the JSON content to verify this and to see what other data items are in there.

In your try block, after fetching the tweet array, add a for loop to iterate through the tweets:

for (int t=0; t<tweetArray.length(); t++) {
}

Inside the loop, get each item as a JSON Object:

JSONObject tweetObject = tweetArray.getJSONObject(t);

Now retrieve the username from this object, appending it to the String Builder along with some additional text for display purposes:

tweetResultBuilder.append(tweetObject.getString("from_user")+": ");

The JSON Object class provides a getString method for this purpose; there is also an alternative get method. Now retrieve and append the tweet content:

tweetResultBuilder.append(tweetObject.get("text")+"\n\n");

Step 7: Display the Results

Now move to after the catch block inside the onPostExecute method. If the processing has worked, we display the result in the Text View, otherwise we display an error message:

if(tweetResultBuilder.length()>0)
	tweetDisplay.setText(tweetResultBuilder.toString());
else
	tweetDisplay.setText("Sorry - no tweets found for your search!");

Our AsyncTask class is now complete.

Step 8: Instantiate & Execute the AsyncTask

We have an AsyncTask to carry out our processing; we just need to call on it. Move back to your searchTwitter method, inside the try block and after we built the search query URL string. Create an instance of the new AsyncTask class and call the execute method on it, passing the URL string:

new GetTweets().execute(searchURL);

Notice that we pass a string, which is what we specified in the AsyncTask declaration and doInBackground method.

That is the basic app complete – you can test it in the emulator and on Android devices. Try it with different search queries such as hashtags. You can of course extend the app to display more information from the tweets and even to link to Twitter on user clicks. As things stand, the user can scroll through the returned tweet feed and carry out repeated searches as they wish.

Conclusion

In these two tutorials we have explored fetching tweets from the Twitter Search API. You can use the same design patterns to fetch data from other Web resources. If your apps are going to involve more complex, ongoing data retrieval processes, you may wish to look into Content Providers and optionally Services to maximize on efficiency. As with any external data-based operations, you cannot assume that retrieving the information you need will be successful, so a level of error handling is essential. The ability to access Internet data is of course one of the key benefits to mobile computing, so these techniques are a vital component in any Android developer’s toolbox.