Monday, November 29, 2010

ADT, Eclipse, and The Mystery of The Frozen Auto-Complete

I like Eclipse...as IDEs go, it's my IDE of choice.  For general day-to-day development, it covers so much straight out of the box, but then along come the wide variety of plugins that are available, and all of a sudden, Eclipse helps make life as a software developer just that little bit sweeter.

And when it comes to Android, the Android Development Toolkit (ADT) plugin is about as nice as they come, from quick and simple project setup, through to easy integration with the Android Debug Bridge, the ADT plugin really does deliver in simplifying Android development.  There has been, however, a slight problem that has left me downright frustrated many times, and that is the terrible slowdown in performance when it comes to auto-complete.

How terrible is terrible?  Well try anything from 5 to 20 seconds or so (on my machine) waiting for the auto-complete box to populate...it doesn't exactly sound like a lot, but it's more than enough to be extremely annoying, it breaks your flow, and it really makes you want to hurl your mouse at the screen as you realize that there's absolutely nothing that you can do about it but wait and seethe.

From what I've read, this is an issue that only affects Eclipse 3.6, and so the interim solution offered was simply to revert back to 3.5.  The only problem with that solution is that I rather like Helios and the increased stability that comes along with it; I really don't want to have to go back to Galileo if I can avoid it.  And I guess others in the same situation agree, because lo-and-behold...a workaround has been found.

How to Have Your Cake And Eat It
As it turns out, the ADT classpath container is looking for source in the standard Android SDK installation directory...except the standard Android SDK doesn't come with the source.  This means that Eclipse attempts to cache entries that are in a directory that doesn't exist.  Everytime.

And so the obvious workaround is to simply download the source, create the directory, and let Eclipse be happy.  To do this, you simply need to do the following:

And that's it...problem solved!  For the guys on this thread where they provided this solution, I offer my utmost thanks.  As do my mouse and monitor. :)

Saturday, November 6, 2010

C2DM: On The Receiving End

My last couple of posts regarding Android's Cloud to Device Messaging framework were purely from the perspective of the server, in that we first had to obtain an authentication token using ClientLogin, and then we actually used the C2DM service to send a message.  But what about the client side?  What's needed from our Android app in order to receive these near-realtime notifications?  Well as it turns out, we simply need to do a couple of things:
  • Register with the C2DM framework to receive messages
  • Pass the registration ID that we get to the server
All in all, that's a pretty small to-do list, don't ya think? :)

Warming Up
Of course, before we can even start to think about using the C2DM framework, we have to do a few things first to make sure our app is fully prepared.  As the ability to use the C2DM framework arrived in the Froyo release, we have to make sure that our application requires version 2.2 or later by specifying it in the manifest:


    
    

    ...

Next, we need to make sure that we have our permissions set up properly. In order to receive notifications, the app must have the following permissions:
  • com.google.android.c2dm.permission.RECEIVE - allows us to register and receive messages.
  • android.permission.INTERNET - we need this so that we can pass the registration key to the third-party server that will be sending us notifications (it needs to know who to send the messages to, right?).
  • <applicationPackage>.permission.C2D_MESSAGE
  • - stops other applications from trying to register as our app in order to steal our messages.
This should be straightforward enough, so let's have a go:


    
    
    

    
    

    
    

    ...

So far, so simple, right?  Good. :)  But, we're not quite out of the woods as far as the permissions go just yet; we still need to make sure that we only receive messages from the C2DM framework, otherwise any other app could send us notifications (which in all likelihood we probably don't want to have happen).  To achieve this, we'll specify a receiver that has the following permissions (each with a category that matches our applicationPackage):
  • com.google.android.c2dm.intent.REGISTRATION
  • com.google.android.c2dm.intent.RECEIVE
So what does this look like?  Well, something like this:


    
    
        
        
            
            
        
        
        
            
            
        
    

    ...

Here, we're specifying a receiver (my.example.app.C2DMReceiver) that will perform the handling of the registration ID and any messages that are sent our way.  Again, all in all relatively straightforward...don't ya just love Android? :)

Registration
Now that we have the permissions sorted, it's time to register our app so that it can receive messages...and here's where it gets super-silly-easy.  All we need to do is create an Intent (com.google.android.c2dm.intent.REGISTER) and pass the email address of the sender (which will be the email address that we set up for the server to use) and our application ID:
public static void register(Context context, String senderEmail) {
    Intent registrationIntent = new Intent("com.google.android.c2dm.intent.REGISTER");
    registrationIntent.putExtra("app",
            PendingIntent.getBroadcast(context, 0, new Intent(), 0));
    registrationIntent.putExtra("sender", senderEmail);
    context.startService(registrationIntent);
}
Wow, pretty easy huh?  (Don't worry, I can sense the nodding) :)  So what happens after we've fired off that Intent?  Well this is where the receiver that we declared in the manifest comes in to play:
public class C2DMReceiver extends BroadcastReceiver {

    @Override
    public void onReceive(Context context, Intent intent) {
        if (intent.getAction().equals("com.google.android.c2dm.intent.REGISTRATION")) {
            String registrationId = intent.getStringExtra("registration_id");
            String error = intent.getStringExtra("error");
            String removed = intent.getStringExtra("unregistered");
            
            if (error != null) {
                // Perform error processing
                ...
            } else if (removed != null) {
                // Perform any post-unregistration cleanup
                ...
            } else if (registrationId != null) {
                // Send registrationId to third-party server
                ...
            }
        }
    }

}
Our receiver simply checks the Intent to see if it's a com.google.android.c2dm.intent.REGISTRATION Intent, and if it is then it grabs the registration_id and, barring any errors (or if this is a response to our application unregistering itself), it goes ahead and passes the received registration ID to our third-party server.  This now ensures that our app is primed and ready to start receiving messages.

Receiving Messages
Handling any messages that come in is done using the same means that we used for receiving the registration ID.  This means we're going to use C2DMReceiver again...specifically, we're going to be using exactly the same method:
public class C2DMReceiver extends BroadcastReceiver {

    @Override
    public void onReceive(Context context, Intent intent) {
        ...
        if (intent.getAction().equals("com.google.android.c2dm.intent.RECEIVE")) {
            // Yay, we've received a message...grab the
            // parameters and do awesome stuff!
            String myParam1 = intent.getStringExtra(SOME_PARAM_1);
            String myParam2 = intent.getStringExtra(SOME_OTHER_PARAM);
            // Perform any processing
            if ("some_condition".equalsIgnoreCase(myParam1)) {"
                doSomethingAwesome(myParam2);
            }
            ...
        }
    }

}
And in a nutshell, that's it...it just about couldn't get any simpler.  Really.  But I'm not quite done with C2DM yet; there are some things that are handy to know when it comes to integration testing...

Testing, Testing, 1-2-3
It's relatively simple to test things out just by using the emulator alone.  As always though, there are some things that we need to make sure of first, and these are:
  • The AVD (Android Virtual Device) must use the package "Google APIs by Google Inc., Android API 8", so make sure you have it installed.  It has to be "Google APIs" and not simply "Android API 8" or it won't work.
  • C2DM requires a valid signed-in Google account in order to establish a connection with the device, which means that the emulator also needs to have a valid Google account that is logged in.  Simply go to Settings > Accounts & Sync > Add Account and provide the account details.
  • If the application server that you will be sending the messages from is running locally, make sure you're not trying to use localhost when sending the registration ID in your app.  Instead, use address 10.0.2.2 to hit the local server instance and save yourself some noodle-scratching. :)
Happy messaging!

Thursday, November 4, 2010

C2DM: Sending Messages

This post follows on from my previous post where we called ClientLogin to obtain an authentication key.  The reason why we needed this key is so that we can do something really cool...send out push notifications to our Android applications.  And similar to that post, the code that is provided here should be just enough to produce a result; a real application will need to implement functionality to handle the different responses that the C2DM service can return, as well as handle things like exponential backoff for when the service is unavailable.

The basics of sending a message are really quite simple; all we need to do is send a well formed HTTPS request to https://android.apis.google.com/c2dm/send and pass along the registration ID of the device that we wish to send a message to, our authentication token, and the payload of the message itself.  So just like we did before when obtaining the authentication token, we start off with the HttpURLConnection object:
HttpURLConnection connection = null;
try {
    URL url = null;
    try {
        url = new URL("https://android.apis.google.com/c2dm/send");
    } catch (MalformedURLException e) {
        // Exception handling
    }
    connection = (HttpURLConnection) url.openConnection();
    connection.setDoOutput(true);
    connection.setUseCaches(false);
    connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
    connection.setRequestProperty("Authorization", "GoogleLogin auth=" + token);
} catch (Exception e) { 
    // Exception handling
}
Once again, I'm using HttpURLConnection as my application is going to be hosted using Google App Engine, and HttpsURLConnection is simply not allowed.  Also, notice how we need to send our authentication token; it needs to be prefixed with the String "GoogleLogin auth=".

Next up, we need to set the parameters that we're going to pass to the C2DM service.  For this, we need to include the registration id that the device received from the C2DM service when the device first registered itself (yes, that means the device will also need to pass that id to us as soon as it gets it).  We also need to specify our message payload in the form of key-value pairs, with the key prefixed with "data.":
StringBuilder sb = new StringBuilder();
addEncodedParameter(sb, "registration_id", deviceRegistrationId);
addEncodedParameter(sb, "collapse_key", "goobr.blogspot.com.someKey");
addEncodedParameter(sb, "data.payload1", "payload1 message data");
addEncodedParameter(sb, "data.payload2", "payload2 message data");
addEncodedParameter(sb, "data.anotherPayload", "even more message data");
String data = sb.toString();
One thing to note is the collapse_key parameter.  This is simply a required key that will be used to collapse our messages so that if the device is switched off, it won't get a ton of messages when it comes online again...in this situation, only the last message will actually be received by the device (although C2DM makes no guarantee on message order, so it might not actually be the last message that was sent).  And just like before, we're using the same addEncodedParameter() method to format our parameters into a String:
public static void addEncodedParameter(StringBuilder sb, String name, String value) {
    if (sb.length() > 0) {
        sb.append("&");
    }
    try {
        sb.append(URLEncoder.encode(name, "UTF-8"));
        sb.append("=");
        sb.append(URLEncoder.encode(value, "UTF-8"));
    } catch (UnsupportedEncodingException e) {
        // Exception handling
    }
}
Finally, we come to the actual sending of the message itself, and processing the response:
try {
    DataOutputStream stream = new DataOutputStream(connection.getOutputStream());
    stream.writeBytes(data);
    stream.flush();
    stream.close();

    switch (connection.getResponseCode()) {
    case 200:
        // Success, but check for errors in the body
        break;
    case 503:
        // Service unavailable
        break;
    case 401:
        // Invalid authentication token
        break;
    }
} catch (IOException e) {
    // Exception handling
}
If the service is unavailable, then we need to check for (and honor) any Retry-After header in the response, and failing that, we need to implement exponential backoff (that is, keep doubling our wait time before trying the request again).  Also, even if we get a 200 response code, that doesn't mean that the message went through.  We still need to check the response body for any of the following:
  • Error=QuotaExceeded
  • Error=DeviceQuotaExceeded
  • Error=InvalidRegistration
  • Error=NotRegistered
  • Error=MessageTooBig
  • Error=MissingCollapseKey
On the first two errors, there's not a lot we can do but wait for a while and try again...it just means that too many messages have already been sent.  For the InvalidRegistration and NotRegistered errors, we need to make sure that we stop sending messages to this registration id as the device has either unregistered itself, uninstalled our Android app (uh-oh!), or simply switched off notifications.  And for the final two errors, well we simply need to look at our code and make sure that:
  • the message is 1024 bytes in size or less and
  • we include a collapse_key in the request
One final thing to note is that the response may include an Update-Client-Auth header from time to time, and all this means is that the authentication token that was used to send the message is about to expire, and so the C2DM framework has generated a new one that we should start using instead.

And that's all there is to it really.  See...told you it was really quite simple! Have fun sending push notifications to your Android apps. :)

Sunday, October 31, 2010

C2DM: Using ClientLogin to Obtain an Authentication Token

As explained in a previous entry, Google's Cloud to Device Messaging (C2DM) framework allows our Android applications to make use of the connection that Google services establishes with Android phones, meaning we can implement push notifications more easily and make our applications near real-time.  But where do we start?  Well, there's two sides that are needed...there's the Android device side, and there's the server side.  This blog entry is going to cover what is needed on the server side so that we can then start sending messages to the Android devices.

Setting The Stage
The first step is to create a new Google email account.  In order to send notifications, the web server needs to be logged in to a valid Google account, and so we simply set a new one up that is specifically for the server application's use only.

Once that has been set up, we then need to request access to the service itself.  At the time of writing, C2DM is in Google Labs, meaning it's necessary to signup to use it; this is easily done via this simple form.  You'll need to provide the package name of the Android app that will be receiving the messages, and you'll need to specify the Role Account Email as the new email account that we just set up.  From my experience, the registration process is quick and painless, with a confirmation email arriving within minutes.

Authenticating
With those prerequisites complete, our server will need to authenticate itself by using an authentication token, and we get one of those from ClientLogin.  Authentication tokens are used as it would be a bad idea to store the actual account credentials on the server, but if it's a bad idea to store the credentials, how does the app log in in the first place to obtain this authentication token?  Well, what we can do is create a very simple servlet that allows us to enter the email and password manually...this then gets sent to ClientLogin for validation, and if the credentials are good, then we should receive back an authentication token that is good for about a day (don't worry, the C2DM framework will provide us with a new token when the current one is about to expire).

To authenticate, we first need a connection object to https://www.google.com/accounts/ClientLogin:

HttpURLConnection connection = null;
try {
    URL url = null;
    try {
        url = new URL("https://www.google.com/accounts/ClientLogin");
    } catch (MalformedURLException e) {
        throw new ServletException(e.getCause());
    }
    connection = (HttpURLConnection) url.openConnection();
    connection.setDoOutput(true);
    connection.setUseCaches(false);
    connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
} catch (Exception e) {
    // Exception handling
}

The more observant may have noticed the use of HttpURLConnection instead of a HttpsURLConnection object.  Well, that's because I'm using Google App Engine to host my app, and HttpsURLConnection isn't a permitted class (although from my understanding, under the covers App Engine uses HttpsURLConnection for secure connections).
 
Now that we have our connection object, it's time to pass in the credentials that we wish to authenticate with:

StringBuilder sb = new StringBuilder(); 

addEncodedParameter(sb, "accountType", "GOOGLE");
addEncodedParameter(sb, "Email", request.getParameter("account"));
addEncodedParameter(sb, "Passwd", request.getParameter("password"));
addEncodedParameter(sb, "service", "ac2dm");
addEncodedParameter(sb, "source", "myCompany-demoApp-1.0.0");
String data = sb.toString();

DataOutputStream stream = new DataOutputStream(connection.getOutputStream());
stream.writeBytes(data);
stream.flush();
stream.close();

As you can see, this is where we're passing the account type, the email address, and the password.  The service parameter must be the Android C2DM service ("ac2dm"), and the source parameter should follow the format of "companyName-applicationName-versionID".  Oh, and the addEncodedParameter() method that I'm using?  That's just a simple helper method as shown below. :)

public void addEncodedParameter(StringBuilder sb, String name, String value) {

    if (sb.length() > 0) {
        sb.append("&");
    }

    try {
        sb.append(URLEncoder.encode(name, "UTF-8"));
        sb.append("=");
        sb.append(URLEncoder.encode(value, "UTF-8"));
    } catch (UnsupportedEncodingException e) {
    }
}
After we've sent the credentials, the next step is to process the response.  Basically, we're looking for a line in the response that starts with "Auth=", as this contains our token.  We're also going to capture any errors that we may have received:

BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream())); 

String line;
String tokenIdentifier = "Auth=";
String errorIdentifier = "Error=";
String token = null;
StringBuilder errors = new StringBuilder();
while ((line = reader.readLine()) != null) {
    if (line.startsWith(tokenIdentifier)) {
        token = line.substring(tokenIdentifier.length());
    } else if (line.startsWith(errorIdentifier)) {
        String error = line.substring(errorIdentifier.length());
        errors.append(error + System.getProperty("line.separator"));
    }
}
reader.close(); 
And that's about it.  Of course, this is just the bare minimum that is required to get an authentication token, as the code doesn't even attempt to handle CAPTCHA challenge responses.  In this event, our servlet must then display the CAPTCHA image url (http://www.google.com/accounts/ + CaptchaUrl) and prompt for the additional user input.  It can then respond to the challenge using another call and providing the additional headers logintoken and logincaptcha.

But for now, this should be enough to get a valid authentication token, and with the token in hand, we can start to send messages to our Android devices. :)

Friday, October 29, 2010

Android C2DM: Push It!

Let's imagine an email application for a cell phone; it obviously needs to notify the user whenever a new email has arrived...but how does it actually know when new emails are available?  Well, it could simply ask the server continously if there are any new emails (poll), or we could have the server notify us when any new emails have arrived (push).  But which one should we be using?

As a developer, polling is the easier solution to implement.  We simply have the client wake-up every now and again, and have it call home to the server to check if there's anything new to report...much akin to the child that keeps asking "Are we there yet?".  However, for cell phone applications this approach introduces the issue of polling frequency; poll too frequently, and we're wasting valuable system resources (battery and bandwidth) when for the most part, the answer will be the same.  But poll too infrequently, and we risk the user receiving stale data.  We can skirt around this issue by simply handing the decision to the user via a preference setting...or we could use the push model instead.

When it comes to mobile devices, the push model is, to me at least, a much better solution; it follows the Hollywood Principle of don't call us, we'll call you, meaning that our application only needs to do something when the server actually tells us that new information is available.  This saves those precious system resources, but it still comes at a cost...namely that of increased complexity.  Thankfully, Google has already addressed this with their C2DM service, and being the good fellows that they are, they're opening the service up so that other Android applications can take full advantage of near real-time notifications.

So how does it work?

It works via an existing Google services connection, meaning the user needs to be signed in to a valid Google account on their Android device.  The application can then register itself to receive notifications via this connection, and the C2DM service will respond with a unique key for the application to use.  The application then has to send this key to any third-party server that it wants to receive notifications from, and viola...any time the server needs to notify that particular application on that particular device of something, it simply asks the C2DM service to send out a message.  When the device receives the message, Android knows where to route the message to...even waking up the application if necessary, and as there's no required UI for the messages, the app is free to do whatever it likes.  It may display a message to the user, or it may perform some silent task in the background, like synchronizing data.  How neat is that?

But what of the caveats?
  • The service is only available as of Android 2.2.
  • The service requires the user to be signed in to a valid Google account.
  • Messages can be no more than 1024 bytes in size.
Requiring at least Android 2.2 will eventually become a non-issue as time goes by.  And restricting the message size to 1024 bytes ensures that servers are simply saying "Hey, data is available...come and get it", rather than  trying to send the actual data itself.  But what about requiring a Google account?  Well, that's an extremely small price to pay in my mind...if the user doesn't have (nor want) a Google account, even if it's just for the sake of receiving push notifications, then they'll just have to go to their preference settings and decide for themselves what a good polling frequency should be. :)

Wednesday, October 27, 2010

Hello, World!

Recently I've been working on a personal Java project of mine, and I've been having a lot of fun slowly watching the pieces come together. The real fun, if I'm completely honest, has been mainly due to the fact that a lot of the frameworks that I've used are new to me, meaning there's been a lot to learn, and also there's a lot to improve upon when I no doubt refactor everything later.  Harnessing the power of bullet points, here's what the project has involved so far:
As you can see from the above list, it's an Android project, and my first one at that.  Armed with a book, some simple Googling skills, and a little head-scratching, I've so far managed to concoct the following:
  • Android device can register itself with Google's C2DM service
  • Android device can call the web service (using Restlet) to request certain functionality, and pass the C2DM registration key to it so that the device can receive near real-time notifications once tasks have been completed
  • Web service can persist the information using Google's BigTable, and can service other requests
  • Web service can send notifications to registered Android apps using C2DM
Of course, I say that all of these pieces are working, but I'll hold my hand up and readily admit that I've only tested things so far in Eclipse using the Android emulator. I don't really intend to test fully using my own device (the rather fantastic Droid X) until I have everything working for sure in the emulator first...don't ask why, it's just how I feel like doing things. :)

So what do I have left to do?  At this point, there's still a lot to be done on the device side of things...mainly UI, and of course I still have to deploy the web service to Google App Engine, but all of this is to come.

So that's pretty much what I'm going to be writing about, and probably starting with C2DM. As all of this is a new playing area for me, I'm pretty sure that what I say will be subject to subsequent corrections; but that's kinda the point...we all learn from our mistakes, and if I can write about something, and if that something actually makes some sense to others, then I may just have achieved a basic understanding of that subject. :)