Twitter and Skype apps crash! == Nullpointer Exception – TestObject

Posted by | August 29, 2013 | Company, Research | One Comment

We took Twitter, Skype and Facebook for a TestObject Stress testing ride and our monkey made them go BANANAS!

We looked at some of the most used Android apps in the market and took them through an intense stress test, trying to see who is the fittest app of them all! Who will be Android’s next app superstar?

Test Scenario:

  • Android 4.1
  • 1000 Random Inputs using TestObject’s MonkeyRunner (Key, Buttons, Clicks)
  • Logged in User
  • Android Twitter, Facebook and Skype Apps
    =  Who crashed?

That’s how the mass looks like: (Click here for Twitter’s full log)

77% of app users are worried! why?

Applications can endure a lot, one would hope, but the reality in the market shows that apps can be as unpredictable as their users’ actions. This is the reality of mobile apps today, otherwise there is no other reason that 77% of mobile app purchasers would worry about the apps they buy (as seen here). With fear of bugs, crashes, performance, speed and numerous other user, device or weather related reasons, users will probably think twice before downloading an app.

Developers are thinking of all possible actions that users might take making sure their app could manage any event. All possible errors that might come up in the apps they are working on are being taken into consideration.

Though, the ones to fearlessly explore the wilderness of user’s behaviour jungle are testers that are designing numerous test cases. Trying to forecast what users might be doing with your app once it’s out in the wilderness might be a tedious task but should never be dismissed. Thought, even the most creative of testers won’t be able to find the specific chain of actions that no one have thought of that would make an app lose it.

Monkey testing

To test or not to test?

To test or not to test?

So how can we simulate what hopefully millions of users will do after downloading your app? That’s where our Monkey testing comes into play. Monkey tests are an automated on going tests that send numerous random actions to the system. All this random pounding on the device might accidentally hit the right combination of letters, commands and actions that will touch our’s app achilles’ heel.

Based on the Infinite monkey theorem, we like to think of the Monkey tests in the following way: based on the idea that you had a million monkeys typing on a million keyboards for a million years, statistically speaking, they might eventually write a Shakespearean play. Meaning, eventually they might find that magic chain of actions that would make an app go bananas!

Crashing the Twitter and Skype app with TestObject

MonkeyRunner, TestObject‘ Monkey testing is integrated using Google’s random testing tool. Using MonkeyRunner is so easy and can be done with a single click. (API info)
A simple click worth 1000 events

A simple click worth 1000 events

So, what we basically did is simply logging in with an account we have created for our MonkeyRunner and gave it the freedom to execute as much as 1000 actions randomly. Our user was so enthusiastic and soon enough shared some tweets, retweeted, followed users and skyped with some imaginary friends – until suddenly the app crashed!

With more than 500,000,000 (500 hundred million) twitter users and an average of 400,000,000 tweets a day, one would assume that Twitter’s app will be bullet proof. Meaning that all the possible scenarios have been cleared and there’s substantially nothing this app won’t be able to take.

But ho boy! Twitter was the first one to fail the race (Full log). Next one was Skype who showed some hesitation during the test and finally gave up crashing with a fierce K.O. (Full log)
ScreenShot of Skype crashes TestObject Stress Testing android app

TestObject recorder interface, Houston we’ve got a problem, Skype is struggling!

What does Nullpointer Exception crash mean?

A null pointer exception is an indicator that there is a malfunction in a program’s code. Typically, the Java program (and consequently, the programmer) attempted to access or reference a Java object that did not exist.

As an analogy: You own a piece of land but didn’t build anything on it yet. Accidentally you started giving out your future house’s address saying “That’s my home address”. The postman, your friends and family are looking for the house but can’t find it in that address. This is the equivalent of getting a NullPointerException.

What does it mean for Twitter and Skype?

While the code was developed, some areas in it were neglected, leaving blank spots that each time reached cause the apps to crash. As seen in our testing example, our monkey easily found such lank spots. Finding those bugs doesn’t require a lot of resources and can be done in less than 5 minutes.

If Facebook doesn’t crash, why should other apps crash? User experience can determine the success or failure of an app and help or harm a brand’s reputation. As a result, almost nine out of ten mobile app users associate negative feelings with brands that have poorly performing mobile apps.

Who’s the fittest of them all? FACEBOOK!

Using Facebook on Android can be nerve wracking sometimes, but, as our monkey test showed, something in this app is just right! 

Not only that Facebook’s Android app run by our user Koko Runner has made it through thousands of random comments, it is actually being extremely social, proving how easy it is even with simple random actions to friend people, tag them around the world and chat, even when all you have to say is: “q*!s& *”. (Don’t forget to friend her!)

Now it’s time for you to Stress test your own apps and make sure they can make it in the wilderness! Don’t forget to run those tests regularly as they have the potential to press buttons you’ll never think of. Click here to learn more about TestObject and learn how to create a test in less than 5 minutes!

Let the monkey games begin!
Stress Testing Monkey Runner

What’s on your mind, koko? MonkeyRunner in action

One Comment

Leave a Reply

Your email address will not be published.

We send newly fresh and interesting content to your inbox twice a month!
First Name
Email address
It's spam free!