Run massively scalable performance tests on web, mobile, and APIs

Request a Demo
Jun. 28th, 2018

Mobile Testing on Real Devices with BlazeMeter: Learn How

69% of digital media today is consumed on mobile platforms, according to comScore. This is a 99% increase in digital media consumption on mobile platforms compared to 2013. This mobile revolution has forced companies to focus on a mobile first strategy - either by building mobile apps or mobile-optimized sites.

 

Mobile apps and mobile sites need performance testing. However, mobile applications are bit harder to test than web-based applications. BlazeMeter enables developers to simply run load and functional tests on mobile apps, as part of their shift left and continuous integration performance testing strategies.

 

Mobile testing with BlazeMeter requires only 3 steps:

  1. Package your app and test scripts
  2. Run a simple command
  3. Analyze results

 

This post will provide a step by step explanation of how to run a functional mobile test with BlazeMeter on a real mobile device. Currently, our mobile testing solution is in Beta, so we would appreciate your feedback after completing the steps in this blog post, here.

 

You can either run the test for your own app, or follow the demo we created for you. Steps will be shown for both ways. The demo shown in this blog post will use a sample ios application and run a basic Appium test. The demo test will show how to run a functional GUI test for an iphone app that is purchasing a ticket on the BlazeDemo website. You will be able to download the test results as well as the video recording. BlazeMeter mobile testing also works for android.

 

Let’s get started.

 

PreRequisites

 

 

 

 

 pip install -r requirements.txt 

 

 

Preparing Your App

 

If you are testing your own app, follow these steps. If you’re following the demo, skip to “Running the Mobile Functional GUI Test” section.

 

Step 1 - Package Your Application for the Test

 

If you are using your iPhone application, make sure that it is signed for ad-hoc deployment. We support Android apps but are working to build APIs for Android and also allow device selection.

 

Add the mobile app that you want to test to the resources directory. The bundled mobile app ipa uses the source code from this repository.

 

mobile app testing

 

Step 2 - Zip Your Test File

 

Add your test to the test-files/python/tests directory. Replace the existing blaze_demo_test.py script. Make sure you add your additional requirements for your Appium test in the requirements.txt file under the test-files/python directory.

 

Since we are running this test on a real device that is pointing to your application, the only configuration you need to pass in the desired capabilities is the bundle id.

 

blazemeter mobile demo

 

Now, bundle the tests by zipping them with the create_bundle script provided in the python directory.

 

./create_bundle.sh 

 

This zipped artifact is what BlazeMeter uses to upload and run the tests.

 

Running the Mobile Functional GUI Test

 

Go to the root of the cloned directory and type the following command

 

python mobile-tester.py --api_key=<YOUR_API_KEY_HERE> --api_secret=<YOUR_API_SECRET_HERE> --name MobileTest

 

This will create and initialize the test.  After a few seconds, the program will then prompt you for an iPhone application. The default application is already pre-selected. Hit enter, or, if you are using your own application, pass the path to the application.

 

Enter application path, hit `enter` for default (resources/BlazeDemo.ipa):

 

mobile functional gui test

 

The next prompt is to enter the zipped test files. The default is already selected so go ahead and hit enter. If you want to pass your own test files, pass the path to your zipped file.

 

Enter zipped test files path, hit `enter` for default (test-files/python/test_bundle.zip):

 

gui mobile testing

 

The system will now install the app on a real device and run the tests. This will take about 5 minutes or so. The output will include a video recording of the entire test, screenshots and the tests results.

 

mobile gui testing blazemeter

 

mobile testing with blazemeter

 

Analyzing Your Mobile GUI Test Results

   

Video

 

The response contains a JSON results object that has a list of entities. Each entity has a type, like VIDEO or DEVICE_LOG, as well as a URL that hosts the video or log file.

 

The video captures how the application ran under test on the device. This is a very useful tool for debugging and understanding how your test ran.


 

 

JUnit XML Result

 

The response also contains a results object that has a list of entities. Look for the type APPIUM_JAVA_XML_OUTPUT. This file contains the results of the test run in JUnit-XML format. This format makes it easy to integrate into CI systems like Jenkins. In addition, you will be able to see the time it took to run the scenario (34.998 in our case). This KPI is especially important when you want to compare the GUI test results to load testing results.

 

mobile testing results

 

We are also working on the UI for the results on Blazemeter platform. This is what the results will look like. If you are interested, please reach out to us so that we can get you access.

 

android and ios testing

 

What about more frameworks and languages?

 

We are working on this, please let us know what you want. We also plan on adding support for Java soon.

 

That’s it! As I mentioned earlier, this is a beta feature and we are interested in your feedback. If you are interested in joining this beta program, please reach out.

 

Learn more about BlazeMeter from here.

Interested in writing for our Blog? Send us a pitch!