QA Best Practices: How to Do Smart TV Application Testing Right

Interest in Smart TV apps is growing, since they are such an interactive way of consuming content. To provide high-quality and highly usable Smart TV apps, it’s essential to apply certain rules and best QA services (i.e. TV application testing), because the process of ensuring quality for TV apps is different from the process for other formats.

How is TV App Testing Different from Other Types of Testing?

Smart TV application testing has several unique features:

• Preparation is key:
  • First and foremost, you need to determine the list of devices that will be used for testing
  • There are a lot of configuration settings to deal with before testing starts, so don’t be shy about googling for instructions (or referring to this article) to get your devices set up
  • Be prepared to gather logs and take screenshots, which can be tricky when testing such devices, so make sure you’ve researched it in advance
• Each new build needs to be installed, and previous builds need to be uninstalled
• Network connection types should be taken into account. Don’t forget the built-in features. Your app should be findable on your console via search and display in the Recent Apps section correctly

Useful Tips for Smart TV Application Testing:

Before you get down to Smart TV application testing, you should be aware of several tips and tricks:

Check how the app works under different network scenarios The app should be stable and not dependent on the network. Testers should be sure that the app works in the following conditions:

• when connected via LAN/WiFi/mobile network
• when the connection is slow
• when there’s no network – in this case, the user receives a notification via a message window, and the app resumes once connection is reestablished

Make sure that the app is using built-in features correctly Smart TV consoles have lots of built-in features such as streaming video services, universal search, music streaming, and casting. Make sure that your app works correctly with those features.

Make sure to follow platform-specific guidelines when uploading your TV application to the target app store For example, buttons should have colors and locations that correspond to store guidelines, and users should be able to use the remote to access the app’s functionality.

When doing TV app testing, do not rely solely on emulators Emulators can make testing easier, but they have several drawbacks that we need to keep in mind before starting:

• Dependencies on the network: your emulator may work great under ideal conditions on your laptop, but once the app gets installed on the target device, it might start returning failures because it wasn’t tested on real-life conditions
• User interactions: the app may be optimal when you use the mouse to click on different tabs or buttons but completely ignore the remote control
• Issues with displaying the app: events that occur on the device while the app is launched (e.g., opening a TV settings modal view) may cause issues with the display of your app, but this can’t be checked on the emulator

How to Do Smart TV Application Testing While Your Application is Not Yet Live

Your app can’t be downloaded from the store for testing while you’re still developing it, so you need to be prepared to follow slightly more complex downloading instructions in order to perform Smart TV application testing.

For example, in order to test an AppleTV app, you need to take the following steps in order to configure it to work with Mac:

Apple Configurator:

• Create a profile with the following values:
  • Add general Information (name)
  • HTTP Proxy (IP and Port)
  • Certificates: Charles Certificate in cer format
• Select device > Click Prepare > Select Supervise Devices > Next > Do not enroll in MDM > Next > Next > Prepare > Erase
• Add profile from step 1 via Apple Configurator

AppleTV Open General > About > Certificate Trust Settings > Certificate should be trusted

Charles

• Open Charles > Help > SSL Proxying > Install Charles Root Certificate
• Open My Certificates in Categories (for Login). Open All Certificates and add them to trusted
• Open Certificates in Categories (for Login). Open All Certificates and add them to trusted
• Open All Items in Categories (for System Root). Open All Apple, Developer, and Geo Certificates to trusted

Then you need to add your Apple ID to use TestFlight and install TestFlight on your device.

You can use Charles for traffic monitoring, Xcode for screenshots and logs, QuickTime to take videos, and Apple Configurator to take logs and configure Apple TV with Mac. And then you’re done!

Author: Karalina Panada