Choosing and Testing Different App Types

Native App, Hybrid App, Progressive Web App, or Responsive Web Solution

How do you choose one app type over another? What characterizes each type, and what impact does the choice have on quality assurance of the solution?

App Types

Click to read more about each app type:

Native apps

Native apps are developed specifically for a given platform, typically Android or iOS. This type of app has very few technical limitations when it comes to leveraging the phone’s capabilities and features, but it requires in-depth knowledge of the platform’s programming language/tools (iOS = Xcode or Swift, Android = Android Studio).

Responsive Design solution

A pure web solution that adapts to different screen sizes via viewport information. It only leverages the mobile device’s capabilities and features to a very limited extent. Optimizing pages for specific devices and browsers requires knowledge and experience. The solution is typically developed using HTML5, CSS, and JavaScript.

Hybrid app

Essentially a web solution encapsulated inside a native app “shell.” It can leverage a large portion of the phone’s features through the app shell, but it is generally experienced as slightly slower. It also requires knowledge of common web technologies such as HTML5, CSS, and JavaScript. Native code is automatically generated for all supported platforms via the development tool. A hybrid app is often used as a “wrapper” for a web solution, making it available as an app in Google Play and the App Store.

Progressive Web App

A web solution similar to Responsive Design, with the same challenges related to optimizing pages. A PWA is an evolution of a Responsive Design solution, and the primary difference is that it can leverage mobile features to a greater extent via HTML5 API calls. Which API calls are available depends on the individual browser.

Product Analysis

Before development of a mobile solution begins, a product analysis is conducted — among other reasons — to determine which technology (app type) best meets the requirements and needs identified through the analysis.

Below are examples of questions that may be included in the product analysis:

What needs should the solution address?
Target audience(s)?
Use of the device’s technologies/features, such as camera, GPS, NFC, BLE, accelerometer, gyroscope, compass, contacts, etc.?
Connection and interaction with other devices via e.g. Bluetooth, NFC, BLE, Wi-Fi?
User interface?
Security and compliance with standards?
Payment options?
Integrations?
Skills/competencies and budget resources?
Offline functionality?
Solution performance?
Installation and maintenance? Platform dependencies?
Restrictions on content, approval processes, and costs?

Once the answers to the questions above are in place, you have the basis for choosing between the four primary app types — each with its own advantages and disadvantages.

Testing Approach

The product analysis forms the basis for many important activities, including:

Establishing the test basis
Selecting the app type
Selecting representative test configurations
Needs for test data and test environments
Needs for tools and technologies
Needs for budget and resources
Needs related to the solution’s ecosystem

This provides the essential foundation for creating the solution’s test strategy and test plan.

Here, it is important to consider that the choice of app type impacts a range of test-related activities — as solutions can range from pure web solutions to platform-specific codebases with nearly full access to all mobile device features.

Below are some of the test activities and areas that this choice can particularly affect.

Test Configurations

The choice of app type determines which parameters you should prioritize when collecting relevant usage statistics, and it therefore has a decisive impact on which recommended test configurations are identified by the subsequent analysis.

For example, collecting statistics on browser versions is not relevant for a native app solution, but it is highly relevant for Responsive Design, PWA, and in many cases Hybrid app solutions. Similarly, OS versions and support for technologies such as NFC and BLE are not relevant for Responsive Design solutions, but they are relevant for native apps — and potentially also to some extent for PWA and hybrid app solutions that use these technologies.

App Distribution and Updates

If you choose a native or hybrid app solution, you must understand the requirements it needs to meet to pass Google’s and Apple’s approval processes — and then verify compliance with those requirements.

Unlike Responsive Design and PWA web solutions — where you simply access a URL in the test environment — native and hybrid solutions require you to set up an app distribution system that also handles updates in both development and test environments.

Test Scope and Methodologies

If you choose a predominantly web-based mobile solution, most of the solution’s implementation and coexistence with the platform’s other functions are handled by the device’s browser.

You can primarily focus on the solution’s integration with the rest of its own ecosystem.

If you choose a native or hybrid app solution, you must to a much greater extent ensure that the app and its features integrate smoothly with the given platform and its ecosystem.

You also need to test a range of other aspects that are specific to these app types, for example: installation, updating, uninstallation, network and connectivity scenarios, memory leaks, battery consumption, interruptions, notifications, gestures, and potentially offline functionality.

For these purposes, there are some strong mobile testing methodologies that we may cover in a future blog post.