README.md

Mobile Testing

• Mobile Testing
  • Building via EAS
  • Building Locally (Android)
  • Local Test Execution
    • Prerequisites
    • Local Test Execution Against Expo Dev Mode
      • iOS
      • Android
      • Single test execution
    • Local Test Execution Against Binary
      • iOS Simulator
      • Android Emulator
  • Remote Test Execution on BrowserStack
  • Writing tests
    • Assumptions
    • API docs
  • Troubleshooting
    • "browser driver is not running"
    • Can't find simctl SDKs
    • Espresso server can't be built / started
    • Can't find ANDROID_HOME nor ANDROID_SDK_ROOT

Building via EAS

Note for iOS: There are two options on how to build an application for iOS: simulator build and archive build. The former can be used to run your application within a simulator, the latter is used for installation on real devices / distribution via app stores. Browserstack works on real devices, so archive builds are needed there.

This requires the EAS command line tool to be installed.

Platform	Yarn Command	Alternatively run within apps/mobile
iOS (Simulator)	yarn mobile:build_ios_sim	eas build --platform ios --profile simulator
iOS	yarn mobile:build_ios	eas build --platform ios
Android	yarn mobile:build_android	eas build --platform android
Android (APK)	yarn mobile:build_android_apk	eas build --platform android --profile preview
Both	yarn mobile:build_all	eas build --platform all

Alternatively, you can use application binaries that have already been built by the build pipeline. It produces iOS and Android artifacts in the stages mobile_staging_build and mobile_production_build. These can either be downloaded from GitLab as a build artifact or from the Expo Dashboards Build View.

Building Locally (Android)

Follow this tutorial.

Local Test Execution

Prerequisites

You need to install Appium in order to run the tests locally.

1. npm i -g appium@next

2. npm install @appium/doctor --location=global

3. appium driver install xcuitest

4. appium driver install uiautomator2

5. run appium-doctor (Then do all the things appium-doctor tells you to do / shows as error.) (You can also pass arguments to only check for a specific platform: --ios resp. --android)

   appium needs the following Android SDK Tools, that are not automatically installed and need to be installed via Android Studios SDK Manager:

   • Android SDK Platform-Tools
   • Android SDK Tools (Obsolete) <- This is only shown when you uncheck "Hide Obsolete Packages"

When appium-doctor is successful, you can run appium:

6. run appium

7. On some systems it might be necessary to add export APPIUM_PATH='/wd/hub' to the .envrc.local file (if you see errors like No route found for /session while executing the tests) and also pass this to appium as base-path:

   appium --base-path $APPIUM_PATH

Local Test Execution Against Expo Dev Mode

Executing the e2e tests locally requires the mobile application to be running on a device and appium to be started. We also expect the user to be logged out initially.

For Android there is a script that takes care of starting the application and an emulator if necessary.

iOS

1. Make sure the application is running on the simulator.
2. run yarn test:ios (The script detects and uses the locally running simulator)

Android

1. Configure environment variables E2E_ANDROID_VIRTUAL_DEVICE_NAME and ANDROID_HOME (see .envrc.local.template, maybe ANDROID_HOME is already set up globally)

   You can use an emulator of your own by using its name as value of E2E_ANDROID_VIRTUAL_DEVICE_NAME or use an actual device (run adb devices to make sure it is properly connected).

2. run yarn workspace @holi/e2e-mobile prepare:android

   This script starts and maybe creates an emulator if necessary and runs appium. You still need to start the mobile application locally (using mprocs from holi-meta would be recommended).

   If the emulator is run for the first time, Expo Go might open some onboarding layer that should be clicked away.

   If the app was running before, maybe close and reopen it (making sure the newest version is used).

3. run yarn test:android

   This starts the actual test execution.

Single test execution

Single tests can be executed by passing an argument with the name of the spec to be executed, e.g.

yarn test:android -t HomeScreen
# resp.
yarn test:ios -t HomeScreen
# or even
yarn test:android -t 'Login should work with valid credentials'

See the documentation of the Jest CLI for more options.

Local Test Execution Against Binary

iOS Simulator

APP_PATH=/path/to/application.app yarn test:ios

You need to use the simulator build here (i.e. yarn mobile:build_ios_sim).

Android Emulator

APP_PATH=/path/to/application.apk yarn test:android

You need to use the APK build here (i.e. yarn mobile:build_android_apk).

⚠️ Make sure to uninstall a previously installed version of the app, as it might not be overwritten by Appium. (You could set enforceAppInstall: true in the appiumConfig.ts, but this also slows down execution times).

Remote Test Execution on BrowserStack

Upload the previously created application bundle via the tooling or the instructions on the BrowserStack Dashboard. Copy and paste the "app_url" value displayed on the dashboard (or e.g. as returned by curl) into the following command:

export BROWSERSTACK_APP_URL=<yourappurl>
export BROWSERSTACK_USERNAME=<yourusername>
export BROWSERSTACK_ACCESS_KEY=<youraccesskey>

export APPIUM_TARGET='browserstack' # optional, as this is the default setting
export APPIUM_PLATFORM='ios' # or 'android'

yarn workspace @holi/e2e-mobile run jest --runInBand

or alternatively in a single line (in this e2e/mobile directory):

BROWSERSTACK_USERNAME=<yourusername> BROWSERSTACK_ACCESS_KEY=<youraccesskey> BROWSERSTACK_APP_URL=<yourappurl> APPIUM_PLATFORM=<yourplatform> yarn run jest --runInBand

Alternatively, add your Browserstack credentials to .envrc.local and have them automatically exported via direnv.

If you feel like automating this process by writing a script, take a look at .gitlab-ci-mobile.yml.

Writing tests

Assumptions

As it is not as easy to instantly navigate to a specific screen on mobile devices as it is on web using URLs, we expect every test to be started on the home screen. When testing using a specific binary (e.g. during CI), this is ensured by starting the app anew before each test. When running the tests locally against the Expo Dev Mode, we keep the same app instance running all the time and make sure to navigate back to the home screen before each test using the helper function navigateBackToHomeScreen (defined in e2e/mobile/helpers/navigation.ts) during setup (e2e/mobile/jest.setup.ts).

We also assume the user to be logged out initially.

API docs

As we use WebdriverIO as Node.JS client for Appium, see the webdriver.io API docs for available commands as well as Appium specific commands. You can also take a look into the Appium API docs - keep in mind that we are using webdriver.io instead of wd and we are using the UiAutomator2 driver for Android and XCUITest for iOS.

There already are some helper functions, e.g. to locate and interact with elements or to perform a login with the e2e user, which you can find in the e2e/mobile/helpers directory.

Troubleshooting

"browser driver is not running"

When running the tests, if you see an error message along the lines of

Unable to connect to "127.0.0.1:4444", make sure browser driver is running on that address.

it might be necessary to manually run the appium process in a separate terminal session, via:

npm install -g appium
appium

Can't find simctl SDKs

If you see an error like the following for an iOS simulator:

Original error: '15.4' does not exist in the list of simctl SDKs. Only the following Simulator SDK versions are available on your system: x.y

you can either install the required version or set the environment variable IOS_PLATFORM_VERSION to the version you have installed (see Appium troubleshooting).

Espresso server can't be built / started

I had trouble that the Espresso server could not be built on AdoptOpenJDK16 on macOS. Changing my java version to SDK 11 as was hinted here helped.