Up until now, Codemagic was able to run Flutter Driver tests on Android emulator or iOS simulator. Thanks to an open-source command-line utility called Sylph, we now have out-of-the-box AWS Device Farm integration available on Codemagic so you can easily test your Flutter apps on a vast selection of physical Android and iOS devices as part of the build.

What is Sylph and why do we need it?

AWS Device Farm offers a huge pool of Android and iOS devices for running integration tests on many real devices at once. This is a great option for ensuring your release builds run as expected before pushing the app out to app stores. Prior to Sylph, there was no easy way to use AWS Device Farm for Flutter integration tests since AWS Device Farm supports native and Appium tests only and Flutter Driver tests are not in the list of supported frameworks.

Sylph, developed by Maurice McCabe, opens the door to AWS Device Farm for Flutter apps. Under the hood, Sylph makes it look as if you’re running Appium tests. Sylph can be set up locally on Mac, Linux or Windows using command line, but it also works in CI environments — such as Codemagic. The entire AWS Device Farm test run configuration, including test suites and the devices to be used for testing, is defined in a sylph.yaml file. You can see a configuration example in the Sylph repository.

Sylph and AWS Device Farm on Codemagic

Codemagic has both Sylph and AWS Device Farm integrated into its build pipeline, making it really easy for Flutter developers to use AWS Device Farm for testing their apps. An AWS account and a simple sylph.yaml configuration file in the repo is all that is needed for making these two seamlessly work. And if you don’t have a Sylph configuration file, we will create one during the build to run the tests with default configuration.

Let’s walk through the process of configuring Codemagic to run Flutter Driver tests on real devices with the AWS Device Farm integration.

Step 1. Set up AWS Device Farm

Running integration tests on AWS Device Farm can be enabled in App settings > Test. First, you need to check Enable Flutter Driver tests and then select where to run them. As soon as you select AWS Device Farm, you will see the fields to enter your AWS access key ID and AWS secret access key.

#coding #testing #codemagic