One of the many battles to develop applications for a mobile audience is testing your app – with so many different devices, you are usually limited to how many you have on hand. AWS DeviceFarm facilitates this process.
What is DeviceFarm?
The concept of DeviceFarm is quite simple. AWS spends money to buy a lot of devices (over 2500 are in its library at the moment) and rent them out when someone needs to use them. You can connect to the device directly and interact with it, just as if you were using a device emulator, but with real hardware at the other end.
Although you can connect to individual devices and test on devices you do not have, the real advantage of DeviceFarm is automation. It allows you to run automatic UI tests using the Appium test framework to test your apps on multiple devices simultaneously. DeviceFarm records the screen and inputs for each automatic test, so you can go in and review what went wrong if something goes wrong.
DeviceFarm charges based on the number of minutes used, at a rate of $ 0.17 per unit minute. This number is multiplied by the number of units you test on if you run automatic tests.
If you make one pulp tests and want to take advantage of DeviceFarm’s automated testing tools, you can purchase unlimited use for $ 250 per month per device. This is only valuable if you do more than 24 hours of testing per month on a single device, which is a lot of hours to run tests, but it may be worthwhile for some corporate customers.
Do I need to test the device?
DeviceFarm supports native apps, hybrid apps, and full web apps. The benefits vary depending on the type of application you are developing.
Native mobile apps definitely need constant testing on the device. While XCode provides iOS emulators, they will not always be perfect, and you will catch many bugs testing on real hardware.
With hybrid apps, you do not need to test the device as much, as almost all built-in code is handled in the library. However, you may still encounter bugs between emulated and real devices, and you should still do on-device testing.
With web apps, you might consider testing your device for the most part; Safari on the desktop uses the same WebKit rendering engine as the iOS browser, so you will not find many differences between the two. Both Chrome and Safari both have tools for testing responsive design. If you open the dev tools, you can choose from a few different device resolutions to preview how your website works.
Although this does not emulate the device, it is only the resolution, but most of the rendering errors are still related to tight mobile restrictions.
Even if web apps are not so necessary, we still recommend that you double-check your site on real devices to make sure the experience is smooth and error-free. You may not need DeviceFarm to do this, but it’s good if you do not have a variety of devices at hand.
How to get started with DeviceFarm
Go to the DeviceFarm console and enter a new project name. Manual access is easier to configure, so get started. Click the “Remote Access” tab and create a new session.
From here you can select a device to connect to. The availability of these devices may depend on who is using them at any given moment, but there seems to be a wide range of device types to choose from.
Create a new session, and after a minute you will be greeted with the home screen. The connection is not perfect (roughly equivalent to a standard view of the remote desktop), but it is enough to do basic tests. Your keyboard should connect to the device if you are focused on it, which makes typing easier.
From here, if you are troubleshooting a web app, you can open it directly in the browser. If you are troubleshooting an app that needs to be installed, you can load the pages
.ipa through the “Install Applications” drop-down menu in the sidebar. This installs your app on the device for testing. If you test multiple devices multiple times, you can choose from a scroll down of your recent uploads rather than recharging.
For automated testing, the process is a little different. You need to go back to the console and instead create a new run under the “Automated Testing” tab:
From here, you choose the type of app you want to develop and upload yours
.ipa if you are developing a native or hybrid app.
Then upload your tests. DeviceFarm supports the Appium automation framework for running UI tests on both web and embedded apps. For built-in apps, a built-in “Fuzz” tester can randomly click around and test for bugs.
Then select all the devices you want to test. By default, DeviceFarm selects the five most popular devices, but you can create your own device pool.
You are now prompted to enter the maximum number of minutes you want the devices to run for, and AWS can show you the total based on the number of devices you are testing in parallel. This is the highest cost you pay, but if your test ends early, you will not be charged earlier.