Tag: testing

  • Android: Testing an Intent Service

    Android: Testing an Intent Service

    There’s a handy thing in Espresso called the ServiceTestRule, which is for testing Services. Yay. I thought it was just what I needed until I read this bit of the documentation

    Note: The ServiceTestRule class does not support testing of IntentService objects. If you need to test a IntentService object, you should encapsulate the logic in a separate class and create a corresponding unit test instead.

    OK, first up – refactoring my IntentService. This class already did very little, poking something else depending on the kind of Intent it received, and then triggering another intent when the work was done. So it was pretty straightforward to refactor everything out into a Helper class. This class takes the intent, does the work, and returns an intent, which is then thrown.


    package co.ortatech.showandhide.service;
    import android.app.IntentService;
    import android.content.Intent;
    import android.support.v4.content.LocalBroadcastManager;
    import javax.inject.Inject;
    import co.ortatech.showandhide.application.ShowAndHideApplication;
    public class ImageProcessingService extends IntentService {
    @Inject ImageProcessingServiceHelper helper;
    public ImageProcessingService() {
    super(ImageProcessingService.class.getName());
    }
    public ImageProcessingService(String name) {
    super(name);
    }
    @Override
    public void onCreate() {
    super.onCreate();
    ((ShowAndHideApplication) getApplication()).component().inject(this);
    }
    @Override
    protected void onHandleIntent(Intent workIntent) {
    Intent localIntent = helper.handleAndReturnAppropriateIntent(workIntent,
    this.getContentResolver());
    // If the action fails, the intent can be null.
    if (localIntent != null) {
    LocalBroadcastManager.getInstance(this).sendBroadcast(localIntent);
    }
    }
    }

    This makes testing more straightforward because I just need to mock the helper, and return the requisite intent for whatever was supposed to happen.

    Testing the helper was also easier, which made it easier to be more thorough. The only thing I ran into was needing an ActivityRule (in order to get the ContentResolver).

    But – what about testing the thing that calls the IntentService? That was harder. Firstly, I set up IdlingResources, as explained by Chiu-Ki. But there’s an intent launched and received as the activity starts, which  complicates some things. I ended up verifying and resetting all my mocks for each test twice – once after setup completes, and the second time after the end of the test.

    First I got all tests running individually, but when I started combining them I was running into issues that I was pretty confident were concurrency issues. I asked Chiu-Ki to check, and on a faster emulator (and probably a better computer) things were fine… but they weren’t for me. I wanted to add UiController.loopMainThreadForAtLeast as explained in this blogpost, but that looked a bit complicated.

    To check if I was right, I added Thread.sleep(100); after each mock of the Helper. And yay! Everything worked! My enquiries into UiController.loopMainThreadForAtLeast hadn’t left me optimistic about it, so I left it at that.

    Ideally you don’t want to be using sleep() in tests. But I subscribe to the idea that any test is better than no test at all. Maybe I’ll be able to rewrite things so that this isn’t necessary, but I’m not that hopeful – part of the reason why this is necessary is because it’s a complicated process, and I’m trying to improve the perceived performance by breaking things up, which is why the Activity is so entwined with the IntentService.

    Apparently one option is Fork, which looks super useful. However if you use it, you need to run your tests from the command line. I really want to be able to run my tests from Android Studio (especially as I haven’t set up CI on this project), so I guess I will live with sleep()

  • Testing Intents on Android: Like Stabbing Yourself in the Eye With A Blunt Implement

    Testing Intents on Android: Like Stabbing Yourself in the Eye With A Blunt Implement

    App running on emulator in Android Studio
    App running on emulator in Android Studio

    The picture above shows what am I testing: the home screen of my app. There is a camera button, a gallery button and an inspire button. All of these launch intents, but the camera and gallery buttons launch intents that are expected to return something – an image – either from the camera or the gallery.

    First up: how the hell do you test an intent? I started with my straightforward Dagger/Mockito setup, but my tests were failing because they were launching things – the camera, or the gallery – which I then couldn’t get out of to continue my tests.

    The answer is the IntentsTestRule, which extends ActivityTestRule (if you’ve written a test for an activity, you’ve probably seen this). This took me a little while to make sense of, mainly because I kept getting this error saying that something had been initialised twice. I was launching the intent in my test, and also calling Intents.init(). Turns out, you don’t need to do that. Having an intent rule is a little bit of magic. It just launches itself.

    I have a very straightforward test that just checks that things have loaded, and it’s one I kept coming back to in order to see how things work. Here it is.


    @Test
    public void testLaunchActivity() {
    onView(withId(R.id.home_camera_button)).check(matches(withText("Camera")));
    onView(withId(R.id.home_gallery_button)).check(matches(withText("Gallery")));
    onView(withId(R.id.home_inspire_button)).check(matches(withText("Inspire")));
    }

    I think sometimes it seems like it’s not worth having tests like this because all of this will be covered elsewhere. Not true. I always include really straightforward tests. If they’re useless, then who cares, you’ll never think about them again. But in practise I return to them again and again when I’m debugging.

    First up I got my gallery test working. I had no idea what I was doing, so basically everything was broken, so I started by being really general and then getting more specific once I had it all working together. For example, figuring out which package it was in was a PITA, so to get thing working I used this handy catchall:

    intending(not(isInternal())).respondWith(result);

    This will return my “result” intent to every intent outside the app. Definitely not as exact as we want in our tests, but really useful for getting things working. In theory you can check ActionType, which in the case of the gallery is Intent.ACTION_PICK. In practise, this Intent was not being captured.

    I eventually managed to find out what the package was by using:

    Intents.assertNoUnverifiedIntents();

    This is a check for unverified intents, and handily in the error message it gave me it returned the package that the intent was coming from. So, for the gallery I had:

    intending(toPackage("com.android.gallery")).respondWith(result);

    Useful! The full test is:


    @Test
    public void testTapGalleryButtonAndReturnOK() {
    // Stub the Uri returned by the gallery intent.
    Uri uri = Uri.parse("uri_string");
    // Build a result to return when the activity is launched.
    Intent resultData = new Intent();
    resultData.setData(uri);
    Instrumentation.ActivityResult result =
    new Instrumentation.ActivityResult(Activity.RESULT_OK, resultData);
    // Set up result stubbing when an intent sent to "choose photo" is seen.
    intending(toPackage("com.android.gallery")).respondWith(result);
    onView(withId(R.id.home_gallery_button)).check(matches(withText("Gallery")));
    onView(withId(R.id.home_gallery_button)).perform(click());
    // Check image processor is reset.
    verify(imageProcessor).resetOriginalImage();
    // New activity should be launched
    intended(hasComponent(ImageEditingActivity.class.getName()));
    }

    Here I had some SUPER FUN debugging. By super fun I mean: like stabbing myself in the eye with a blunt implement. Luckily there were no blunt implements to hand and I have a strong sense of self preservation. Once I got my intent working, it seemed like the test was done. I added some validation of a new activity getting launched. Also, I decided to add validation on something else should happen – that the image processor gets reset. This should be straightforward, right? Everything is set up with Dagger and injected, I have my Mock Providers, and Mock Component for my tests. I just need to verify.

    Nope. It didn’t work, and it took me a long time to figure out why.

    The issue was that I didn’t have my mock objects in my activity. I thought this was because the activity was being relaunched, but that was a red-herring. The mock objects were being interacted with in the new activity, so I knew that my setup was somewhat right, but clearly not completely! Obsessively trying to hunt down the cause for this did help me fix a bug though – I had a fall through in a switch statement in my activity.

    Breakpoints had failed me (I was putting breakpoints in my tests, but the debugger wasn’t stopping on them – grr) I returned to my very straightforward test, and started just adding some log statements. This helped me figure out the problem: onCreate() in the HomeActivity was being called before setup (labelled with the annotation @Before) in my HomeActivityTest. So I had real objects in my activity under test, because it was using the usual dagger components, not the test ones.

    Once I knew what the problem was, I knew what to search for to find a fix for it. The conclusion: I needed to subclass the IntentsTestRule, and override beforeActivityLaunched() to put my pre-activity setup code in it.


    private static class HomeActivityTestRule extends IntentsTestRule {
    private HomeActivityTestRule() {
    super(HomeActivity.class);
    }
    @Override public void beforeActivityLaunched() {
    Instrumentation instrumentation = InstrumentationRegistry.getInstrumentation();
    ShowAndHideApplication app =
    (ShowAndHideApplication) instrumentation.getTargetContext().getApplicationContext();
    ShowAndHideTestComponent component = DaggerShowAndHideTestComponent.builder()
    .mockFileUtilitiesModule(new MockFileUtilitiesModule())
    .mockImageProcessorModule(new MockImageProcessorModule())
    .build();
    app.setComponent(component);
    }
    }

    There was no new code here – this was in my standard test setup. It’s just in a new place. I made it within the test, so it’s a private static class. As I add more tests I may move it out to be reusable. For now, it’s only binding the components I need in this test. If I move it out, I’ll need to bind everything.

    Then I added the test for the result failed test. This is more straightforward so it’s tempting to add this first, but in practise I never add my tests for nothing happening first because I never have any confidence that they actually work until I can break them.


    @Test
    public void testTapGalleryButtonAndReturnCancel() {
    // Build a result to return when the activity is launched.
    Intent resultData = new Intent();
    Instrumentation.ActivityResult result =
    new Instrumentation.ActivityResult(Activity.RESULT_CANCELED, resultData);
    // Set up result stubbing when an intent sent to "choose photo" is seen.
    intending(toPackage("com.android.gallery")).respondWith(result);
    onView(withId(R.id.home_gallery_button)).check(matches(withText("Gallery")));
    onView(withId(R.id.home_gallery_button)).perform(click());
    // Check image processor is not reset.
    verify(imageProcessor, never()).resetOriginalImage();
    }

    Notice that in the first test, I assert my image processor gets reset. In this one I assert that it doesn’t get reset.

    Next, testing the camera intent. It’s basically the same, but I also have to mock the Uri, because when you take a photo on Android you first have to allocate space for it. I have complained about this before so I will refrain here.


    @Test
    public void testTapCameraButtonAndReturnOK() {
    // Uri needed to launch the Camera intent.
    Uri uri = Uri.parse("uri_string");
    stub(fileUtilities.getOutputMediaFileUri()).toReturn(uri);
    // Build a result to return when the activity is launched.
    Intent resultData = new Intent();
    Instrumentation.ActivityResult result =
    new Instrumentation.ActivityResult(Activity.RESULT_OK, resultData);
    // Stub result for camera intent.
    intending(toPackage("com.android.camera")).respondWith(result);
    onView(withId(R.id.home_camera_button)).check(matches(withText("Camera")));
    onView(withId(R.id.home_camera_button)).perform(click());
    // Check image processor is reset.
    verify(imageProcessor).resetOriginalImage();
    // New activity should be launched
    intended(hasComponent(ImageEditingActivity.class.getName()));
    }

    Notice at the end of it we assert that we’ve transitioned to the next activity – the ImageEditingActivity. This is what this line does.

    intended(hasComponent(ImageEditingActivity.class.getName()));

    And again, we check that it works when the result is cancelled.


    @Test
    public void testTapCameraButtonAndCancel() {
    // Uri needed to launch the Camera intent.
    Uri uri = Uri.parse("uri_string");
    stub(fileUtilities.getOutputMediaFileUri()).toReturn(uri);
    // Build a result to return when the activity is launched.
    Intent resultData = new Intent();
    Instrumentation.ActivityResult result =
    new Instrumentation.ActivityResult(Activity.RESULT_CANCELED, resultData);
    // Stub result for camera intent.
    intending(toPackage("com.android.camera")).respondWith(result);
    onView(withId(R.id.home_camera_button)).check(matches(withText("Camera")));
    onView(withId(R.id.home_camera_button)).perform(click());
    // Check image processor is not reset.
    verify(imageProcessor, never()).resetOriginalImage();
    }

    It was tempting to delete the mock fileUri here, because it seems like we don’t need it, but actually if I did that it would not be testing what I want it to, even though the behaviour (nothing happens) would be the same.

    In both the cancel intents, I wanted to assert that no new intent is launched, but it’s wasn’t clear how to do that. Intents.assertNoUnverifiedIntents() was failing on things I’d stubbed. I realised that I had to stub and expect things, so I needed an intended() for each intending(), and voila! It works. Once I discovered this, I went back through all of my tests and added the requisite lines.

    Finally! My most straightforward intent test, and you may wonder why I didn’t start with it. As do I. There’s a third button, inspire, which launches the web browser with a specific URL. Note that this test uses intended() rather than intending(), and that it comes after the intent is launched – not before. At the end, we assert no unverified intents.


    @Test
    public void testTapInspireButton() {
    onView(withId(R.id.home_inspire_button)).check(matches(withText("Inspire")));
    onView(withId(R.id.home_inspire_button)).perform(click());
    // Capture web browser intent.
    intended(toPackage("com.android.browser"));
    Intents.assertNoUnverifiedIntents();
    }

    Note: one tutorial I found suggested stubbing all external intents with the blanket catchall, and setting it up in a method annotated with @Before. I didn’t do this and I don’t think it’s a good idea. Launching an external intent is something that you should be capturing deliberately, and that’s the whole point of testing it. I would be more likely to go the other way and assert no unverified intents in a method annotated @After!

    Complete code for this HomeActivityTest.

    Useful Resources

    • Setting up
    • Sample code
    • Chiu-Ki took me through getting Dagger and Espresso set up in the first place, ages ago. And if you’re not lucky enough to be friends with her (and passing through Denver), she has a bunch of resources for testing on Android on her website.
  • OCMock and Values

    OCMock and Values

    magnifying glass shows a bug underneath
    Credit: Pixabay / testbytes

    OCMock is mostly great! And I use it a lot! But there’s one problem with it – handling values. I was debugging some asynchronous tests and having this problem, which I assumed at first came from the asynchronicity but it turns out, no, it’s just CGFloat being CGFloat.

    Clue for this kind of bug is getting two causes of test failure: method unexpectedly called, and method expected but not called and they both look the same because to 6dp they are the same at 6dp but not quite beyond that. And there’s no way to set the margin of error.

    One way to fix it is to get rid of CGFloats and use NSNumber instead, but I really didn’t want to do this. Sometimes you want to pass a value! And that should be fine!

    It took me a while to figure this out, and I actually had to go back and use OCMock2 syntax to fix it [reference]. But: you can ignore non-object arguments. For some cases this would mean I wasn’t testing what I wanted to, but here I know that it being the correct CGFloat is covered elsewhere, so in this test I could safely ignore it.

    [[[mock expect] ignoringNonObjectArgs]       someMethodWithIntArgument:0]

    This made using andDo a bit weird (which because Asynchronous I was using to fulfil my expectation).  In the snippet below block is what gets called when the stub is invoked.

    [[[[mock stub] andDo:block] ignoringNonObjectArgs] someMethodWithIntArgument:0];

  • Replacing KIF Tests with XCUI Tests

    Replacing KIF Tests with XCUI Tests

    app screenshots

    I thought about doing this in Swift but decided to take this one thing at a time (I haven’t written any Swift yet). My strategy: 1) get tests working and then 2) convert them to Swift. This post focuses on (1). As for what tests to write, I had a full suite of KIF tests before I did a visual refresh, so my starting point was replacing those.

    Note: I initially got errors from swift complaining about deployment target needing to be 9.0. This didn’t seem to be the case once I moved to Objective-C.

    Issue 1: “No target application path specified”

    Resolution: I had been trying to put my UI tests in the same target as my unit tests. Apparently you can fix this with some build settings, but for now I opted to put them in a separate target – I want my unit tests to be fast enough to develop against, and I don’t know how long the UI tests will take to run. I figure I can move my unit tests into the UI target later if they are fast enough.

    Issue 2: Build errors from KIF

    Resolution: Remove KIF. I’m not going to spend too much time on this since KIF is going anyway.

    Test 1: Rotate home screen

    Simple test that rotates the phone four times (to result in a full rotation), and after each rotation checks that the home screen buttons are still there.

    I use record (launches the app and inserts the programatic equivalent of your actions) to get the code, and then refactor it to my taste. To verify each rotation, in KIF I had a method that waited for the buttons, here I use XCTest to assert the button exists, and that they are “hittable”.

    Note: I declare all strings in the file “SHAStrings.h”, which means I can reference them from UI tests and don’t have to change my tests with copy changes.


    – (void)testRotateHomeScreen {
    XCUIDevice *device = [XCUIDevice sharedDevice];
    [self verifyHomePageButtons];
    [device setOrientation:UIDeviceOrientationLandscapeRight];
    [self verifyHomePageButtons];
    [device setOrientation:UIDeviceOrientationPortraitUpsideDown];
    [self verifyHomePageButtons];
    [device setOrientation:UIDeviceOrientationLandscapeLeft];
    [self verifyHomePageButtons];
    [device setOrientation:UIDeviceOrientationPortrait];
    [self verifyHomePageButtons];
    }


    – (void)verifyHomePageButtons {
    XCUIElement *cameraButton = [app_ buttons][[SHAStrings cameraButtonTitleString]];
    XCTAssertTrue([cameraButton exists]);
    XCTAssertTrue([cameraButton isHittable]);
    XCUIElement *galleryButton = [app_ buttons][[SHAStrings galleryButtonTitleString]];
    XCTAssertTrue([galleryButton exists]);
    XCTAssertTrue([galleryButton isHittable]);
    XCUIElement *inspireButton = [app_ buttons][[SHAStrings inspireButtonTitleString]];
    XCTAssertTrue([inspireButton exists]);
    XCTAssertTrue([inspireButton isHittable]);
    }

    Test 2: Press inspire button

    Test that taps one of the buttons on the home screen, and launches a web view with the tumblr page. I found a bug writing this test, so added in code that would wait for the webpage title to load (found here).


    – (void)testOpenInspireView {
    [[app_ buttons][[SHAStrings inspireButtonTitleString]] tap];
    // Verify page load by checking the title.
    XCUIElement *title = [app_ otherElements][@"Show and Hide"];
    NSPredicate *predicate = [NSPredicate predicateWithFormat:@"exists == 1"];
    [self expectationForPredicate:predicate evaluatedWithObject:title handler:nil];
    [self waitForExpectationsWithTimeout:2.0 handler:nil];
    // Go back.
    [[[[[app_ navigationBars][@"SHAInspireView"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    // Should be at Home Page.
    [self verifyHomePageButtons];
    }

    Issue 3: Blocked HTTP Request

    I actually found a bug writing this test (I love it when that happens!): “App Transport Security has blocked a cleartext HTTP (http://) resource load since it is insecure. Temporary exceptions can be configured via your app’s Info.plist file.” I fixed that following these instructions (also: need to be consistent with use of www in the plist and in the URL).

    Test 3: Open the gallery and cancel

    Tests that the gallery opens and then closes again.

    Previously any cancel button was tapped, but the nice thing about XCUI is that I can specify the cancel button in the navigation bar.


    – (void)testOpenGalleryAndCancel {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[[[app_ navigationBars][@"Photos"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Cancel"] elementBoundByIndex:0] tap];
    // Should be at Home Page.
    [self verifyHomePageButtons];
    }

    Test 4: Open the gallery, select an image, and go back

    Tests that a photo can be selected, the edit view opens, and goes back.

    Using the Photos stuff was one of the most annoying things about working with KIF, to pick a photo I had to tap on the screen at a place where there was usually a photo, which could be a little flaky (the “usually” is the clue there). I couldn’t find anything googling, and then I remembered the “record” button and voila: painless. Yay!


    – (void)testChooseImageAndGoBack {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    [self verifyEditPageButtonsVisible];
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }


    – (void)verifyEditPageButtonsVisible {
    // Colors / Balance should be there.
    XCUIElement *colors = [app_ staticTexts][[SHAStrings accuracySliderDescription]];
    XCTAssertTrue([colors exists]);
    XCTAssertTrue([colors isHittable]);
    XCUIElement *balance = [app_ staticTexts][[SHAStrings toleranceSliderDescription]];
    XCTAssertTrue([balance exists]);
    XCTAssertTrue([balance isHittable]);
    }

    Test 5: Test rotation on the edit view

    Test open the gallery, select an image, rotate and verify the controls disappear in landscape and re-appear in portrait.

    Here I assert the buttons exist, but aren’t hittable.


    – (void)testChooseImageAndRotate {
    XCUIDevice *device = [XCUIDevice sharedDevice];
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    [self verifyEditPageButtonsVisible];
    [device setOrientation:UIDeviceOrientationLandscapeRight];
    [self verifyEditPageButtonsNotVisible];
    [device setOrientation:UIDeviceOrientationPortrait];
    [self verifyEditPageButtonsVisible];
    [device setOrientation:UIDeviceOrientationLandscapeLeft];
    [self verifyEditPageButtonsNotVisible];
    // Return to portrait.
    [device setOrientation:UIDeviceOrientationPortrait];
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }


    – (void)verifyEditPageButtonsNotVisible {
    // Colors / Balance should be there.
    XCUIElement *colors = [app_ staticTexts][[SHAStrings accuracySliderDescription]];
    XCTAssertTrue([colors exists]);
    XCTAssertFalse([colors isHittable]);
    XCUIElement *balance = [app_ staticTexts][[SHAStrings toleranceSliderDescription]];
    XCTAssertTrue([balance exists]);
    XCTAssertFalse([balance isHittable]);
    }

    Test 5 and 6: Tap the respective sliders

    Opens gallery, selects image, taps slider, goes back.

    I never actually got anything happening on the sliders using KIF, and they are still not changing value, but the tap should trigger the image being regenerated, which is something.


    – (void)testChangeAccuracySlider {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    XCUIElement *slider = [app_ sliders][[SHAStrings accuracySliderA11yLabel]];
    [slider tap];
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }
    – (void)testChangeToleranceSlider {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    [self verifyEditPageButtonsVisible];
    XCUIElement *slider = [app_ sliders][[SHAStrings toleranceSliderA11yLabel]];
    [slider tap];
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }

    Test 7: Change the picture from the edit page

    Opens gallery, selects image, on edit page selects another image and goes back.


    – (void)testChangeImageFromEditPage {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    [self verifyEditPageButtonsVisible];
    XCUIElement *navBar = [app_ navigationBars][[SHAStrings imageEditingViewControllerTitle]];
    [[navBar buttons][[SHAStrings galleryButtonA11yLabel]] tap];
    [[[app_ tables] buttons][@"Camera Roll"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, March 12, 2011, 4:17 PM"] tap];
    [self verifyEditPageButtonsVisible];
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }

    Test 8: Test Swipe

    Opens gallery, selects image, swipes back and forth, goes back.

    Recording didn’t give me the swipe gesture, so instead tapped on the indicator instead of swiping. Then looked it up and used [app_ swipe].


    – (void)testSwipe {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    [self verifyEditPageButtonsVisible];
    XCUIElement *element = [[[app_ scrollViews] childrenMatchingType:XCUIElementTypeOther] element];
    XCUIElement *imageButton = [[[element childrenMatchingType:XCUIElementTypeOther]
    elementBoundByIndex:0] buttons][[SHAStrings processedImageA11yLabel]];
    [imageButton pressForDuration:1];
    XCUIElement *page1Of2PageIndicator = [app_ pageIndicators][@"page 1 of 2"];
    [page1Of2PageIndicator tap];
    XCUIElement *page2Of2PageIndicator = [app_ pageIndicators][@"page 2 of 2"];
    [page2Of2PageIndicator tap];
    [app_ swipeLeft];
    XCTAssertTrue([page2Of2PageIndicator isHittable]);
    [app_ swipeRight];
    XCTAssertTrue([page1Of2PageIndicator isHittable]);
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }

    view raw

    TestSwipe.m

    hosted with ❤ by GitHub

    Test 9: Test share.

    Opens gallery, selects image, selects share, goes back.


    – (void)testShare {
    [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
    [[[app_ tables] buttons][@"Moments"] tap];
    [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
    [self verifyEditPageButtonsVisible];
    XCUIElement *editNavigationBar = app_.navigationBars[@"Edit"];
    [editNavigationBar.buttons[@"share"] tap];
    [[[app_ sheets] buttons][@"Cancel"] tap];
    // Go back to Home Page.
    [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
    matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
    [self verifyHomePageButtons];
    }

    view raw

    TestShare.m

    hosted with ❤ by GitHub

    Space and Time

    According to RescueTime: 2h 23m of software development + 30 minutes in notes (writing what turned into this post).

    Code: KIF file deleted: 232 loc. UI tests added: 234 loc. Bonus: the UI tests have better coverage. Although the biggest benefit of switching over is removing a dependency, especially one that has been a bit annoying to maintain.

    Observations

    • Record makes a great starting point for tests, once I got into the habit of using it things went faster. I ended up rewriting a lot of the code it produced, though.
    • I think these tests run faster than KIF, but not by that much. Worth keeping a separate target. Bonus: means the inner workings of your app are not exposed (for me, just that strings file).
    • Sometimes a bit flakey – I think this might be because there’s some asynchronicity and my computer was running slowly.
    • So far the images have renamed themselves (with a different timestamp – weird) once. I changed it to just pick the first image in the collection.

    Resources

    More on  unit testing, with a focus on UI code, can be found in my unit testing workshop.

  • A Brief Rant about TDD

    A Brief Rant about TDD

    success ln / failure dr
    Credit: Flickr / StockMonkeys.com

    I’ve been giving this talk about unit testing UI code lately and of course when you talk about testing, TDD (test driven development) keeps coming up.

    The question ranges from “have you embraced TDD as the One True Way Of Testing, and if not why not because you’re doing it wrong” and “I have heard that TDD is the One True Way Of Testing, but I don’t unit test at all and I’m overwhelmed”.

    Note that neither of these are actually questions, but I digress.

    When it comes to my own workflow, I don’t do strict TDD (gasp!)

    • I don’t debug; I write tests.
    • When I have something that is inputs in -> outputs out I quite often write my tests first. In practise very little application code is like this.
    • Testing is largely an architectural problem, and I architect my code such that it is testable, usually writing my tests concurrently.
      • I don’t consider my code to be “done” until it is tested.
      • A lot of my testing involves testing UI code, e.g. I test that the right thing happens when a button is tapped. I guess technically I could stub out the view element and write tests for it, in practise I tend to complete my train of thought and write the view and then test it.

    Here’s my answer to the second none-question: I don’t believe in telling people to TDD. I believe in testing your code. That’s it. I’m happy to share my workflow, and techniques I use (buy my workshop!) but people should do what works for them.

    I do think though that TDD is something that you work up to (some point of), not what you start with. Because:

    • When you start writing tests you’ll probably find more bugs in your test code than your actual code. This kind of undermines the idea of the tests as the spec.
    • It’s super intimidating. You’re almost certainly going to discover that however you have been writing your code it’s not in a way that is testable. You’re going to need to refine your tests and your code together.
    • The idea that something has to be done “The Right Way” rather than “At All” is a huge impediment to starting. If you want to write more tests don’t wait until you have a new class that you can do TDD on, set aside an hour, choose something simple and get going.

    TL;DR: I don’t really TDD. I don’t care if you TDD. I do think everyone should test their code, though.

  • Unit Testing on iOS

    Unit Testing on iOS

    Broken glass
    Credit: Wikipedia

    Historically, there hasn’t been a lot of testing done on iOS, and we see the results of that every day—regressions, crashes, consistently reproducible failures. As we build more complex applications, manual testing takes more and more time and automated testing becomes increasingly necessary.

    The biggest challenge of testing on iOS starts at the UIViewController with the tangling of view and control code. Sometimes it continues all the way down, and model and database code also becomes entwined. Testing on iOS is a vicious cycle – the architecture patterns are hard to test, so we don’t test, so we don’t consider testability in our architecture, so things become even harder to test.

    The first step is to break this cycle, which starts with breaking up the UIViewController. There’s a Reactive Cocoa pattern called MVVM but regardless of the approach we take, we need to get that control code out of the UIViewController, then we can test them separately. I’ve outlined a strategy for that in more detail here.

    This might seem a bit overwhelming, so a good place to start when writing unit tests is the class with the least complexity and the fewest dependencies. Typically this is our model classes. When it comes to model classes it can be tempting to think that this is so simple that there’s no need to test it. But I think it’s still worthwhile because:

    1. How would we decide when something is “complicated” enough to test?
    2. What if it later became more complicated?
    3. If it were that easy, the test would be very quick to write.
    4. We want to be confident this piece works well, so that we can rely on it in larger tests later.

    As we get to more complicated objects we need to master the art of Dependency Injection. This is when we pass an argument in the initialiser, rather than creating it in the class. Dependency injection helps make our code more testable by clarifying dependent objects that we can replace with mocks or otherwise control or observe.

    Unit-Testing UI Code

    As mentioned above, it’s challenging to test UI code on iOS because of that entangling of control and view code. Let’s talk about one example: testing buttons.

    Typically, when testing buttons, we don’t actually test the buttons—we test that the methods that are added to the buttons work as they should. There are two downsides to this.

    Firstly, we have to expose a method for testing. Not that big a deal—this is why Java has the @VisibleForTesting annotation. But we might prefer not to do that. In Objective-C we can also declare a category in our test class to allow our tests to “see” the private method, but I like this strategy even less.

    Secondly—and most importantly—this doesn’t actually test what happens when the button is tapped. What if the wrong @selector is set? Or if we autocomplete to UIControlEventTouchUpOutside instead of UIControlEventTouchUpInside?

    Alternative: we write tests to return real UIButtons, then we can tap it and verify what happens. I’ve covered how in more detail here.

    UIAutomation

    UIAutomation tests are also known as monkey tests—something that will go through and tap buttons on your app and make things happen. These are proper “black box” tests—all they know about is the UI, not the inner workings of your code.

    UIAutomation tests are integration tests, rather than unit tests, so it’s a complementary testing strategy to writing extensive unit tests. While unit tests check the internal workings of your app, UIAutomation tests are a good way to check the flow of the app—that each view controller loads, for example.

    UIAutomation tests are:

    • Slower to run.
    • Hard to test corner cases on.

    UIAutomation tests are great for:

    • Showing that the things you built are glued together correctly.
    • Making sure your app is accessible throughout.
    • Checking that each view loads.
    • Testing things like carousels.

    I like KIF (which stands for “Keep It Functional”), which is a wrapper around Apple’s UIAutomation framework that allows you to write the KIF tests in Objective-C rather than JavaScript.

    If you’re interested in learning about testing iOS code in more detail, I have a workshop with sample app that covers these strategies in more detail.

  • Some Thoughts on Mocking

    Some Thoughts on Mocking

     

    Magritte Pipe
    Credit: Wikipedia

     

    When do we use mocks and when shouldn’t we? Some thoughts:

    Don’t Mock Data Structures

    We wouldn’t mock an NSArray or an NSDictionary, so why would we mock our own data structures? Data structures should be simple and well tested, so we can trust them to behave as they should. It will be more work to mock them than just to use them.

    Mock Behaviour, Not State

    If we did decide to mock a data structure, we’d have to specify all the interactions between the mock and the code. That would make our tests more verbose.

    More importantly, the relevant thing isn’t how the data structure is used (the behaviour), but what happens given the state.

    Combining This

    We use mocks when we have a complex object that we want to control. For example, we’d use a mock for things where we can’t set the state. So for instance, we might want a mock to return a fake location or return that the device has a camera (even though the simulator doesn’t have a camera). If we used a mock to return a fake location, we would return a real location object (which we can create with whatever data we want), but through a mock.

     

    If you found this interesting, I’ve covered testing iOS apps in more detail in this workshop – there’s a sample app and a guide that takes you step by step through adding tests.

  • Launching! iOS Unit Testing: Beyond the Model

    Launching! iOS Unit Testing: Beyond the Model

    I’m super excited to release something that I’ve been working on for a while.

    Unit testing on iOS is… not common. And part of the problem is that people don’t know where to start. It can be overwhelming.

    Building on my years of experience leading iOS apps with over 80% test coverage, including at Google, and my extensive experience in curriculum development and education, I’ve put together a workshop that takes you through the process step by step. It covers:

    • Writing unit tests on model classes.
    • Using mocks.
    • Unit-testing UI code (and how to test UIViewControllers!).
    • Writing your first UIAutomation tests.

    The (MIT-licensed) sample code is ready to go (just check it out from GitHub) so you can focus on writing better tests, not wrangling dependencies.

    It’s fully digital. Download the PDF, and work through. Email support included.

    Who is this for? If you have iOS experience and you:

    • Haven’t written any tests and don’t know where to start, start at Section 1.
    • Are comfortable writing unit tests but want to better test your UI code, start at Section 2.

    This isn’t designed for people without iOS experience looking to learn iOS.

    Some nice things people have said already:

    “Thanks so much… it’s helped me a lot. I have a fairly large, complex code base that I want to add tests to and the workshop has helped me to think about how to even begin to do that.”

    —Cathy @catshive

    “I really like the workshop. I like the format and pacing, how you’ve structured the Xcode project and using unfulfilled/failing tests for the participant to add themselves. I’ll definitely check out KIF for my own projects, even if it’s just as a way to ensure my apps are properly Accessible.”

    Harry @inquisitivesoft

    Buy it for 20 USD

    Please note: this focuses on Obj-C not Swift, although the patterns should be applicable to Swift as well. I’m looking into a Swift version so let me know if that’s something you are interested in.

    If you’re a student or underemployed, contact me and I’ll send you a discount code.

    Thanks so much to the many people who tested, reviewed, and gave feedback.

  • Creating and Comparing Images on Android

    Creating and Comparing Images on Android

    IMG_8440

    A while ago, I wrote this blog post on creating and comparing UIImages. That code allowed me to develop the image processing part of the app against my unit tests, which was really, really helpful given that I rewrote it about four times to make it performant enough.

    So, when I started writing Android code it was one of the first things I ported. Firstly let me say – way easier on Android than iOS. A tiny difference in the API was a gotcha, iOS takes arguments x, y, width, height, and a comparable function on Android takes x1, y1, x2, y2. But other than that it was much more straightforward, basically because of how much easier it is to delve into the pixels.

    To create a one color image, you can just set the color and draw to the canvas:


    /**
    * A one color image.
    * @param width
    * @param height
    * @param color
    * @return A one color image with the given width and height.
    */
    public static Bitmap createImage(int width, int height, int color) {
    Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
    Canvas canvas = new Canvas(bitmap);
    Paint paint = new Paint();
    paint.setColor(color);
    canvas.drawRect(0F, 0F, (float) width, (float) height, paint);
    return bitmap;
    }

    
    

    But if you want to set individual pixels you can just call setPixel(). So to create a 3×3 2-color-alternating image (I find this a really useful test image):


    /**
    * A 3×3 2-color image.
    * @param color1
    * @param color2
    * @return A 3×3 image alternating the two colors.
    */
    public static Bitmap createTwoColorImage(int color1, int color2) {
    Bitmap bitmap = Bitmap.createBitmap(3, 3, Bitmap.Config.ARGB_8888);
    bitmap.setPixel(0, 0, color1);
    bitmap.setPixel(2, 0, color1);
    bitmap.setPixel(1, 1, color1);
    bitmap.setPixel(0, 2, color1);
    bitmap.setPixel(2, 2, color1);
    bitmap.setPixel(1, 0, color2);
    bitmap.setPixel(0, 1, color2);
    bitmap.setPixel(2, 1, color2);
    bitmap.setPixel(1, 2, color2);
    return bitmap;
    }

    This is similar to the way we can create an image from an array of colors:


    /**
    * A image from an array of colors.
    * @param width
    * @param height
    * @param colors
    * @return image of given width and height filled with the given color array.
    */
    public static Bitmap createImage(int width, int height, int[] colors) {
    Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
    int x = 0;
    int y = 0;
    for (int color : colors) {
    bitmap.setPixel(x, y, color);
    x++;
    if (x == width) {
    x = 0;
    y++;
    }
    }
    return bitmap;
    }

    On iOS creating from an array was sufficiently complicated that I felt like the simpler creation methods were also worthwhile. On Android I’m less certain! I may refactor them to just call the array function.

    But now we have made our test images, we need to be able to compare them. As before, I’m defining two images as the same iff (if and only if) they have the same width, height, and the pixels are the same color. For now I’m able to do an exact comparison, but I may add some kind of tolerance here as the code evolves.


    /**
    * Compare two images.
    * @param bitmap1
    * @param bitmap2
    * @return true iff both images have the same dimensions and pixel values.
    */
    public static boolean compareImages(Bitmap bitmap1, Bitmap bitmap2) {
    if (bitmap1.getWidth() != bitmap2.getWidth() ||
    bitmap1.getHeight() != bitmap2.getHeight()) {
    return false;
    }
    for (int y = 0; y < bitmap1.getHeight(); y++) {
    for (int x = 0; x < bitmap1.getWidth(); x++) {
    if (bitmap1.getPixel(x, y) != bitmap2.getPixel(x, y)) {
    return false;
    }
    }
    }
    return true;
    }

    These helper methods have been a really important part of my testing strategy on both platforms – the image processing is the core of the app, and I want to be sure it works really well.