Replacing KIF Tests with XCUI Tests

app screenshots

I thought about doing this in Swift but decided to take this one thing at a time (I haven’t written any Swift yet). My strategy: 1) get tests working and then 2) convert them to Swift. This post focuses on (1). As for what tests to write, I had a full suite of KIF tests before I did a visual refresh, so my starting point was replacing those.

Note: I initially got errors from swift complaining about deployment target needing to be 9.0. This didn’t seem to be the case once I moved to Objective-C.

Issue 1: “No target application path specified”

Resolution: I had been trying to put my UI tests in the same target as my unit tests. Apparently you can fix this with some build settings, but for now I opted to put them in a separate target – I want my unit tests to be fast enough to develop against, and I don’t know how long the UI tests will take to run. I figure I can move my unit tests into the UI target later if they are fast enough.

Issue 2: Build errors from KIF

Resolution: Remove KIF. I’m not going to spend too much time on this since KIF is going anyway.

Test 1: Rotate home screen

Simple test that rotates the phone four times (to result in a full rotation), and after each rotation checks that the home screen buttons are still there.

I use record (launches the app and inserts the programatic equivalent of your actions) to get the code, and then refactor it to my taste. To verify each rotation, in KIF I had a method that waited for the buttons, here I use XCTest to assert the button exists, and that they are “hittable”.

Note: I declare all strings in the file “SHAStrings.h”, which means I can reference them from UI tests and don’t have to change my tests with copy changes.

– (void)testRotateHomeScreen {
XCUIDevice *device = [XCUIDevice sharedDevice];
[self verifyHomePageButtons];
[device setOrientation:UIDeviceOrientationLandscapeRight];
[self verifyHomePageButtons];
[device setOrientation:UIDeviceOrientationPortraitUpsideDown];
[self verifyHomePageButtons];
[device setOrientation:UIDeviceOrientationLandscapeLeft];
[self verifyHomePageButtons];
[device setOrientation:UIDeviceOrientationPortrait];
[self verifyHomePageButtons];
}

– (void)verifyHomePageButtons {
XCUIElement *cameraButton = [app_ buttons][[SHAStrings cameraButtonTitleString]];
XCTAssertTrue([cameraButton exists]);
XCTAssertTrue([cameraButton isHittable]);
XCUIElement *galleryButton = [app_ buttons][[SHAStrings galleryButtonTitleString]];
XCTAssertTrue([galleryButton exists]);
XCTAssertTrue([galleryButton isHittable]);
XCUIElement *inspireButton = [app_ buttons][[SHAStrings inspireButtonTitleString]];
XCTAssertTrue([inspireButton exists]);
XCTAssertTrue([inspireButton isHittable]);
}

Test 2: Press inspire button

Test that taps one of the buttons on the home screen, and launches a web view with the tumblr page. I found a bug writing this test, so added in code that would wait for the webpage title to load (found here).

– (void)testOpenInspireView {
[[app_ buttons][[SHAStrings inspireButtonTitleString]] tap];
// Verify page load by checking the title.
XCUIElement *title = [app_ otherElements][@"Show and Hide"];
NSPredicate *predicate = [NSPredicate predicateWithFormat:@"exists == 1"];
[self expectationForPredicate:predicate evaluatedWithObject:title handler:nil];
[self waitForExpectationsWithTimeout:2.0 handler:nil];
// Go back.
[[[[[app_ navigationBars][@"SHAInspireView"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
// Should be at Home Page.
[self verifyHomePageButtons];
}

Issue 3: Blocked HTTP Request

I actually found a bug writing this test (I love it when that happens!): “App Transport Security has blocked a cleartext HTTP (http://) resource load since it is insecure. Temporary exceptions can be configured via your app’s Info.plist file.” I fixed that following these instructions (also: need to be consistent with use of www in the plist and in the URL).

Test 3: Open the gallery and cancel

Tests that the gallery opens and then closes again.

Previously any cancel button was tapped, but the nice thing about XCUI is that I can specify the cancel button in the navigation bar.

– (void)testOpenGalleryAndCancel {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[[[app_ navigationBars][@"Photos"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Cancel"] elementBoundByIndex:0] tap];
// Should be at Home Page.
[self verifyHomePageButtons];
}

Test 4: Open the gallery, select an image, and go back

Tests that a photo can be selected, the edit view opens, and goes back.

Using the Photos stuff was one of the most annoying things about working with KIF, to pick a photo I had to tap on the screen at a place where there was usually a photo, which could be a little flaky (the “usually” is the clue there). I couldn’t find anything googling, and then I remembered the “record” button and voila: painless. Yay!

– (void)testChooseImageAndGoBack {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
[self verifyEditPageButtonsVisible];
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}

– (void)verifyEditPageButtonsVisible {
// Colors / Balance should be there.
XCUIElement *colors = [app_ staticTexts][[SHAStrings accuracySliderDescription]];
XCTAssertTrue([colors exists]);
XCTAssertTrue([colors isHittable]);
XCUIElement *balance = [app_ staticTexts][[SHAStrings toleranceSliderDescription]];
XCTAssertTrue([balance exists]);
XCTAssertTrue([balance isHittable]);
}

Test 5: Test rotation on the edit view

Test open the gallery, select an image, rotate and verify the controls disappear in landscape and re-appear in portrait.

Here I assert the buttons exist, but aren’t hittable.

– (void)testChooseImageAndRotate {
XCUIDevice *device = [XCUIDevice sharedDevice];
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
[self verifyEditPageButtonsVisible];
[device setOrientation:UIDeviceOrientationLandscapeRight];
[self verifyEditPageButtonsNotVisible];
[device setOrientation:UIDeviceOrientationPortrait];
[self verifyEditPageButtonsVisible];
[device setOrientation:UIDeviceOrientationLandscapeLeft];
[self verifyEditPageButtonsNotVisible];
// Return to portrait.
[device setOrientation:UIDeviceOrientationPortrait];
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}

– (void)verifyEditPageButtonsNotVisible {
// Colors / Balance should be there.
XCUIElement *colors = [app_ staticTexts][[SHAStrings accuracySliderDescription]];
XCTAssertTrue([colors exists]);
XCTAssertFalse([colors isHittable]);
XCUIElement *balance = [app_ staticTexts][[SHAStrings toleranceSliderDescription]];
XCTAssertTrue([balance exists]);
XCTAssertFalse([balance isHittable]);
}

Test 5 and 6: Tap the respective sliders

Opens gallery, selects image, taps slider, goes back.

I never actually got anything happening on the sliders using KIF, and they are still not changing value, but the tap should trigger the image being regenerated, which is something.

– (void)testChangeAccuracySlider {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
XCUIElement *slider = [app_ sliders][[SHAStrings accuracySliderA11yLabel]];
[slider tap];
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}
– (void)testChangeToleranceSlider {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
[self verifyEditPageButtonsVisible];
XCUIElement *slider = [app_ sliders][[SHAStrings toleranceSliderA11yLabel]];
[slider tap];
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}

view raw
TestChangeSliders.m
hosted with ❤ by GitHub

Test 7: Change the picture from the edit page

Opens gallery, selects image, on edit page selects another image and goes back.

– (void)testChangeImageFromEditPage {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
[self verifyEditPageButtonsVisible];
XCUIElement *navBar = [app_ navigationBars][[SHAStrings imageEditingViewControllerTitle]];
[[navBar buttons][[SHAStrings galleryButtonA11yLabel]] tap];
[[[app_ tables] buttons][@"Camera Roll"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, March 12, 2011, 4:17 PM"] tap];
[self verifyEditPageButtonsVisible];
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}

Test 8: Test Swipe

Opens gallery, selects image, swipes back and forth, goes back.

Recording didn’t give me the swipe gesture, so instead tapped on the indicator instead of swiping. Then looked it up and used [app_ swipe].

– (void)testSwipe {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
[self verifyEditPageButtonsVisible];
XCUIElement *element = [[[app_ scrollViews] childrenMatchingType:XCUIElementTypeOther] element];
XCUIElement *imageButton = [[[element childrenMatchingType:XCUIElementTypeOther]
elementBoundByIndex:0] buttons][[SHAStrings processedImageA11yLabel]];
[imageButton pressForDuration:1];
XCUIElement *page1Of2PageIndicator = [app_ pageIndicators][@"page 1 of 2"];
[page1Of2PageIndicator tap];
XCUIElement *page2Of2PageIndicator = [app_ pageIndicators][@"page 2 of 2"];
[page2Of2PageIndicator tap];
[app_ swipeLeft];
XCTAssertTrue([page2Of2PageIndicator isHittable]);
[app_ swipeRight];
XCTAssertTrue([page1Of2PageIndicator isHittable]);
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}

view raw
TestSwipe.m
hosted with ❤ by GitHub

Test 9: Test share.

Opens gallery, selects image, selects share, goes back.

– (void)testShare {
[[app_ buttons][[SHAStrings galleryButtonTitleString]] tap];
[[[app_ tables] buttons][@"Moments"] tap];
[[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap];
[self verifyEditPageButtonsVisible];
XCUIElement *editNavigationBar = app_.navigationBars[@"Edit"];
[editNavigationBar.buttons[@"share"] tap];
[[[app_ sheets] buttons][@"Cancel"] tap];
// Go back to Home Page.
[[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton]
matchingIdentifier:@"Back"] elementBoundByIndex:0] tap];
[self verifyHomePageButtons];
}

view raw
TestShare.m
hosted with ❤ by GitHub

Space and Time

According to RescueTime: 2h 23m of software development + 30 minutes in notes (writing what turned into this post).

Code: KIF file deleted: 232 loc. UI tests added: 234 loc. Bonus: the UI tests have better coverage. Although the biggest benefit of switching over is removing a dependency, especially one that has been a bit annoying to maintain.

Observations

  • Record makes a great starting point for tests, once I got into the habit of using it things went faster. I ended up rewriting a lot of the code it produced, though.
  • I think these tests run faster than KIF, but not by that much. Worth keeping a separate target. Bonus: means the inner workings of your app are not exposed (for me, just that strings file).
  • Sometimes a bit flakey – I think this might be because there’s some asynchronicity and my computer was running slowly.
  • So far the images have renamed themselves (with a different timestamp – weird) once. I changed it to just pick the first image in the collection.

Resources

More on  unit testing, with a focus on UI code, can be found in my unit testing workshop.

5 thoughts on “Replacing KIF Tests with XCUI Tests

  1. This is so great–thanks!! Two questions:

    1. You mention that “the UI tests have better coverage”. Can you explain what you mean?
    2. In what way were the tests flakey? How did they fail? Did XCUI fail to discover elements, despite them being on the screen?

    [WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.

    1. 1. I was able to test some things that I couldn’t in KIF. Specifically – swiping the images.
      2. No, I think things just timed out sometimes.

      [WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.

  2. Can you share your KIF Tests? It would be nice to see what changed

    [WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.