Less drudgery, more fun: using Kaleidoscope with XCTest failures

  • Florian 
  • 5 min read

Most developers for Apple platforms deal with tests in some way or another. XCTest is probably the most popular framework because it’s built directly into Xcode and can be integrated with build processes and automation.

However, one problem with those tests is that more complex failures are not easy to interpret. And if things aren’t easy (and fun) to use, developers will have a resistance to using them, and that is bad for the end product.

When a test fails, you get an entry in the Xcode Issues Navigator. The screenshot on the right shows an example of what a failure looks like in the Issues Navigator on a vanilla macOS Monterey system in Xcode 13.2.1. The only useful information is that a test for equality (XCTAssertEqual) failed in some function with a long name located in a file with a long name.

For this article, we set up a small sample project using the Swift Markdown package provided by Apple on their Github page and made one test fail intentionally by adding arbitrary text to the expected dump that is being compared to a debug description in that test.

The Issues Navigator in Xcode 13

When you click the entry, Xcode shows you the line in source code causing the failure, and also highlights the issue, see the tiny red area on the right.

With a little practice, you’ll be able to correctly single-click the red X icon. After a short pause, you’ll get a tooltip showing the full error, which is a mouthful in this case:

But even with that full message, what exactly is wrong here? As this is an XCTAssertEqual call, we are seeing two things that should be equal according to the test, but they are not. With a sharp eye, we can spot the “is not equal to” somewhere in the middle. Hmm, that sounds like a good use case for a diff tool. Wait a minute…

Wouldn’t it be nice to skip the drudgery of selecting and searching, and instead see something like the screenshot below, showing you the exact differences?

Kaleidoscope 3 shows an XCTAssertEqual in a usable format.

Luckily, Kaleidoscope 3.1.3 and later can show XCTest failures in that format for you! No further trick or conversion is needed. Kaleidoscope parses the incoming content and automatically splits it into A and B parts, so the exact differences can be seen instantly.

Getting results into Kaleidoscope

The challenge remains how to get from that error to a Kaleidoscope window. Unfortunately, we weren’t able to find a direct integration with Xcode for this. Here are four different ways to get from Xcode to Kaleidoscope.

All following examples assume Xcode is the front app and the Issues Navigator is showing the test failure. To bring up the Issues Navigator, select View > Navigators > Issues from the menu, or press ⌘5 if you prefer using the keyboard.

Mouse and Menu

The first method involves traditional mouse pointing and menus:

  • In the sidebar, select the entry showing the error
  • From the menu bar, select Edit > Copy > Copy
  • In the Dock, click the Kaleidoscope icon
  • From the menu bar, select File > New from Clipboard

Mouse and Keyboard

Let’s speed up things by using the keyboard and the mouse:

  • In the sidebar, bring up the context menu on the error
  • Select Copy
  • Press ⌘⇥ (command + tab) to change to Kaleidoscope
  • In the toolbar, click Add from Clipboard

Drag onto the Dock icon

Things get faster with Drag & Drop:

  • In the sidebar, select and drag the entry showing the error
  • Drop it onto the Kaleidoscope app icon in the macOS Dock

Drag onto the Kaleidoscope window

Use the keyboard while dragging:

  • In the sidebar, select and drag the entry showing the error
  • While dragging, press ⌘⇥ (command + tab) to change to Kaleidoscope
  • Drop your selection above the Kaleidoscope window

What’s Next

We know that we haven’t caught all the cases, so there will be improvements to parsing results in the future. Have you found cases that are not yet supported in Kaleidoscope? Please let us know!

Have an idea for tighter integration? Get in touch, we’d love to hear about it!

Until then, we hope this little addition to Kaleidoscope will help you write more and better tests, and will make the process of testing a bit more fun. In the long run, this will result in better apps for all of us!