Strategies for Testing Ruby Services in a Rails app

At PipelineDeals, we follow the microservice architecture pattern. Many of our features are separate applications that expose a REST API. But this poses a challenge for testing our service applications.

This post describes a strategy for using an adapter to isolate the service in question, and then outlines a different strategies for testing the service integration.

Before going further, it’s important to have a good definition of a unit test vs integration test. I realize this may be slightly pedantic, but it will help frame the discussion going forward.

Unit tests

  • Tests behavior of low-level functions
  • OK to use test doubles
  • Can serve as documentation for a function, for other developers
  • Very specific and fine-grained.

Integration tests

  • Tests and ensures the behavior of the system as a whole
  • Generally not the best time to be using mocks.
  • Tests from a high level or a user perspective

It will be important to keep these distinctions in mind when we go forward and talk about strategies for testing services.

Architecture of the core app

The best strategy a client application can take is to abstract off all calls to the remote service to its own adapter class. Let’s say you have a client app that needs to communicate with Dropbox (The external service app). In that case, the best thing to do is set up a DropboxAdapter class, and move all communication to and from Dropbox to that class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
module DropboxService

  class DropboxEntry
    attr_accessor :path, :content, :mtime
  end

  class DropboxAdapter
    attr_reader :user, :client, :access_token

    def initialize(user)
      @user = user
      @client = DropboxClient.new(access_token)
    end

    def get_file(path)
      hash = @client.get_file(path)
      entry = DropboxEntry.new
      entry.path = hash["path"]
      entry.content = hash["content"]
      entry.mtime = hash["mtime"]
      entry
    end

    def put_file(path, file)
      @client.put_file(path, file)
      return true
    end

    def get_changes(cursor = nil)
      deletions, additions, changes = [],[],[]
      @client.delta.each do |change|
         # do the logic here to parse the output of Dropbox's delta function, and format it into a logical way for our app.
      end
      {deletions: deletions, additions: additions, changes: changes}
    end
  end
end

When creating the adapter class strive for the following:

  • One call to the external service per method. We can compose these methods together to form a behavior in other classes.
  • If the response from the external service is crazy, it is ok to sanitize that response into a simple hash or class.
  • Return true for 200 responses.
  • Throw an exception for other types of responses.

If you follow the rules above, you’ll find that testing will be a breeze.

What does wrapping up the DropboxClient give us? The ability to stub out a specific response later, in tests!

Unit Testing the adapter class, and avoiding VCR abuse

At PipelineDeals, we are big fans of the VCR gem. However, we find that its use in test can easily be abused. If you find that making an innocuous change in your client app leads to 50 spec failures because the VCR cassettes need to be re-recorded, that’s a sign of VCR abuse.

Testing the service adapter class with VCR is an appropriate use of VCR. We are using it to assert the sanity of our adapter class, which we will use in other classes to compose behaviors we want. So long as we strive to have only one remote service call per service adapter function, things will remain sane..

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class DropboxSerivceUnitTest < ActiveSupport::TestCase
  let(:user) { users(:bob) }
  let(:service) { DropboxService::DropboxAdapter.new(user) }

  it "can get a file from dropbox" do
    VCR.use_cassette(:get_file) do
      note = service.get_file("/notes/meeting-notes.md")
      note["path"].must_equal "/notes/meeting-notes.md"
      note["content"].must_equal "Here are the meeting notes"
    end
  end

  let(:file) { File.open("fixtures/test-file.md") }
  it "saves a file to dropbox" do
    VCR.use_cassette(:save_file) do
      note = service.put_file("/notes/test-file.md", file)
      service.get_file("/notes/test-file.md")["content"].must_equal file.read
    end
  end
end

If possible, always strive to have a minimal number of external requests in a single VCR cassette. 1 request per cassette is ideal. However, in some cases you need 2 requests – one to make the state change in the remote service, and another request to verify the state change. The put_file request needs 2 requests to ensure the state change was correct on the remote service.

Using VCR like this is acceptable because after our adapter is fleshed out, it’s unlikely to change often. Perhaps if the external service’s API changes, or if we make major changes to our method signatures, but the service adapter should be low-level enough that these changes should happen but once in a blue moon.

Unit testing the classes that use the service adapter

After we are satisfied that our service adapter is doing its job, we can then build the business logic on top. Imagine, for instance, that we have a photo sharing app that utilizes Dropbox and keeps photos in sync.

Maybe we have a DropboxSync class that handles the nitty gritty of keeping things in sync, and that class relies on our DropboxAdapter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
class DropboxSync
  attr_reader :user
  attr_accessor :service

  def initialize(user)
    @user, @service = user, DropboxService::DropboxAdapter.new(user)
  end

  def perform_sync
    @service.get_changes.each do |change|
      delete_local_file(change) and next if deleted?(change)

      if new_file?(change)
        create_new_local_file(change)
      else
        update_existing_local_file(change)
      end
    end
  end

  def delete_local_file(change)
    ...
  end

  def create_new_local_file(change)
    ...
  end

  def update_existing_local_file(change)
    ...
  end
end

The purpose of this class is to pull changes from Dropbox and update our local state. We will want to unit test perform_sync, as well as all the sub-methods. It is at this point that our DropboxAdapter class comes in handy.

The strategy for unit testing this class is we’ll inject a fake DropboxAdapter instance into our class, with predetermined outputs for a given input. Remember, the goal here is to unit test the methods in this application, so it is not appropriate to actually call the external service here. We’ve already guaranteed the functionality of our DropboxAdapter in the previous section.

At the top of our test, we’ll define our fake dropbox service. It will have exactly the same method signatures as our regular Dropbox service. If you wanted, you could use inheritance to guarantee the method signatures are the same. For this example I did not do that, but be aware that if you change DropboxService, you’ll also need to change FakeDropboxAdatper as well!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class FakeDropboxAdapter
  attr_reader :user
  attr_accessor :get_file_results, :put_file_results, :get_changes_results, :access_token

  def initialize(user)
    @user = user
    @client = DropboxClient.new(access_token)
  end

  def get_changes
    @get_changes_results
  end

  def get_file(path)
    @get_file_results
  end

  def put_file(path, file)
    @put_file_results
  end
end

As you can see, we will explicitly tell FakeDropboxAdapter the exact response we want back for each method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class DropboxSyncUnitTest < ActiveSupport::TestCase
  let(:user) { users(:bob) }
  let(:fake_service) { FakeDropboxAdapter.new(user) }
  let(:dropbox_sync) { DropboxSync.new  }

  context "when there is a new file on dropbox" do
    before do

      # drop in our fake service
      sync.service = fake_service

      # Simulate the response where there is a new file on dropbox
      service.get_changes_results = {additions: [DropboxEntry.new(name: "/notes/test.md", content: "here is some new content")], deletions: [], changes: [] }
    end

    it "adds the file to the local store" do
      previous_file_count = local_files.count
      dropbox_sync.perform_sync
      local_files.count.must_equal previous_file_count + 1
    end

    it "adds the correct info to the new local file" do
      dropbox_sync.perform_sync
      local_files.last.path.must_equal "/notes/test.md"
      local_files.last.content.must_equal "here is some new content"
    end
  end
end

The unit test above uses a simulated service. This is mainly for speed of development and to ensure that the logic in the DropboxSync class is correct.

Putting it all together: Integration tests

Integration tests should not use any stubbing. Ideally we would be testing calls to and from the real service, from the perspective of the user. Therefore we need to think about ways to reliably create and destroy data we need on the external service, without affecting production data. A sandbox of the service in question comes in really handy for these types of tests.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
class DropboxSyncIntegrationTest < ActiveSupport::TestCase
  let(:user) { users(:bob) }
  context "when there is a new file on dropbox" do

    before do
      # before running the tests in question, physically add the file we need to dropbox
      DropboxService::DropboxAdapter.put_file("/notes/test.md", File.read("support/test_files/test.md"))
    end

    after do
      # perform "cleanup" on the external service
      DropboxService::DropboxAdapter.clear_all_files!
    end

    it "adds the file to the local store" do
      previous_file_count = local_files_count
      dropbox_sync.perform_sync
      local_files.count.must_equal previous_file_count + 1
    end

    it "adds the correct info to the new local file" do
      dropbox_sync.perform_sync
      local_files.last.path.must_equal "/notes/test.md"
      local_files.last.content.must_equal "here is some new content"
    end
  end
end

The integration test above guarantees the functionality of the system by using the physical service. It’s important to remember that just like a database, there needs to be a reliable way of adding the test data to the service before the test runs. Conversely, we also need to clean up the data from the service when we are finished testing.

It’s probably not necessary to have these full integration tests run as part of a developer’s unit test suite. These types of tests are better suited towards when it is appropriate for testing at the suite level, such as when you’re just about to push the changes to the repo.

Conclusion

Testing external services need not be a pain, as long as you keep the right perspective. If you think of a external service as just another place to store data and state, much like a database, then that can frame your thinking about how to go about testing that service. In the example above, you could easily substitute a database for the service. Taking that perspective, there is nothing in this post that is revolutionary.

To recap:

  1. It is OK to stub functionality at the unit test level.
  2. VCR is a good choice to unit test and stub a service adapter, but it should not be used for integration tests.
  3. VCR is not a good choice when testing actual logic that the service adapter facilitates. In the case above that was demonstrated when unit testing the DropboxSync class. We want to tightly control the response of the service adapter in that case.