Using Test Fixture in Google Test #7

In this Blog you will learn how to optimize setting up common environments for your tests using test fixtures in google test. 

What is a test Fixture

A Test Fixture allows you to creates setup and teardown method to run before each test is run in your test case. Test Fixtures are easy to create in google test. You just need to create a class that is the same name as your test case and inherit from the testing::Test class.

class BitcoinWallet_ULT : public testing::Test { public:  virtual void SetUp() {  }  virtual void TearDown() {  }};

How to use the Test Fixture

Now that you have a test fixture you need to let you tests know how to use the fixture. This is easy as well. Use the TEST_F macro instead of the TEST macro.

• Without Test Fixture
  • TEST(BitcoinWallet_ULT,updateLocalBalanceTest)
• With Test Fixture
  • TEST_F(BitcoinWallet_ULT,updateLocalBalanceTest)

Only use the TEST_F macro for tests in you test case that you want to use the test fixture. You don't have to use TEST_F if you test does not require setup.

What happens when TEST_F is called

The TEST_F macro will connect your test into the test fixture. This means that the following things will happen each time the TEST_F macro is run.

1. Instantiate a fixture class - Calls new on the BitcoinWallet_ULT class
2. Call SetUp on the fixture 
3. Call the test
4. Call TearDown on the fixture
5. Destroy the fixture - Calls delete on the BitcoinWallet_ULT object created in step 1

What goes in Setup

Anything that is required to run the test (environment) that is common between all of a majority of the tests. Many tests in the same suite will have common variables that they use to run the tests. This typically include Mocks, and setup environment variables. I typically put these variables in the fixture as attributes. Then in the Setup Method I instantiate the objects and assign them to the attributes so they can be used in the tests. Also don't forget to destroy any objects you create in SetUp in the TearDown method. 

class BitcoinWallet_ULT : public testing::Test { public:  string myAccount;  BitcoinWallet* wallet;  BitcoinBank_Mock* bankMock;  BitcoinCommerce_Mock* commerceMock;  BitcoinCurrency_Mock* currencyMock;  virtual void SetUp() {   ExternalService::theBank = bankMock = new BitcoinBank_Mock();   ExternalService::theCommerce = commerceMock = new BitcoinCommerce_Mock();   ExternalService::theCurrency = currencyMock = new BitcoinCurrency_Mock();   wallet = new BitcoinWallet();  }  virtual void TearDown() {   delete(wallet);   delete(bankMock);   delete(commerceMock);   delete(currencyMock);  }};

Notice in the SetUp that I am setting variables for the ExternalService as well as the attribute to be used in the TEST_F macro. This allows me to change the behavior of the mock in the test itself. Also for each object I create in the SetUp I have a corresponding delete in the TearDown.

Using the TEST_F Macro

Now that the fixture is complete setup I can use it in my test case. All I need to do is change the TEST macro to TEST_F and remove any of the environment setup that the fixture now handles for me.

Before TEST_F

TEST(BitcoinWallet_ULT,updateLocalBalanceTestNoAccount) {  BitcoinBank_Mock* bankMock; ExternalService::theBank = bankMock = new BitcoinBank_Mock();   ON_CALL(*bankMock, changeBalance(_,_))    .WillByDefault(Return(-1));   BitcoinWallet* wallet = new BitcoinWallet();   ASSERT_EQ(wallet->updateLocalBalance(),ActionStatus::AccountNotExist); }

With TEST_F

TEST_F(BitcoinWallet_ULT,updateLocalBalanceTestNoAccount) {  ON_CALL(*bankMock, changeBalance(_,_))   .WillByDefault(Return(-1));  ASSERT_EQ(wallet->updateLocalBalance(),ActionStatus::AccountNotExist); }

I hope this helps with making your Unit Level Testing more effective.

DWP

VS2012 and Gmock - Using ON_CALL and EXPECT_CALL Macros - Video #6

This is the sixth video in a series on how to use GoogleTest and GoogleMock with the VS2012 framework. Before watching this video you should make sure that you have already setup your Visual Studio solution to have a googlemock, googletest and a unit level test project. If you do not know how to do this check out the previous videos/blogs in the series. Here at "Using VS2012 and Google Test Video #1" in this blog.

In this video I will show you how to use the ON_CALL and EXPECT_CALL macros to set up default behavior for a mock object and to set expectations on method calls to the mocked object.

ON_CALL macro

—This macro helps you set the behavior of the call to the method for the specified object. —This is the default behavior of the object when the method is called with the specified arguments (matchers). The macros can be called as follows.

ON_CALL(mock_object, method(matchers))
    .With(multi_argument_matcher)  ?
    .WillByDefault(action);

The mactchers allows you to specify the values of the arguments that are passed to the method. This gives you the flexibility to set behavior based on parameters passed to the method. Very powerful when you are writing white box testing. The .With allows you to specify multiple matchers for the method call. The WillByDefault method allows you to specify the default action to perform when the method is called. Most of the time that is "Return".

Matchers

Matchers let developers set the default behavior based on the parameters passed into the method specified in the macro. This gives you the ability to change the behavior of the mocked object based on the parameters passed into the method. Googlemock gives several matching macros:

• Wild Cards - 
• “_” - Match anything
• “A<type>()” – Match any type “int”, “char”, “Class”
• Comparators
• Eq(value), Ne(value), IsNull(), NoNull(), Ge(), Le(), …
• String Matchers
• StrEq(value), StrNe(value), ContainsRegex(string), …

Here is an example of using the ON_CALL macro to set the behavior of the method changeBalance when the first parameter is "MyAccount".

ON_CALL(*bbm, changeBalance(StrEq("MyAccount"),_))

  .WillByDefault(Return(100));

In this example any call to changeBalance with the first parameter equal to "MyAcount" will return 100. Pretty simple example, but you can see the power of being able to set behavior of a mock method in the test that is using the mock method. (High Cohesion, Low Coupling) :)

EXPECT_CALL macro

This macro is used to set the expectation of a mock method for a specific mock object. This allows you to write pinpoint white box testing for the class without having to write complex test frameworks for you specific architecture.

EXPECT_CALL(mock_object, method(matchers))

   .Times(cardinality)

   .WillOnce(action)

   .WillRepeatedly(action);

The Times, WillOnce and WillRepeatedly calls are options and can be used more than once.

Times

How many times will the method be called. If the number does not match then an assertion is set and the test will fail.

WillOnce

WillOnce specifies what we are expecting the function should do. The action that will take place. The order of WillOnce specifies the order of the expected actions. You can chain WillOnce calls together to show that a method is called several times with different behaviors. You can finish your change with WillRepeatedly to specify that the method can be called more times with specific action.

using ::testing::Return;
…
EXPECT_CALL(turtle, GetY())
   .WillOnce(Return(100))
   .WillOnce(Return(200))
   .WillRepeatedly(Return(300));

There are several good resources availble on the google mock site:

• GoogleMock Cheatsheet
• GoogleMock for dummies
• GoogleMock Cookbook

I hope this helps you understand googlemock a bit more and helps you move to a more Test Driven Development model. Happy developing.

DWP

VS2012 and Gmock - Writing your mock classes

This video is the fifth in a series of videos that show how to use VS2012 and google test and mock. Before watching this video you should make sure that you have already setup your Visual Studio solution to have a googlemock, googletest and a unit level test project. If you do not know how to do this check out the previous videos/blogs in the series.
In this video I will show you how to write a mock class and then how to use the mock class in writing you unit level tests.



First you should understand what a mock class is. A Mock is fake object in the system that replaces a dependency and verifies whether the object under test interacted as expected with the dependency.

To understand this better check out the architecture of the BitcoinWallet Class.

BitcoinWallet has dependency on three other classes. BitcoinBank, BitcoinTransaction, and BitcoinCommerce. But we only want to focus our testing on BitcoinWallet. We don't want to call the implementation of BitcoinBank. BitcoinBank could have an implementation that actually calls a bank. That would be a bad thing if every time we ran our unit level tests it transferred money from one account to another. :) So we need to create a Mock to replace the real classes.

Put the mocks in place allows Bitcoin to be tested by itself and without exercising the other classes in the architecture. Here are some examples of what the mock classes look like.

#include <gmock/gmock.h>
#include "BitcoinTransaction.h"

class BitcoinTransaction_Mock : public BitcoinTransaction {
public:
MOCK_METHOD3(set,void(const string&, const string&, long));
MOCK_METHOD0(amount, int(void));
};

There are a couple things you need to watch out for when using Google Mock.

• Only virtual methods can be Mock(ed)
• If you don't have virtual methods check out the documentation on gmock on how to work around the problem. https://code.google.com/p/googlemock/
• You must always put a mock method definition (MOCK_METHOD*) in a public: section of the mock class
• Your Mock Method must match the cardinality of the real method.
• Mocked classes do not do anything. They just pass back data.

Check my next video where I discuss how to use ON_CALL and EXPECT_CALL macros. But here is a small example of how to use the Mock classes and the ON_CALL macro.

#include <gtest/gtest.h>

#include <gmock/gmock.h>

#include "BitcoinWallet.h"

#include "BitcoinBank_Mock.h"

#include "BitcoinCommerce_Mock.h"

#include "BitcoinCurrency_Mock.h"

using ::testing::Return;

using ::testing::_;

BitcoinBank* ExternalService::theBank = new BitcoinBank_Mock();

BitcoinCommerce* ExternalService::theCommerce = new BitcoinCommerce_Mock();

BitcoinCurrency* ExternalService::theCurrency = new BitcoinCurrency_Mock();

TEST(BitcoinWalletTest,updateLocalBalanceTestNoAccount) {

   BitcoinBank_Mock* bbm = new BitcoinBank_Mock();

   BitcoinWallet bw;

   ExternalService::theBank = bbm;

   ON_CALL(*bbm, changeBalance(_,_))

  .WillByDefault(Return(-1));

   ASSERT_EQ(bw.updateLocalBalance(),ActionStatus::AccountNotExist);

}

VS2012 and GMock - Getting Started Video #4

Using GoogleMock and VS2012 can be tricky. This video shows how to get things started so you aren't wasting time trying to figure out include paths, Pre-Processor directives and project configuration.
This video does not show you details of gmock just how to get it setup and begin using in your project. The next video will go into more details on how to use gmock effectively.

1. Compile googlemock into a library.
• Create a new project of static library: Add->New Project->Win32 Project->Static Library (no precompiled header)
• Add include path <gmock_dir>, <gmock_dir>/include, <gtest_dir>, and <gtest_dir>/include
• Add source file <gmock_dir>/src/gtest_all.cc and gtest_main.cc
• Change the pre-processor rules to include the definition._VARIADIC_MAX =10
2. Add references to Your Unit Test project.
• Add the gmock project as an external reference.
• Add the gmock include directories to Include Path <gmock_dir> and <gmock_dir>/include

When you have completed these steps you can now try and Mock one of the classes in the solution. The following is an example of a mock for the BitcoinTransaction class in our bitcoin solution.

BitcoinTransactionMock.h
#include <gmock/gmock.h>
#include "BitcoinTransaction.h"
class BitcoinTransactionMock : public BitcoinTransaction {
public:
   MOCK_METHOD3(set,void(const string&, const string&, long));
   MOCK_METHOD0(amount, int(void));
};

Now that I have a Mock(ed) class I can use the mocked class in my tests.

#include <gtest/gtest.h>
#include "BitcoinTransactionMock.h"
#include <gmock/gmock.h>
using ::testing::Return;
TEST(BitcoinTransactionMock,setGoodValues) {
   BitcoinTransactionMock* bt = new BitcoinTransactionMock();
   ASSERT_NE(bt, nullptr);
   ON_CALL(*bt, amount())
      .WillByDefault(Return(100));
   bt->set("MyAccount","YourAccount", 1000);
   ASSERT_EQ(bt->amount(), 100);
}

In this example The Macro ON_CALL is used to set the return value of the call to amount.
The next video and blog will go into more depth on how to use gmock and the benefits of isolation in unit level testing.

DWP.

VS2012 & Google Test - Running Unit Level Tests - Video #3

This is the next in a series of videos on how to use VS2012 and Google Test. The typical "Ctlr-F5" in VS2012 will bring up and run the unit level tests for the project. But that can be hard to filter through all of the tests that are run. Especially when you are running hundreds or thousands of tests in your project.

This video will show you how to run unit level tests in the VS2012 using a gtest addin. It will show also how to run unit level tests from the command line. 

The video covers the following items.

1. Adding the Google Test Add-in
2. Run Tests(All, Selected, Failed)
3. Debugging Tests
4. Running from the command line
5. Integrating into CI

First we need to download and install the visual studio google test add-in. It can be found at http://visualstudiogallery.msdn.microsoft.com/f00c0f72-ac71-4c80-bf8b-6fe381548031/file/87766/3/GoogleTestAdapter.vsix.
Once you have install this you will need to restart visual studio.

Now you will see a new menu item TEST. Select TEST and you can see many options for you, including Run All, Run Selected, and Run Failed. This plug in makes it much easier to work with your tests and your code. You can also debug your tests with this plugin. Just select the Debug Test options in the TEST menu.

Integrating your Unit Level Tests into you build or continuous integration system requires running the tests from the command line. Google Test has that ability as well. But you will need to write your own main program. First create a main.cpp file and put the following code in the file:

int main(int argc, char** argv) {
  // This allows the user to override the flag on the command line.
  ::testing::InitGoogleTest(&argc, argv);

  return RUN_ALL_TESTS();
}

It is important to include the ::testing::InitGoogleTest call as it sets up the options for running the tests. This allows us to pass command line arguments to the executable.Some of the most common arguments are:

• --gtest_filter=* - Runs all the tests
• --gtest_filter=BitcoinTransaction.* - Only run the BitcoinTransaction tests
• --gtest_repeat=1000 – Repeat tests 1000 times
• --gtest_repeat=-1 – Run forever
• -- gtest_output=xml – Great for CI that can parse junit
• --help  - list all of the options for running test commands.

Hope this helps with the velocity and quality of your product development.  Check out my next videos that will discuss architecting tests and using test mocks.

DWP

VS2012 and Google Test - Using TEST, ASSERT, and EXPECT (Video #2)

This video shows the basic steps to writing Tests in the googletest framework. This is a followup video to Getting Started. When running Unit Level Tests you want to be able to quickly find the tests that failed and get to the code and conditions that caused the failure. One of the tricks you can use is to come up with a naming standard and to use them.

Here is a naming standard that I use:

• Test Filename - Name of the class or file testing followed by _ULT
• Test Case Name - Name of the class or filename that you are going to test
• Test Name - Name of the method followed by the condition you are testing.
• methodName-PositiveCondition
• methodName-NegativeCondition

Using these naming standards in googletest is easy using their TEST Macros.
TEST(testCaseName, testName) {
...
}
Where testCaseName is the name of the Test Case and testName is the name of the test. Here is an example of these standards in action.

BitCoinTransaction_ULT.cpp

TEST(bitCoinTransactionULT,addMoney-PositiveAmount) {
...
};

TEST(bitCoinTransactionULT,addMoney-NegativeAmount) {
...
};

In this case we added two tests both in the bitCoinTransactionULT Test Case. Both tests are testing the method addMoney. The conditions PostiveAmount and NegativeAmount are being tested. It is ok to be descriptive. It will make it much easier to find out what is going in when you are looking through test results of hundreds if not thousands of tests.

Here is a quick overview of ASSERT and EXPECT.

—ASSERT_*

—If an assert fails it causes a fatal failure and returns from the current function.

—ASSERT_TRUE(condition) - True

—ASSERT_FALSE(condition) - False

—ASSERT_EQ(expected, actual) - Equal

—ASSERT_NE(expected, actual) - Not Equal

—ASSERT_LT(val1,val2) - Less Than,

ASSERT_LE(val1,val2) - Less Than Equal To,

ASSERT_GT(val1,val2) - Greater Than,

ASSERT_GE(val1,val2) - Greater Than Equal To

—EXPECT_*

—If an expect fails it does not cause a fatal failure. It just flags the failure in the logs

—EXPECT_TRUE(condition) - True

—EXPECT_FALSE(condition) - False

—EXPECT_EQ(expected, actual) - Equal

—EXPECT_NE(expected, actual) - Not Equal

—EXPECT_LT(val1,val2) - Less Than,

EXPECT_LE(val1,val2) - Less Than Equal To,

EXPECT_GT(val1,val2) - Greater Than Equal,

EXPECT_GE(val1,val2) - Greater Than Equal To

Using VS2012 and Google Test - Getting Started #1

I recently helped teach a class on Unit Level Testing using GoogleTest and VS2012. It was not a trivial as I thought it would be. But once I figured out some tricks to get everything running it works great. This blog and video is the first in a series of posts that will show how to effectively perform Unit level testing using GoogleTest and VS2012.

Setup GoogleTest in VS2012

• Download googletest https://code.google.com/p/googletest/downloads , and extract to a directory:
• Create a new project.
• Right-mouse click on the Solution, and Add->New Project
• Name the project googletest
• Create a new project of static library: Add->New Project->Win32 Project->Static Library; Uncheck “precompiled header”
• Compile googletest into a library.
• Add include path , /include
• Add source file /src/gtest-all.cc and gtest_main.cc
• Change the pre-processor rules to include the definition. Project->Properties->C/C++->Preprocessor->Definitions Add _VARIADIC_MAX=10 (warning! Make sure not to have spaces in this definition)
• Build the Project (F7)
• Add a unit level test Project
• Right-mouse click on the Solutuion, and Add->New Project
• Create a new “Win32 Project-> Console application” project; Call it BitCoinTest; Uncheck “precompile header”.
• Add include path and References…->Add Reference check both Bitcoin and googletest
• Change the pre-processor rules to include the definition _VARIADIC_MAX=10.
• Right-click on the project, and “Set as StartUp Project”
• Build the solution
• Run the default Project (Ctl-F5)

Look for the next blog GoogleTest and VS2012 - Writing you first Test Cases.