APITester was originally designed to test REST API’s and Thrift calls. It’s designed using Python with Django framework and hosted as a web-server. It has a simple UI interface to build tests, execute them and view results. The framework has evolved over time from just being an automation tool into an automation platform which can deliver some of its own features as a service.

History:

Let’s understand how the REST API / Thrift call were tested earlier.

To test any REST API:

  1. Tester needs to generate XML requests with varied combination of Query Param, Header and Body.
  2. They had to ensure that the input values met the minimum requirement and the XML syntax was correct.
  3. After building the request it was executed using a Third party tool which can make REST calls like Dev HTTP Client.
  4. The XML response is then analyzed to see if the request worked as expected.
  5. Every time to generate a request, they had to modify the required params in the XML and re-execute it.
  6. At times they had to validated the result in the Database too.

For Thrift based testing:

  1. Tester had to understand the various Methods, Objects declared in Thrift service.
  2. Use one of the supported language to make those Thrift calls.
  3. Execute them in the same machine in which these service were hosted.
  4. Validate the Test by looking at the response and checking the Database.

In the above two scenarios, the Tester spent most of the time in:

  1. Generating the XML / Thrift requests.
  2. Analyzing the complex XML / Thrift responses.
  3. Connecting to various Database and Thrift servers for validation (4 DB servers + 1 Thrift server per Cluster)

The Simplified Flow:

Let’s see how APITester simplifies this:

  1. Tester adds the testcase which has the required params to make the request and what needs to be validated in the response inside the APITester.
  2. Choose the Cluster/ Organization they would like to run the test on in the UI.
  3. Execute it.

What happens in the background:

  1. The tool auto-generates all the dynamic values in the request.
  2. Converts it proper XML / Thrift Objects.
  3. Executes it on the right Cluster/Organization.
  4. Validates the response with various assertion techniques.
  5. Queries the Database and/or the Thrift server for the respective Cluster to validate the result.

For this, Tester needn’t know any language. All the tests are written in simple JSON format. Even if they are not comfortable writing JSON testsuites, we have a Test Builder. This helps in converting the Request and Assertion into JSON steps.

Below is a sample request for an API test:

testcase

  1. We set a base-state for the testcase by setting Primary Key registration as Mobile.
  2. Add a Customer using customer/add API.
  3. Query back the added customer using customer/get API.
  4. Validate that customer is added by using XML Assertion.

The tool auto-generates the sample data for mobile0, email0, datetimeY+0 and replaces them in the request. It also ensures to map the same set of data while validating the response. The auto-generation is not limited to pre-defined values like mobile, email but for other parameter as well. In the above request, if we add “some_id”: “some_id0” then the tool auto-generates even for the tag “some_id” without any modification in the backend.

We have various options while building the Requests and Assertions:

  1. Declare Variables and refer it back in further steps.
  2. Make a batch request ( 100 of customers added in a single request)
  3. Repeat the same request multiple times.
  4. Assert the entire XML response or few specific values in it.
  5. Assert the length or type of the response.

Once the testcases are created, we have the option to execute just a single test or a group of tests representing a feature or the complete Regression. The session management page helps in choosing the required tests and saving them for future use. It also helps user to choose the Cluster / Organization on which the test needs to be executed and this choice is saved per User. The test execution can also be triggered from a REST API. This is useful for automated triggers like Continuous Integration or even in case when an Organization is not defined in UI.

When a tests is fired, the tool checks for mismatch in the selected Cluster / Testsuite and reports the same if found. If there is no mismatch, then it automatically redirects to the result page where the live status of each tests is shown. The result has a quick summary and a detailed report. The quick summary has an Abort button to cancel the currently running test. The detailed report lists all the completed testcases with status, Assertions, execution time and a debug log file. It also has a Re-run option. This is useful to run any failed tests or subset of the existing test set. Once the execution is complete, it sends a copy of the result over email. Multiple tests can be triggered in parallel on different Clusters / Organization and the tool handles it seamlessly. The tool also supports Keyboard Shortcuts to ease the navigation across web-pages and for triggering tests.

With this setup, Tester can focus more on building tests and leave the execution part to the tool. The execution process is so simple that Developers could run it on their clusters to validate the changes before it’s pushed to QA and Devops could run it on Prod clusters to do a sanity check whenever there is a logistic management in the Cloud.

 

Architechture:

apitester

The above figure shows a brief overview as to how APITester is designed and it’s interaction with various components.

Automation as a Service:

One of the powerful feature of this tool is the re-usablity of its components not just within the automation but beyond it. It’s quite common that most of the automation would need to hit the Database / Thrift servers either to validate or fetch a value to complete a Request. APITester has exposed many API’s which helps in supplementing these requirements. Instead of each team setting up their own pool of connections, this helps in sharing / re-using the existing setup across teams. Infact the tool makes it so simple that the user who makes this request needn’t know the logic of creating complex objects to make the Thrift calls.

The future of APITester is to ensure that many of it’s components are exposed as an API. With this, Automation across teams can make basic requests like Add a Customer or make a Transaction or even run a group of tests which simplifies their automation needs / demands.