Home

Blog

Case Studies

Services

Contact

Imagine using an automation testing tool that saves you and your team as much as 90% of your testing time. Let's say your team spends 10 hours per week testing. That's about 36 hours every month or 468 hours per year your team could be putting effort in elsewhere.

That's what a customizable test automation framework can do for you. It can give you back time and resources that can go toward improving your project or your business. It will also give you better peace of mind that your product is running smoothly and efficiently.

This modular, API framework took our team of two engineers just 100 hours to build. While we've built many projects over the years, this one was particularly meaningful. Soon after developing this customizable test automation framework, we were able to configure it for one of our customers--a San Diego-based biotechnology firm--in just 60 hours. They used this framework for their cancer treatment diagnostics software.

Right out of the gate, we knew we were building a tool that could truly help change the world. The framework would directly impact the quality of our customer's diagnostic software, and our customer could in turn provide better information for cancer patients and their families to make the most informed treatment decisions.

In this article, we dig into the "nuts and bolts" behind the custom, modular API testing automation framework we built: why we built it the way we did, how it works, and the benefits it provides.

1. Showcasing Our Technology Stack: What We Chose and Why
2. Why We Built Our Own Modular API Testing Automation Framework
3. A Look Under the Hood: How Our Custom Testing Framework Works
4. Total Time to Build & Test Our Framework

In the ever-changing world of software development, with different teams building and testing different segments of code at any given time, it's important to develop a standard of testing that everyone can agree on. When developers are assigned to different components of an application, such as an underlying framework or APIs, they will implement unit tests.

Unit tests are pieces of code that make claims about how parts of an application--its libraries, frameworks, APIs, and so on--will behave under certain conditions. Developers use these tests to determine when a change in one unit interferes with the functionality of another.

This is where the importance of a Test Automation Framework comes into play. A testing framework is a set of guidelines, best practices, tools, and rules used for creating and designing test cases so QA professionals can test their code more efficiently.

There are millions of applications on the market and they all serve a particular need, or solve a particular business problem. But what if your application connects cancer patients to their targeted therapies? What if you had the chance to build a diagnostics software that allows doctors and researchers to understand, on the basis of blood samples, for instance, what types of cancer treatments will be most effective before any administration of treatment. 

This would have a considerable impact on the quality and comfort of cancer patient care.

In building our testing framework, there were four primary technologies we chose.

Node.js

We developed our testing framework in Node.js and JavaScript for the simple reason that we wanted to run our framework in any environment and these technologies are ubiquitous and cross-platform.

TypeScript

We wrote the actual code in TypeScript, which is a syntactical superset of JavaScript. The advantage here is that TypeScript provides more structure, and decreases the chances of breaking the core code. Also, TypeScript compiles to JavaScript in the runtime, so we were able to run our framework on top of Node.js. This essentially means that we are truly a cross-platform framework -- run anytime, anywhere.

Mocha

For our testing environment we used Mocha. Mocha is a feature-rich JavaScript test framework for Node.js programs, featuring browser support, asynchronous testing, test coverage reports, and use of any assertion library.

InversifyJS

We also used InversifyJS, which is a lightweight inversion of control (IoC) container that can be used with TypeScript and JavaScript systems. The advantage of InversifyJS is that it serves as a mechanism for automatically creating the class dependencies in TypeScript, therefore saving time and improving accuracy.

After researching ready-made test automation frameworks on the market, we soon realized that none of the out-of-the-box solutions gave us everything we wanted in terms of customizability, maintainability, ease of use, and more. Our Technical Architect and senior-most Quality Engineer took to the task of building Integrant's own test automation framework, completing the framework itself in just 100 hours.

Our framework combines all the powerful and most effective features of ready-made frameworks from around the web into one easy-to-use platform. We have full control over its functionality and can easily perform basic REST operations and validations on the status code, message, headers, and body of the response.

Test Automation Framework Benefits

Some of the immediate benefits that we've seen from designing and developing our own customized, modular API testing framework, including best practices are below.

One of the goals for designing this platform was that it should function like plug-and-play devices. In other words, members of a quality assurance (QA) team should be able to use a framework even if they don't necessarily understand how it works.

Our framework requires no additional coding because it uses English-like syntax and you only need to remember one command to execute any or all test cases.

Our test automation framework needed to have a robust and transparent reporting mechanism, so our QA engineers could provide details on each test case, how many test cases executed, how many failed, and how many passed.

We wanted our API test automation framework to cover most test cases and scenarios without requiring extra development. We designed it so that new APIs could be tested on the fly, simply by creating a new YAML file -- and without writing a single line of code.

One of the key features we built into this framework was maintainability. This means that:

• Changes and modifications in the API are applied within the API file.
• Test data is also easily changed.
• Codes are maintainable and can easily adapt to any changes.
• Test cases do not need to be rewritten to be retested.
• Only one command can be used to rerun the tests.

The ability to reuse our custom framework for multiple projects was a key design decision in going with a custom solution. Not only does reusability make testing quicker, but it also lowers maintenance costs by simplifying processes for both developers and end-users alike.

Extending the code or framework, reconfiguring aspects, and setting up other test cases all become much easier to accomplish when reusability is built into the core structure.

We needed to ensure our testing framework would support all major applications, platforms, and operating systems. Technology stacks and digital ecosystems change all the time, so we wanted to guarantee our platform is forward and backward-compatible with all major systems.

There are a number of features and steps that describe the "nuts and bolts" of how our modular API testing framework operates.

The test scenarios for our modular API test framework are written in easy, human-readable YAML files. The custom framework executes the YAML files and analyzes the test scenarios.

Because YAML files use plain text and parse all testing scenarios, minimal development effort is needed to execute the API testing. In other words, this framework can do most of the API testing in plain text format, without requiring engineers to write or add extra codes.

An example of a YAML test file is shown here:

The YAML files also make the test framework extendable and allow it to complete more sophisticated, business-driven test scenarios.

After the framework runs the test scenarios, two files are generated. One is an HTML report that includes the number of passed and failed test cases, in addition to the comparison of expected versus actual results achieved.

The second file generated is a log file that can be used by QA teams and engineers for further debugging or logging.

Our customized API testing framework supports multiple features to ensure that all test scenarios are run as effectively as possible.

The before-each and after-each features are used to run a method before and/or after each test case. There are two benefits to this:

Saves engineers the hassle of having to write the API every time.

We created a robust but flexible dataset; attributes can easily be turned off and on and it’s easy to enter additional products to the dataset. 

The intense customization of this dataset and website helped our client’s sales team to discover products and sell products they didn’t even know they had in their own catalogs. 

A second feature of the framework is that it supports data-driven testing. This means that:

The same test scenario can be run on multiple test data by configuring multiple test data sets.

A list of variables can be added and the framework will run it as many times as the number of variables listed.

Write it once and then run as much as you want.

The main class functions of the test framework are easily accessible, which enables developers and engineers to extend the framework without writing any code. The API framework can be extended through API tasks, and then used to create a scenario that can be integrated with more than one system. This construction has the following advantages:

Modifications can be easily made within this framework. You just apply changes to the targeted test case and re-execute the desired scenarios and test cases with no code modifications needed.

Developers can write their own JavaScript code into the YAML file. The framework will digest the JavaScript in the YAML file and execute it. Using this methodology, developers can extend the testing scope and add functionality without modifying the source code.

Using best practices we've learned through our extensive testing experience, our team spent 100 hours building and then 60 hours customizing a modular test automation framework.

Maintaining and Modifying the Stack

In looking back on the life-cycle for our testing framework, we determined that there are two major changes we would implement for the future.

Indemnify JavaScript

Sometimes a client will ask for certain customizations that need to be done either by writing the move files or by indemnifying the JavaScript. In these cases, we need the ability to touch the code so we can customize it for a particular file type.

Framework Customization

The second change we would implement is to make it easier to add a common functionality in this framework, such as telling the system to force out "verbs" from the test scenarios. We also need to cover batch testing of our test scripts.

In the case of one biotechnology partner, our framework covered 23 APIs and 460 test cases. As a result of our work, our customer was able to save as much as 90% of their time with this automated framework.

Ease of use and maintainability were features that made our custom test automation framework particularly attractive. Adding test cases is as easy as opening a new document in Microsoft Word.

The tradeoff is clear: by saving time with automated software testing, our partners can significantly boost their productivity. Instead of focusing on software testing issues, you focus more energies on continuing to build effective, innovative solutions in your field whether they be in cancer diagnostics, manufacturing, and more.

Do you want to see what our test automation framework can do for you and your team? Schedule a demo with one of our test automation experts today!

Company

Contact

Follow US

About Us

Address: 16870 W Bernardo Dr, Suite 250

San Diego, CA 92127

Email: info@integrant.com
Phone: +1 858.731.8700

Privacy Policy

© 2021 Integrant, Inc. All Rights Reserved