How We Built a Customizable API Test Automation Framework in 100 Hours
Imagine using an automation testing tool that saves you and your team as much as 90% of your testing time. Let's say your team spends 10 hours per week testing. That's about 36 hours every month or 468 hours per year your team could be putting effort in elsewhere.
That's what a customizable test automation framework can do for you. It can give you back time and resources that can go toward improving your project or your business. It will also give you better peace of mind that your product is running smoothly and efficiently.
This modular, API framework took our team of two engineers just 100 hours to build. While we've built many projects over the years, this one was particularly meaningful. Soon after developing this customizable test automation framework, we were able to configure it for one of our customers--a San Diego-based biotechnology firm--in just 60 hours. They used this framework for their cancer treatment diagnostics software.
Right out of the gate, we knew we were building a tool that could truly help change the world. The framework would directly impact the quality of our customer's diagnostic software, and our customer could in turn provide better information for cancer patients and their families to make the most informed treatment decisions.
In this article, we dig into the "nuts and bolts" behind the custom, modular API testing automation framework we built: why we built it the way we did, how it works, and the benefits it provides.
In the ever-changing world of software development, with different teams building and testing different segments of code at any given time, it's important to develop a standard of testing that everyone can agree on. When developers are assigned to different components of an application, such as an underlying framework or APIs, they will implement unit tests.
Unit tests are pieces of code that make claims about how parts of an application--its libraries, frameworks, APIs, and so on--will behave under certain conditions. Developers use these tests to determine when a change in one unit interferes with the functionality of another.
This is where the importance of a Test Automation Framework comes into play. A testing framework is a set of guidelines, best practices, tools, and rules used for creating and designing test cases so QA professionals can test their code more efficiently.
There are millions of applications on the market and they all serve a particular need, or solve a particular business problem. But what if your application connects cancer patients to their targeted therapies? What if you had the chance to build a diagnostics software that allows doctors and researchers to understand, on the basis of blood samples, for instance, what types of cancer treatments will be most effective before any administration of treatment.
This would have a considerable impact on the quality and comfort of cancer patient care.
Showcasing Our Technology Stack: What We Chose and Why
In building our testing framework, there were four primary technologies we chose.
Why We Built Our Own Modular API Testing Automation Framework
After researching ready-made test automation frameworks on the market, we soon realized that none of the out-of-the-box solutions gave us everything we wanted in terms of customizability, maintainability, ease of use, and more. Our Technical Architect and senior-most Quality Engineer took to the task of building Integrant's own test automation framework, completing the framework itself in just 100 hours.
Our framework combines all the powerful and most effective features of ready-made frameworks from around the web into one easy-to-use platform. We have full control over its functionality and can easily perform basic REST operations and validations on the status code, message, headers, and body of the response.
Test Automation Framework Benefits
Some of the immediate benefits that we've seen from designing and developing our own customized, modular API testing framework, including best practices are below.
Ease of Use
One of the goals for designing this platform was that it should function like plug-and-play devices. In other words, members of a quality assurance (QA) team should be able to use a framework even if they don't necessarily understand how it works.
Our framework requires no additional coding because it uses English-like syntax and you only need to remember one command to execute any or all test cases.
Our test automation framework needed to have a robust and transparent reporting mechanism, so our QA engineers could provide details on each test case, how many test cases executed, how many failed, and how many passed.
We wanted our API test automation framework to cover most test cases and scenarios without requiring extra development. We designed it so that new APIs could be tested on the fly, simply by creating a new YAML file -- and without writing a single line of code.
One of the key features we built into this framework was maintainability. This means that:
- Changes and modifications in the API are applied within the API file.
- Test data is also easily changed.
- Codes are maintainable and can easily adapt to any changes.
- Test cases do not need to be rewritten to be retested.
- Only one command can be used to rerun the tests.
The ability to reuse our custom framework for multiple projects was a key design decision in going with a custom solution. Not only does reusability make testing quicker, but it also lowers maintenance costs by simplifying processes for both developers and end-users alike.
Extending the code or framework, reconfiguring aspects, and setting up other test cases all become much easier to accomplish when reusability is built into the core structure.
We needed to ensure our testing framework would support all major applications, platforms, and operating systems. Technology stacks and digital ecosystems change all the time, so we wanted to guarantee our platform is forward and backward-compatible with all major systems.
We help our clients create impact across many industries and in different fields. Check out our case studies!
A Look Under the Hood: How Our Custom Testing Framework Works
There are a number of features and steps that describe the "nuts and bolts" of how our modular API testing framework operates.
The test scenarios for our modular API test framework are written in easy, human-readable YAML files. The custom framework executes the YAML files and analyzes the test scenarios.
Because YAML files use plain text and parse all testing scenarios, minimal development effort is needed to execute the API testing. In other words, this framework can do most of the API testing in plain text format, without requiring engineers to write or add extra codes.
An example of a YAML test file is shown here:
YAML Test Scenarios
The YAML files also make the test framework extendable and allow it to complete more sophisticated, business-driven test scenarios.
After the framework runs the test scenarios, two files are generated. One is an HTML report that includes the number of passed and failed test cases, in addition to the comparison of expected versus actual results achieved.
The second file generated is a log file that can be used by QA teams and engineers for further debugging or logging.
Test Automation Framework Core Features
Our customized API testing framework supports multiple features to ensure that all test scenarios are run as effectively as possible.
1. Method Call (API Token)
The before-each and after-each features are used to run a method before and/or after each test case. There are two benefits to this:
Saves engineers the hassle of having to write the API every time.
We created a robust but flexible dataset; attributes can easily be turned off and on and it’s easy to enter additional products to the dataset.
The intense customization of this dataset and website helped our client’s sales team to discover products and sell products they didn’t even know they had in their own catalogs.
2. Data-Driven Testing
A second feature of the framework is that it supports data-driven testing. This means that:
The same test scenario can be run on multiple test data by configuring multiple test data sets.
A list of variables can be added and the framework will run it as many times as the number of variables listed.
Write it once and then run as much as you want.
3. Extensible Frameworks
The main class functions of the test framework are easily accessible, which enables developers and engineers to extend the framework without writing any code. The API framework can be extended through API tasks, and then used to create a scenario that can be integrated with more than one system. This construction has the following advantages:
Modifications can be easily made within this framework. You just apply changes to the targeted test case and re-execute the desired scenarios and test cases with no code modifications needed.
Looking for more than test automation? Check out what we can offer you!
Total Time to Build & Test Our Framework
Using best practices we've learned through our extensive testing experience, our team spent 100 hours building and then 60 hours customizing a modular test automation framework.
Maintaining and Modifying the Stack
In looking back on the life-cycle for our testing framework, we determined that there are two major changes we would implement for the future.
The second change we would implement is to make it easier to add a common functionality in this framework, such as telling the system to force out "verbs" from the test scenarios. We also need to cover batch testing of our test scripts.
Let's get together. Set up your free technical consultation and find out how we'll bring our vision, mission, and values to successfully execute your next project.
We Want to Help You
In the case of one biotechnology partner, our framework covered 23 APIs and 460 test cases. As a result of our work, our customer was able to save as much as 90% of their time with this automated framework.
Ease of use and maintainability were features that made our custom test automation framework particularly attractive. Adding test cases is as easy as opening a new document in Microsoft Word.
The tradeoff is clear: by saving time with automated software testing, our partners can significantly boost their productivity. Instead of focusing on software testing issues, you focus more energies on continuing to build effective, innovative solutions in your field whether they be in cancer diagnostics, manufacturing, and more.
Do you want to see what our test automation framework can do for you and your team? Schedule a demo with one of our test automation experts today!