A Novel Framework Design for Test Case Authoring and Auto Test Scripts Generation

Every product has defects and identifying defects in the process of development and rectifying them before the launch of the product is very important. Embedded software testing process find the bugs in the software and report to the developer to fix issues. Sometimes to meet the product release deadlines, test engineers will not get much time to cover all test cases. That is why most software testing depends on test automation. In this paper, we focused on the area of automotive and home appliances embedded software test automation. Test automation is the only solution to improve the test phase and meet the timeline of the product launch. There are many test Automation tools like LabVIEW, test stand, and automation desk to automate testing embedded software. However, there is still manual efforts are required to use these tools. This paper deals automate those manual efforts. This Works shows how to generate test scripts from test cases to reduce the manual efforts, time, and cost.


Introduction
Embedded Software Development Life Cycle aims to deliver high-quality product software that satisfies all the requirements. Its starts with Planning and requirements analysis where senior people design product, based on market requirements and technical experts' suggestions. Once the product design has been done then the next stage is to document called SRS (Software Requirement Specification) the product requirements and get the approval from the client. The SRS document decides the product architecture and gets reviewed by the top management for final changes. Based on the SRS document, software developers develop the product software and develop test cases that cover all software functionalities. The software quality assurance team used the test cases and performed either manual Regression or do automation testing. SWQA team used test management software to automate the manual test scenarios. If there is any part of the software updated, the testing team must perform entire software testing to find any other aspects affected with part of the software update, and this is called the regression test.
Software developers follow one of the common software life cycle development models which are mainly waterfall model, Iterative model, Spiral model, V-model and Agile model. The waterfall model divides software development into different phases, and each phase depends on the previous phase. Moreover, it is not suitable for complex projects as it has high risk and uncertainty. The iterative model repeatedly improves the developing versions until the full system is ready as this suite for large projects, but more resources are required. In the spiral model, the development covers three phases like identification, design and build. Requirement changes can easily manage in the spiral model, but the process is complicated. In the Verification and Validation model, implementation of processes happens sequentially like a V-shape. Here each phase of development is directly linked with testing. The V -model is best suited for a small project where requirements are well defined, but it is not suitable for significant and ongoing projects. The agile model divides the product into minor incremental builds and a very realistic approach to software development. These builds follow the iteration method. Each iteration naturally takes from about one to two weeks. Every iteration contains sub functional teams working simultaneously on various areas.
Manual regression is essential for software validation and its time-consuming task. The solution is test automation where one needs to create script manual test cases using any test management software. Later, scripts can be executed as many as times to get results reports and can reduce manual effort and time. In embedded software industries, mostly using test management software are NI Test Stand and dSPACE Automation Desk. The Test Stand is test management software and allows test engineers to develop automated test scripts using code modules written in any programming language. One of the popular languages is LabVIEW, and it is a graphical programming language. At the same time, the automation desk is using mostly in the automotive industry for Hardware in loop test automation. Software testing is a crucial stage in the software development and maintenance cycle. It has been estimated that software testing involves between 30-50 percent of software development. With the wrong choice of testing, the software development process may become costly, inefficient and causes a high frequency of product release delay. We can script manual test cases using test automation tools, but this is costly, and maintenance required highly skilled test engineers. Our framework gives the solution to this problem.

Related Work
Software developers seek test automation methods owing to the costs involved in manual testing. Reusable functionality is widely useful and can reduce engineering costs. Researchers worked on IEC 61131-3 programming languages and classified script generation methods. The study provides a fast method for generation for automation in different industries. The full validation of system functions by simulation at the sub-system and component level can significantly reduce system simulation time. This paper implemented a new framework for automation in the wood and the chemical industry. The framework talks about the requirements of key intelligent automation features and data characteristics, also presented the development of support decision systems into automation modules. A Middleware for Intelligent Automation has been proposed in the experiment using Big data and cloud infrastructure.
A robot operation and testing simulation framework have been proposed to detect logical and physical design flaws. The study uses High Level Architecture (HLA) as a middleware for real-time testing for any hardware device. Test automation approach reduces the software cost and improves the quality. An attempt has been made to measure the effect of automation of testing. A well-trained model can generate test scripts from the design document in the web application testing. For the experiment, a testing tool, TesMa was developed at the NTT Labs. Test automation method has been explored for various appliances employing embedded software. It introduced a novel method to select required test cases from large test suites to reduce time and cost.

Methodology
The proposed script generation framework is implemented in three phases. Phase-1 is the process of writing test cases as per the required format using the software requirements document. Phase -2 is the key part that generates test scripts using pre-defined library set based on intelligent logic. Phase -3 is about to execute scripts and to get test reports. Test script execution depends on the test management software. The three phases have been described in figure 1.

Test Case Authoring
Phase-1 aims to generate standard test cases and deals with Requirements documents, Natural Language Processing-based Test case Generation (NLP TCG) and Test case editor. Test case editor is an interface where we can write test cases manually using software requirement documents. Test case authoring is built on the structure of a pre-defined key with sections. The test case editor user interface is shown in figure 2.
Test steps are divided into three groups: setup, Main steps and Clean-up. Setup group covers pre-condition steps, Main steps group covers the actual test steps of the test scenario, and clean group covers the postconditions of the test scenario.
Five sections are provided to write the test step to cover most of the functionalities.

Action:
Action has the following key terms to define the action of the test step. We used four key terms write, read, set and check to cover the significant test actions. We can add new key terms into action depending on the requirement as shown in figure 3.  The keys belong to this section to determine the service type of test step. For example, if we are using protocols like LIN, CAN or Ethernet in software development, writing into these protocol bus messages and reading from these protocol bus steps can be determined here as shown in figure 4.

Figure 4
Sample key set at Function/Service type section

Parameter:
Test parameterization is an efficient method and utilizing parameters rather than fixed values. So, one can execute the same test with different input values of test steps used as inputs or expected outputs that are stored separately from the test steps. Test Parameterization is a powerful practice to enhance the testing scenarios and reuse the same. During the execution of test steps, the parameters are filled in with their actual value, so a parameterized variation of the original test case is produced. The main benefits of parameterization are as follows: • By externalizing the changing parts of a test case(s) as parameters, it is easier to manage complex test cases • Users can automatically execute multiple variations of each test. Multiple Parameter values provided that will cause several executions of each test for each variation of the parameters.
Test Parameterization allows sharing information between multiple test cases.

Value:
In this section, we can define the value of the parameter and can define the expected output value. The remarks section can use to explain any additional notes, which is useful to consider complicated test steps.
Oven, a home appliance used for baking food items and given below is sample oven software requirement. The oven light should turn ON when the oven door is open, or the oven light switch is turned on. Also, the count of oven light ON and OFF, count of the door open and close should be stored. Test case writing using the proposed framework test case editor is simple and easy to understand as shown in figure 5.

Figure 5 Sample Testcase
Embedded software development uses standard documents like JSON or XML files to maintain microcontroller variable data. Here we do parse these files and save them into test label data. Test label data useful write test cases easily. OvenDoorOpenCloseCount, Oven Door and oven light are the test label which parsed from the oven software JSON file.

Phase -2: Test Script Generation
Test steps library based on test management software required to prepare before script generation. Test steps library is part test automation process. Test steps library need to fill with all basic functionality with respective test management software's code modules. In NI test stand mostly, LabVIEW code module used, and for Dspsce automation desk python code used. The test configuration is part of the proposed framework which configure test management software. Script Generation Engine is the business logic which generates test script taking test cases as input and identify the functions based on key terms then replace parameters and values. For example, Step1 action key is Set, the Service type is Function, Parameter is Test Unit and value is Power ON. Based on this combination, it identifies the test unit function and set the write value as Power ON. Research Article Vol.12 No.6 (2021), 1479-1487

Figure 6 Test Script generation block diagram
The general process takes a minimum of 6 hours to write test cases and for scripting 8 hours. So total time approximately 14 hours to automate this requirement testing. Our proposed framework takes 4 hours to write test cases and script generation and validation take a maximum of 15 minutes.
The Script generation Engine has almost had all combinations of logic to generate script generation. Moreover, it is practically not possible to write all combinations of logic. Artificial intelligence and machine learning-based algorithm to predict the unknown combination have been introduced.
Machine learning is part of the Artificial Intelligence technique which will minimize the code to cover unknown cases. Supervised Machine learning algorithms are used to train the model using labelled data, such as an input where the desired output unknown. We trained the script generation model using the k-nearest neighbours (KNN) algorithm with pre-defined key section combinations to predict the test function from the test library for the unknown combination of input keys. However, we have not tested much prediction accuracy and precision. We will discuss the ML algorithm related information in future work.

Phase -3: Test Script execution
The Scripts generated in phase-II is loaded into test management software which is used to execute the test scripts and will get the automation report. We can see the sample automation report in figure 9.
As per the above

Figure 9
Test Automation Report generated by Test Stand Table 1 shows the time required for traditional agile test automation and new approached test automation.

Conclusion
The proposed method is very useful in the field of test automation to reduce the time and cost. This paper need improvements in the selection of right library for key combination. In future, this library selection can replace with trained model with machine learning algorithms.