loader image

Test Lösungen für Simulink Modelle und Seriencode

Challenges of Virtual Validation for ADAS/AD

How can I create the needed millions of scenarios?

How can I avoid the test explosion problem?

How can I find out if my tests are passed or failed?

Blog

What you should know about code coverage

Wolfgang Meincke

Stuttgart, Germany

Each time we talk about testing, we also talk about code coverage. But why is it important to keep an eye on code coverage and what are possible code coverage goals?

What is code coverage?

Code coverage is the sum of all coverage goals that are covered by a set of test cases. Coverage goals are e.g. statements or decisions. The ISO 26262 names coverage goals like statement, decision/branch and modified condition/decision coverage for unit testing. Depending on the risk class (=ASIL level) a subset of these coverage goals are highly recommended. Nevertheless, these coverage goals are just a subset of the available coverage goals for C Code in general. There are more coverage goals like function, function call, switch case, relation operator coverage. In addition, it is useful to consider functional scenarios, derive equivalence classes, boundary value checks or individually specified critical situations. For all the coverage goals the tester wants to achieve a high (if possible) 100% coverage. Beside the code coverage goals named before, we should consider robustness goals like division-by-zero, critical down casts, range violations and unreachability as well.

It is not easy to reach high coverage results with handwritten test cases. However, based on a technology called Model Checking it is possible to generate tests automatically and to create a mathematical proof, that a certain coverage or robustness goal can never be reached. As a result, all available coverage goals are analyzed and get reported as either covered or unreachable.

Let’s have a look on some samples of typical coverage goals. Coverage goals can have a different amount of properties that can either be covered or unreachable. For all of the coverage goals high coverage should be achieved.

Main Code Coverage goals
Extended code coverage goals

Now let’s have a look at robustness goals. For these code coverage goals an unreachable result is expected. Otherwise this might lead to unintended behavior.

Robustness code coverage goals

Of course, not all projects focus on all of these code coverage and robustness goals but a sufficient subset that fulfills the project safety goals. In general, there are three levels test goal are related to:

  • Unit Testing level with a detailed focus on the implementation and complex test goals like MC/DC.
  • Integration level that focuses on the interaction of several units and components
  • Architecture level that focuses on the whole software project and
 

What test goals are taken into account for each level is based on the risk class of the component and is usually defined by the customer and the quality assurance. 

Code Coverage is derived with test cases. Also have a look at the blog article “What are the differences between functional and structural test cases?“.

Wolfgang Meincke

Stuttgart, Germany

Senior Application Engineer
Senior Business Development Manager India

Wolfgang Meincke studied Computer Science at the University Ravensburg-Weingarten where he graduated in 2006. He then worked at EWE TEL GmbH where he was responsible for requirements engineering and project management for software development projects as well as agile software development processes. In 2014 he joined BTC as senior pilot engineer to support the customers to integrate the tools into their development processes as well as giving support and training on test methodologies regarding the ISO 26262 standard. One of his main areas of interest is the formalization of safety requirements and their verification based on formal testing and model-checking technologies for unit test, integration test and HIL real-time-testing.

Connect on LinkedIn

Popular Videos

Video abspielen
Video abspielen

Request Evaluation License

If you would like to try out our tools, we will gladly provide an evaluation license free of chargeEvaluations include a free launch workshop and also provide an opportunity for you to meet one-on-one with our support and engineering teams.

Schedule a Meeting

Do you have any questions or want to see our tools in action? If so, please use the link below to schedule a meeting, where a member of our engineering team will be happy to show you the features and use cases and directly answer any questions you might have.

Melden Sie sich für unseren Newsletter an!

Für den Newsletterversand wird Ihre Email Adresse vom DSGVO zertifizierten Dienstleister Newsletter2Go verarbeitet. Weitere Informationen finden Sie in unserer Datenschutzerklärung.

Videos

Discover some of the main features of our products in these short videos.