loader image

针对Simulink模型和量产代码的测试解决方案

针对 ADAS/AD 虚拟验证的挑战

如何创建所需的数百万个方案?

如何避免测试爆炸问题?

如何确定我的测试是通过还是失败?

Blog

Should I test the Model or the Code?

Wolfgang Meincke

Stuttgart, Germany

Model-based software development (MBD) with tools like Simulink and TargetLink in the automotive industry has grown over the past years and is a well-established development methodology today. Despite all the advantages, it can’t be denied that MBD introduces one additional step in the development process. In addition to the production C code there is also the model level and test cases can be executed on both. But should we focus on model or code testing in order to get an efficient and ISO 26262 compliant process. Let’s have a look at our choices:

Test only the code

Model or Code Testing - Overview code only

Since many development teams have written their code manually in the past, they are very used to test on the code level. They might still use the same test solution as before because they are familiar with it. As the code is what finally goes into production, this approach is compliant to the ISO 26262 and therefore a valid approach for safety critical projects.

However, the code is still treated like the handwritten code from the past. The model is treated like some kind of PowerPoint and does not play a role during testing. This means first of all that we are missing out on one of the greatest benefits of MBD, as testing and debugging on the model is more transparent and convenient than on the code level. And even worse, if an issue is detected during testing of the code, the developer has to find the place in the model that generates the issue and fix it there. Once this is done the code needs to be generated again and only then can be tested to check if the problem is fixed (or not), which leads to time consuming iterations.

Test only the model

Model or Code Testing - Overview model only

The second option in the model or code testing comparison, is to test the model only. We take all the advantages that come with model-based development into account and do not care about the generated code. Testing and debugging are intuitive and transparent and it looks like a consistent approach.

The downside of this approach is that it is unclear how the code will behave. There are usually at least minor differences between model and code because you may use floating-point on the model and a fixed-point representation on the code. Even if both use identical data types the behavior can be different and code generators integrated in Matlab/Simulink (e.g. TargetLink or EmbeddedCoder) are not certified to always guarantee the same behavior on model and code level. This means we’d have no evidence about the correctness of the code which goes into production

This approach would also not be compliant with the ISO 26262, which asks for testing being done as close as possible to the final product.

Testing the model and the code

Model or Code Testing - Overview model and code

Testing on model and code level combines the advantages of both test approaches mentioned before and leads to an intuitive model-based testing workflow that also considers the code and therefore the final product. It is an ISO 26262 compliant approach and covers all aspects of testing. First, you get fast response if you test on the model and you keep also the focus on the code. The functional testing is done on the model. A back-to-back comparison ensures a correct translation into code.

This is also explicitly what the ISO 26262 standard recommends. Chapter 6, Section 9.4.6 NOTE 4 states, that software unit testing can be carried out on the model level, if followed by a back-to-back comparison between model and code. So it’s not a question of model or code testing but to look at both.

However, since we take two implementation levels into account now, the efficiency of this approach depends on the handling of the model and the code as well as the handling of the test data. This can become quite complex and lead to a not well integrated toolchain of different applications and scripts. It would be in particular very inefficient to have one test tool for the model and another one for the code. First of all, it would mean that you have to create a test environment for both of them. If an issue is fixed you need to go back and test again with both tools and beside the additional testing effort you have to take care to keep the test data consistent between the two tools. That creates a lot of effort and is easily prone to errors.

To face these challenges, an efficient and well-integrated tool support is needed that takes the model and the code into account, integrates different test methods for functional testing and back-to-back comparisons, provides automation capabilities to reduce manual effort and should be qualified for the ISO 26262 standard.

Conclusion

To summarize the model or code testing comparison, we can clearly see, that the model-based development is well-established today and will spread and improve even more.

To enjoy the benefits of the model-based development approach also in the testing process and still remain ISO 26262 compliant, the test workflow needs to take the model and the code into account. To face the additional effort of testing model and code, the test tool must be thoroughly designed to support model-based testing as well as code testing.

Wolfgang Meincke

Stuttgart, Germany

Senior Application Engineer
Senior Business Development Manager India

Wolfgang Meincke studied Computer Science at the University Ravensburg-Weingarten where he graduated in 2006. He then worked at EWE TEL GmbH where he was responsible for requirements engineering and project management for software development projects as well as agile software development processes. In 2014 he joined BTC as senior pilot engineer to support the customers to integrate the tools into their development processes as well as giving support and training on test methodologies regarding the ISO 26262 standard. One of his main areas of interest is the formalization of safety requirements and their verification based on formal testing and model-checking technologies for unit test, integration test and HIL real-time-testing.

Connect on LinkedIn

Popular Videos

播放视频
播放视频

Request Evaluation License

If you would like to try out our tools, we will gladly provide an evaluation license free of chargeEvaluations include a free launch workshop and also provide an opportunity for you to meet one-on-one with our support and engineering teams.

Schedule a Meeting

Do you have any questions or want to see our tools in action? If so, please use the link below to schedule a meeting, where a member of our engineering team will be happy to show you the features and use cases and directly answer any questions you might have.

Join our newsletter

Your email address will be submitted to the privacy-certified newsletter software Newsletter2Go for technical distribution. For further information go to our privacy policy.

Videos

Discover some of the main features of our products in these short videos.