{"id":14532,"date":"2023-08-09T14:58:19","date_gmt":"2023-08-09T12:58:19","guid":{"rendered":"https:\/\/www.btc-embedded.com\/?p=14532"},"modified":"2024-01-17T10:26:34","modified_gmt":"2024-01-17T08:26:34","slug":"what-you-shoud-know-about-testing-variants","status":"publish","type":"post","link":"https:\/\/www.btc-embedded.com\/de\/what-you-shoud-know-about-testing-variants\/","title":{"rendered":"What you should know about testing variants"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"14532\" class=\"elementor elementor-14532\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3b5f0493 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3b5f0493\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-559a247a\" data-id=\"559a247a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a1ff07f elementor-widget elementor-widget-text-editor\" data-id=\"a1ff07f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"TextRun SCXW1184418 BCX8\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW1184418 BCX8\">Creating <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">different variants of a software <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">component<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\"> is an efficient <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">way of<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\"> reus<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">ing<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\"> existing and developed features for different versions of a specific <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">component<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\"> or controller.\u00a0<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">Thus, testing variants is an\u00a0<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">ubiquitous <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">task in the automotive industry <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">that <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">needs to be <\/span><span class=\"NormalTextRun AdvancedProofingIssueV2Themed SCXW1184418 BCX8\">taken into account<\/span><span class=\"NormalTextRun SCXW1184418 BCX8\"> during <\/span><span class=\"NormalTextRun SCXW1184418 BCX8\">testing.<\/span><\/span><span class=\"EOP SCXW1184418 BCX8\" data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2cb1475 elementor-widget-tablet_extra__width-initial elementor-widget elementor-widget-heading\" data-id=\"2cb1475\" data-element_type=\"widget\" data-settings=\"{&quot;motion_fx_motion_fx_scrolling&quot;:&quot;yes&quot;,&quot;motion_fx_devices&quot;:[&quot;desktop&quot;,&quot;tablet_extra&quot;,&quot;tablet&quot;,&quot;mobile&quot;]}\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">How are variants created in a Model-based Software development process?<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-82a242a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"82a242a\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-4479338\" data-id=\"4479338\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-44fec12 elementor-widget elementor-widget-text-editor\" data-id=\"44fec12\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">When looking at the development process, we can distinguish between different moments in which the decision for a concrete variant gets \u201cfrozen\u201d:<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><ol><li><b><span data-contrast=\"auto\">During Model construction<\/span><\/b><span data-contrast=\"auto\">: The variant binding is realized by model adaptation. Only the active variant is visible in tools like Simulink or TargetLink. In this way, the model and the generated code are tailored to the current variant configuration.<\/span><span data-ccp-props=\"{}\"><br \/><\/span><\/li><li><b><span data-contrast=\"auto\">During Code generation<\/span><\/b><span data-contrast=\"auto\">: All variants are present in the model, but only one variant is\u00a0considered during code generation. As above, the generated code contains only one variant<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><li><b><span data-contrast=\"auto\">During Compilation<\/span><\/b><span data-contrast=\"auto\">: The variant configuration is done before compilation by preprocessor macros. The C Code contains all variants, but the binary code (=object code) reflects the variant configuration.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><li><b><span data-contrast=\"auto\">At runtime<\/span><\/b><span data-contrast=\"auto\">: The variant is selected at or before runtime, i.e. by changing a calibration variable or by configuring a piece of configuration memory. In this case, the same binary code is used for all variants.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><\/ol><p>\u00a0<\/p><p><span data-contrast=\"auto\">From the code perspective, variants 1 and 2 are identical. For variants 3 and 4, you will have one code file but for variant 3 the variant will be fixed during compilation while for variant 4 it can be modified later by setting the appropriate calibration values (even during runtime).<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-2fc3c16\" data-id=\"2fc3c16\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-96dcd1e elementor-widget elementor-widget-image\" data-id=\"96dcd1e\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"800\" height=\"1156\" src=\"https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/07\/Binding-Times-1.png\" class=\"attachment-large size-large wp-image-14565\" alt=\"What you should know about testing variants - Binding Times\" srcset=\"https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/07\/Binding-Times-1.png 1222w, https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/07\/Binding-Times-1-768x1110.png 768w, https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/07\/Binding-Times-1-1063x1536.png 1063w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffe7789 elementor-widget elementor-widget-heading\" data-id=\"ffe7789\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">What does this mean for testing variants?<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-db0e1cc elementor-widget elementor-widget-text-editor\" data-id=\"db0e1cc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">First things first. Before talking about the different aspects to consider during testing, let\u2019s define the term \u2018test project\u2019. A test project in the context of this article contains:<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><ul><li data-leveltext=\"-\" data-font=\"Calibri\" data-listid=\"3\" data-list-defn-props=\"{&quot;335551671&quot;:0,&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Calibri&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;-&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"0\" data-aria-level=\"1\"><span data-contrast=\"auto\">The model from which the production code will be generated\u00a0<\/span><\/li><li data-leveltext=\"-\" data-font=\"Calibri\" data-listid=\"3\" data-list-defn-props=\"{&quot;335551671&quot;:0,&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Calibri&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;-&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"0\" data-aria-level=\"1\"><span data-contrast=\"auto\">A simulation environment to drive the inputs and calibration values as well as read the internal measurement points and outputs.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><li data-leveltext=\"-\" data-font=\"Calibri\" data-listid=\"3\" data-list-defn-props=\"{&quot;335551671&quot;:0,&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Calibri&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;-&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"0\" data-aria-level=\"1\"><span data-contrast=\"auto\">A clear interface definition for the used signals<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><li data-leveltext=\"-\" data-font=\"Calibri\" data-listid=\"3\" data-list-defn-props=\"{&quot;335551671&quot;:0,&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Calibri&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;-&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"0\" data-aria-level=\"1\"><span data-contrast=\"auto\">Test data<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><\/ul><ul><li data-leveltext=\"-\" data-font=\"Calibri\" data-listid=\"3\" data-list-defn-props=\"{&quot;335551671&quot;:0,&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Calibri&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;-&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"0\" data-aria-level=\"1\"><span data-contrast=\"auto\">Reporting<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><li data-leveltext=\"-\" data-font=\"Calibri\" data-listid=\"3\" data-list-defn-props=\"{&quot;335551671&quot;:0,&quot;335552541&quot;:1,&quot;335559684&quot;:-2,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Calibri&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;-&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"0\" data-aria-level=\"1\"><span data-contrast=\"auto\">Optionally, requirements<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/li><\/ul><p><span data-contrast=\"auto\"><br \/>Next let\u2019s have a look at the constraints we have depending on the different binding times and also discuss the advantages or disadvantages.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dce2943 elementor-widget elementor-widget-heading\" data-id=\"dce2943\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-medium\">1. During model construction time <\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6cdaf06 elementor-widget elementor-widget-text-editor\" data-id=\"6cdaf06\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">If defining the variants in separate models you will derive a dedicated model and the corresponding code for each variant.<\/p><p style=\"font-weight: 400;\"><strong>Code Coverage<\/strong><\/p><p style=\"font-weight: 400;\">The derived or handwritten code represents one variant, this means that you should derive full code coverage to test the variant completely. This also applies for the model coverage and the code is a concurrent representation of the model.<\/p><p style=\"font-weight: 400;\"><strong>Test Interface<\/strong><\/p><p style=\"font-weight: 400;\">While looking at one variant, the test interface between model and code will be the same which enables an easy handling of test data for both instances and the test data can be reused for both as well. Apart from that this might not apply for other variants and the reusability of test data might suffer.<\/p><p style=\"font-weight: 400;\"><strong>Test Execution<\/strong><\/p><p style=\"font-weight: 400;\">The constraints from the test interface has a direct link to the execution records. At a first glance this seems to be a simple task as well because the test interfaces of model and code match and therefore you can simulate the test data on both instances, however, in the context of variants, you will have several models\/code for each instance and the reusability of the test data becomes difficult if the interface differ between the instances.<\/p><p style=\"font-weight: 400;\"><strong>Reporting<\/strong><\/p><p style=\"font-weight: 400;\">The reporting is also included in the previous limitations. You will get a separat report for each variant.<\/p><p style=\"font-weight: 400;\"><strong>ISO 26262<\/strong><\/p><p style=\"font-weight: 400;\">The ISO 26262 will be fulfilled by this approach. One model will be tested against its derived code and there are no unexecuted parts on one of the instances.<\/p><p style=\"font-weight: 400;\">This approach is useful if you have big differences between variants and it makes sense to keep things separate on that level. However, keep in mind that on implementation level this are not variants anymore but independent models\/code.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-20671d1 elementor-widget elementor-widget-heading\" data-id=\"20671d1\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-medium\">2. During code generation time <\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c87fa42 elementor-widget elementor-widget-text-editor\" data-id=\"c87fa42\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">The next approach of handling variants is to represent the different variants in the model but the code will one handle one variant. When talking about handwritten code or a code only view, the same implications are valid then for the \u201eduring model construction time\u201c approach. This changes if we consider a model-based development and testing approach.<\/p><p style=\"font-weight: 400;\"><strong>Code Coverage<\/strong><\/p><p style=\"font-weight: 400;\">The code represents one variant and therefore must be fully tested but you will not be able to derive full coverage on model level with the created test data, because the model represents additional variants.<\/p><p style=\"font-weight: 400;\"><strong>Test Interface<\/strong><\/p><p style=\"font-weight: 400;\">Depending on the variants, the test interface on model level might have more inputs than on code level for a specific variant.<\/p><p style=\"font-weight: 400;\"><strong>Test Execution<\/strong><\/p><p style=\"font-weight: 400;\">The reuse of test cases might become tricky since you either have missing interface elements or you must get rid of some or both.<\/p><p style=\"font-weight: 400;\"><strong>Reporting<\/strong><\/p><p style=\"font-weight: 400;\">This use case will also create one report for each variant, because the code is handled separately and only reflects the handling of one variant but does not fully represent the model part, of course.<\/p><p style=\"font-weight: 400;\"><strong>ISO 26262<\/strong><\/p><p style=\"font-weight: 400;\">It would certainly be going too far to say that this approach is not ISO compliant but for a model-based development and testing workflow this approach is not recommended since model and code differ which makes especially the reporting complex and more difficult to follow.<\/p><p style=\"font-weight: 400;\">When looking at handwritten code you should keep in mind that you have the same constraints like the \u201eduring model construction time\u201c approach.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-497e238 elementor-widget elementor-widget-heading\" data-id=\"497e238\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-medium\">3. During compilation <\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5e03895 elementor-widget elementor-widget-text-editor\" data-id=\"5e03895\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div><p><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-size: var( --e-global-typography-text-font-size ); font-style: var( --e-global-typography-text-font-style ); font-weight: var( --e-global-typography-text-font-weight ); letter-spacing: var( --e-global-typography-text-letter-spacing ); text-transform: var( --e-global-typography-text-text-transform ); background-color: var( --e-global-color-42a07f6 );\" data-contrast=\"auto\">To avoid getting separate models and code for each variant, you may use pre-processor #defines to distinguish between the different variants. Since you have only one version of the model and code, it reduces the effort to maintain different variants and things that will be used for all variants only need to be developed once. This also may bring up the conclusion that you can also test all variants within one test project. While this might apply for static test methods, things become tricky for dynamic testing.<\/span><\/p><\/div><p><b><span data-contrast=\"auto\">Code Coverage<\/span><\/b><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">Most important, you will not be able to measure code coverage with high confidence. If supported at all, the code coverage measurement is a patchwork quilt of the different variants and does not reflect the parts of the code from the different variants separately. This might even lead to ambiguous results. And which code coverage should be shown? The one of a single variant, a subset of variants, or all of them? And most importantly: The fact that some code is covered in one variant doesn&#8217;t mean it can be covered in other variants. This means, showing the absence of unreachable code in each variant is only possible by creating separate test projects.\u00a0<\/span><\/p><p><b><span data-contrast=\"auto\">Test Interface<\/span><\/b><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">Things become even more complex if the interfaces of the variants are different. One variant might have more interface objects than another one. This brings additional challenges to the handling and writing of test data.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><b><span data-contrast=\"auto\">Test Execution<\/span><\/b><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">For the simulation, each variant must be compiled before-hand. In the worst case scenario, this happens before each test execution. The test cases must be adapted to the corresponding interface and if running them on both model and code, the test case must in addition carry the data to turn on the correct variant on the model level.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><b><span data-contrast=\"auto\">Reporting<\/span><\/b><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">Like the Code Coverage, a report covering all variants is not meaningful regarding the simulated software, which you finally want to test. This means you will generate all reports (Code Coverage, Test Execution, Interface, Back-to-Back Testing Results, etc.) separately for each variant.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span style=\"font-weight: bold;\">Debugging<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">When it comes to debugging your code, what you see is actually not what you get. During debugging, you will look at the C Code but the test execution happens on the compiled object code. And it&#8217;s not always easy, to identify the code elements which belong (or don&#8217;t belong) to a certain variant.\u00a0<\/span><\/p><p><b>Consequences<\/b><\/p><p><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-size: var( --e-global-typography-text-font-size ); font-style: var( --e-global-typography-text-font-style ); font-weight: var( --e-global-typography-text-font-weight ); letter-spacing: var( --e-global-typography-text-letter-spacing ); text-transform: var( --e-global-typography-text-text-transform ); background-color: var( --e-global-color-42a07f6 );\" data-contrast=\"auto\">Even though it might be technically possible to manage all the test data and different interfaces, execution records, reports, etc. within one test project, it will be hard to keep track of all the different artifacts. Also, minor changes to the code affecting a single variant will impact the whole test project. As much as I understand the idea of why you want to put all the variants in one test project, it makes things complex and error-prone.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-size: var( --e-global-typography-text-font-size ); font-style: var( --e-global-typography-text-font-style ); font-weight: var( --e-global-typography-text-font-weight ); letter-spacing: var( --e-global-typography-text-letter-spacing ); text-transform: var( --e-global-typography-text-text-transform ); background-color: var( --e-global-color-42a07f6 );\" data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">If you cannot avoid this approach, our recommendation is to create a separate test project for each variant. Please, also read the article\u00a0<a href=\"https:\/\/www.cqse.eu\/en\/news\/blog\/living-in-the-ifdef-hell\/\" target=\"_blank\" rel=\"noopener\">Living in the #ifdev Hell<\/a> from Dr. Florian Dei\u00dfenb\u00f6ck and the whitepaper <a href=\"https:\/\/www.cs.cmu.edu\/~ckaestne\/pdf\/ecoop15.pdf\" target=\"_blank\" rel=\"noopener\">The Love-Hate Relationship with the C Preprocessor: An Interview Study<\/a>.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-75ce13e elementor-widget elementor-widget-heading\" data-id=\"75ce13e\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-medium\">4. At runtime<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3d411c4 elementor-widget elementor-widget-text-editor\" data-id=\"3d411c4\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">In this case you will have one model\/code version and only one compiled version but keep the flexibility of binding at compiling time. There are still all variants within the test project, however, the different variants are handled by the calibration data.<\/p><p style=\"font-weight: 400;\"><strong>Code Coverage<\/strong><\/p><p style=\"font-weight: 400;\">The code represents all variants available. You will derive full code coverage and full model coverage for all of them and the test data applies to all variants. This also considers the model coverage in addition.<\/p><p style=\"font-weight: 400;\"><strong>Test Interface<\/strong><\/p><p style=\"font-weight: 400;\">The test interface is stable and variants are only handled by calibration parameters that can be different in each test case. The interface matches between both, model and code.<\/p><p style=\"font-weight: 400;\"><strong>Test Execution<\/strong><\/p><p style=\"font-weight: 400;\">One execution of all test cases will cover all variants implemented and can be executed on both levels (model and code).<\/p><p style=\"font-weight: 400;\"><strong>Reporting<\/strong><\/p><p style=\"font-weight: 400;\">You will only get one report for all variants.<\/p><p style=\"font-weight: 400;\"><strong>ISO 26262<\/strong><\/p><p style=\"font-weight: 400;\">This approach is fully ISO 26262-compliant AND considers all variants in one report.<\/p><p style=\"font-weight: 400;\">This approach is useful because it also considers a valid and good handling of the project from a testing and ISO 2626 perspective. One could argue that this approach might increase memory consumption and execution time on the controller. But looking at today\u2019s controllers, the memory and computation capacities are much larger than ever before. And if it is really about each bit, byte, and macrosecond, I recommend anyway to go with \u201cbinding at model or code level\u201d approach.\u00a0<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5471b75 elementor-widget elementor-widget-heading\" data-id=\"5471b75\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Conclusion<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-cdbefc8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"cdbefc8\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-6bfd5f2\" data-id=\"6bfd5f2\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7920b18 elementor-widget elementor-widget-text-editor\" data-id=\"7920b18\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\u201cBinding at model construction time\u201d will lead to a project setup, in which each variant is independent on model-level, code-level, and then also for the test project.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-55d18ee\" data-id=\"55d18ee\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7015ab7 elementor-widget elementor-widget-image\" data-id=\"7015ab7\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"308\" src=\"https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/08\/Variants_Table.png\" class=\"attachment-large size-large wp-image-14849\" alt=\"What you should now about testing variants - Variants_Table\" srcset=\"https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/08\/Variants_Table.png 2770w, https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/08\/Variants_Table-768x296.png 768w, https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/08\/Variants_Table-1536x591.png 1536w, https:\/\/www.btc-embedded.com\/wp-content\/uploads\/2023\/08\/Variants_Table-2048x788.png 2048w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<div class=\"elementor-element elementor-element-414f299 elementor-widget elementor-widget-text-editor\" data-id=\"414f299\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\u201cBinding at model construction time\u201d will lead to a project setup, in which each variant is independent on model-level, code-level, and then also for the test project.<\/p><p>\u00a0\u201cBinding at code generation time\u201d allows you to test all variants together on the model level, followed by an independent Back-to-Back test per variant. If we want to test model and code together, we&#8217;d setup independant test projects for each variant.\u00a0<\/p><p>When talking about variants from a testing point of view, \u201cBinding at compilation time\u201d brings up the most constraints and makes the management of a test project very complex. In the end, it usually leads to a workflow that is similar to the one mentioned above for \u201cBinding at code generation time\u201d.<\/p><p>From a test perspective, \u201cBinding at runtime\u201d seems to be the most convenient method for testing software variants. Compared to other approaches, it enables you to easily manage all variants in one test project. With a single model\/code version and compiled code, variants can be managed through calibration data within a single test project. This approach reduces complexity, ensures a stable interface, and enables comprehensive reporting.\u00a0<\/p><p>If you find this blog article useful, you might also be interested in my blog article about <a href=\"https:\/\/www.btc-embedded.com\/why-back-to-back-testing-is-still-needed-for-floating-point\/\">Why Back-to-Back Testing is still needed for floating-point<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9f0c85d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9f0c85d\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b8e7065\" data-id=\"b8e7065\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Creating different variants of a software component is an efficient way of reusing existing and developed features for different versions of a specific component or controller.\u00a0Thus, testing variants is an\u00a0ubiquitous task in the automotive industry that needs to be taken into account during testing. How are variants created in a Model-based Software development process? When [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":14541,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"elementor_theme","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[49],"product":[],"use_cases":[],"class_list":["post-14532","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-model-based-development"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/posts\/14532","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/comments?post=14532"}],"version-history":[{"count":0,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/posts\/14532\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/media\/14541"}],"wp:attachment":[{"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/media?parent=14532"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/categories?post=14532"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/tags?post=14532"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/product?post=14532"},{"taxonomy":"use_cases","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/de\/wp-json\/wp\/v2\/use_cases?post=14532"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}