{"id":14048,"date":"2023-04-26T15:38:56","date_gmt":"2023-04-26T13:38:56","guid":{"rendered":"https:\/\/www.btc-embedded.com\/?p=14048"},"modified":"2025-02-13T15:57:42","modified_gmt":"2025-02-13T13:57:42","slug":"4-reasons-to-avoid-writing-test-cases-in-excel","status":"publish","type":"post","link":"https:\/\/www.btc-embedded.com\/zh-hans\/4-reasons-to-avoid-writing-test-cases-in-excel\/","title":{"rendered":"4 reasons to avoid writing test cases in Excel"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"14048\" class=\"elementor elementor-14048\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3b5f0493 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3b5f0493\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-559a247a\" data-id=\"559a247a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a1ff07f elementor-widget elementor-widget-text-editor\" data-id=\"a1ff07f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">I once heard someone say:<\/p><p style=\"font-weight: 400;\"><strong>\u201cIt\u2019s impossible to develop a software tool that isn\u2019t a competitor to Excel\u201d. <\/strong><\/p><p style=\"font-weight: 400;\">And that\u2019s also true when it comes to the creation of test cases for testing embedded software. Over the past 20 years, Excel has been a very popular format and language to create, manage and store test cases in the embedded software domain. This is the case not only for in-house tools developed within a particular project context, but also for many commercial tools which support or rely on Excel as a test specification language.<\/p><p style=\"font-weight: 400;\">And the advantages are clear. Excel is usually already available on most PCs and the table structure is not a bad way to represent test data consisting of signals (columns) and timesteps (rows).<\/p><p style=\"font-weight: 400;\">However, in this blog article, I want to share 4 aspects about why Excel is not a complete test case language.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2cb1475 elementor-widget elementor-widget-heading\" data-id=\"2cb1475\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">1.\tAnalysis and Update of the Test Architecture <\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-732d3d4 elementor-widget elementor-widget-text-editor\" data-id=\"732d3d4\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">Before I can start to write a test case, I need information about the interface of the system under test. This includes at least information about variable names and roles (e.g., input, output, calibration parameter, local measurement point). Of course, Excel is not able to extract this information, so I either need to manually create a template document with the right interface names or use a script to generate this template from my (for instance) Simulink model or production C code. But even if this seems to work reliably, there are several limitations:<\/p><ul><li>What happens if the interface of the SUT changes? If signals are added or removed or even renamed, it breaks the test case and it\u2019s usually not possible to automatically update all tests.<\/li><li>What happens with large interfaces that include structures, vectors, or arrays? This quickly leads to an unreadable document, as there\u2019s no special handling to efficiently display these complex data types.<\/li><li>What happens with enumerated data types? Usually, they just show up as integer values, instead of having a dropdown menu with the enumeration strings.<\/li><li>What happens when I enter invalid data? This could be, for example, a value outside of the value range. As Excel doesn\u2019t know about datatypes and ranges, it won\u2019t provide a warning and I won\u2019t discover the issue until I run a simulation.<\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffe7789 elementor-widget elementor-widget-heading\" data-id=\"ffe7789\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">2.\tSimulation and Reporting<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-24084d2 elementor-widget elementor-widget-text-editor\" data-id=\"24084d2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">What else is missing in Excel? Exactly: The \u201cplay\u201d button to directly run my test case. Of course, I can go to a different UI to trigger an import of the Excel file and then convert the values into something which can be used in the simulation (e.g. a .mat file).<\/p><p style=\"font-weight: 400;\">But:<\/p><ul><li>It\u2019s still an extra (and maybe error prone) step in the process.<\/li><li>It usually won\u2019t feed the results from the simulation back to Excel, so I again need a different interface to access the simulation results.<\/li><li>And even if I find a way to bring the simulation results back into Excel, I won\u2019t get much support to highlight and find the parts of the test case which made it fail.<\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f4dea72 elementor-widget elementor-widget-heading\" data-id=\"f4dea72\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">3.\tMissing Features<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6cdaf06 elementor-widget elementor-widget-text-editor\" data-id=\"6cdaf06\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">For small projects and small test interfaces, a test case representation in Excel could definitely be sufficient. But for larger projects and larger components I would expect several features from a test authoring language which are usually not available in Excel:<\/p><ol><li>Requirements-traceability: Usually test cases are derived from requirements, so I need a test authoring environment which allows me to import and view requirements in the same interface in which I\u2019m writing my test case. Usually the requirements are not visible within Excel test cases<\/li><li>Plotting: For longer test cases, it is of course very useful if I can plot signals to visualize patterns<\/li><li>Signal generators<\/li><li>Automatic management of unused signals\/steps<\/li><li>Test macros<\/li><\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffdae69 elementor-widget elementor-widget-heading\" data-id=\"ffdae69\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">4.\tISO 26262 Compliance<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7c64fc2 elementor-widget elementor-widget-text-editor\" data-id=\"7c64fc2\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">Last but not least, a lot of automotive embedded software projects need to consider the ISO 26262 standard, and even for lower ASIL ratings it\u2019s usually necessary to perform a tool qualification for the test environment. (Please find out more about ISO 26262 tool qualification in this <a href=\"https:\/\/www.btc-embedded.com\/when-and-how-to-qualify-tools-according-to-iso-26262\/\"><u>article<\/u><\/a>). As Excel doesn\u2019t come with an ISO 26262 certificate, it means the confidence in the tool needs to be produced by a tool qualification process performed on the user side, which is usually very time consuming.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5471b75 elementor-widget elementor-widget-heading\" data-id=\"5471b75\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Conclusion<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2a01264 elementor-widget elementor-widget-text-editor\" data-id=\"2a01264\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"font-weight: 400;\">Excel is a powerful and versatile tool with a flexible data model and because it\u2019s available on most PCs around the globe it appears in many different use cases. But when it comes to testing safety critical embedded software (especially within a larger team) it might make sense to use a dedicated test authoring environment to benefit from advantages including:<\/p><ol><li>Integration with the development environment and system-under-test<\/li><li>Ability to run simulations directly from the test case editor<\/li><li>Advanced features for handling complex data types and data structures<\/li><li>ISO 26262 certification<\/li><\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>I once heard someone say: \u201cIt\u2019s impossible to develop a [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":13445,"comment_status":"open","ping_status":"closed","sticky":false,"template":"elementor_theme","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[53,49,82],"product":[],"use_cases":[],"class_list":["post-14048","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-iso-26262","tag-model-based-development","tag-requirements-based-testing"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/posts\/14048","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/comments?post=14048"}],"version-history":[{"count":3,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/posts\/14048\/revisions"}],"predecessor-version":[{"id":19730,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/posts\/14048\/revisions\/19730"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/media\/13445"}],"wp:attachment":[{"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/media?parent=14048"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/categories?post=14048"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/tags?post=14048"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/product?post=14048"},{"taxonomy":"use_cases","embeddable":true,"href":"https:\/\/www.btc-embedded.com\/zh-hans\/wp-json\/wp\/v2\/use_cases?post=14048"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}