Many different aspects must be considered when designing a medical device. This is particularly true with more complex devices that rely on internal computers to support their function. Every device must be safe, effective, affordable, easy to use, etc. But one aspect that can be overlooked in favor of the shinier factors is testability.
Design for Testability refers to keeping testability as a major design concern throughout the whole design process, ensuring that all parts of the product can be both manipulated and monitored allowing for thorough testing.
“Shiny” refers to the aspects of the device which are fun to work on, show off, or that grab attention. Engineers often like to get deep into solving the complex technical problems, for them this is shiny. Designers like to make the device look smooth, and attractive so that clients, investors, and users say “wow, this looks amazing! When can I have mine?” While these are very important, a device must also be tested and validated to the satisfaction of regulators before it can be sent to market, this is typically not seen in the same “shiny” context.
Putting time and energy into the less than shiny Design for Testability can yield strong benefits in later stages of a project, particularly when dealing with software. The benefits range from improving the effectiveness of the engineering team, to making additional testing possible, to saving time, hassle, and cost during Validation tasks. This does take additional time and effort, and in some of the more automated cases can raise security concerns.
Pros of Design for Testability
First, a device designed for easy testing makes the job of the engineering team simpler for the entire duration of the project. Starting in the prototyping stage, the engineers will be able to get direct feedback on device performance faster and more easily. This may be as simple as having mounts to attach protractors to monitor the angle of a joint including integrated accelerometers or thermocouples that can log to an external computer or including software Unit Tests. Later in the project this kind of data can be invaluable in determining the source and solution for issues that come up within complex systems. Tools like Unit Testing can inform software engineers if an alteration to the code has unintended side effects that will prevent proper function of the software.
Second, designing the device to be testable reduces the time it takes for the engineering team to get data back from their design will allow them to iterate more rapidly. This will also assist verification efforts by allowing the QC teams to work directly with built-in tools to more easily gain information required for their testing. Over the course of a project this can save valuable time and reduce frustration to test and validate the device.
Third, if testing interfaces are setup for autonomous function or external computer control, a more diverse set of testing becomes significantly more viable. Fatigue testing becomes simpler to perform, as the device can be directed to continuously operate for hours on end without stopping, or to run with a dynamic duty cycle. Additionally, batch testing becomes an option with a large number of different configurations run sequentially and all of the relevant data logged for the test set. This kind of testing can allow the device software to be repeatedly sent into edge cases enabling the full scope of the software to be tested. Without the ability to automate, testing could take weeks of time for the development or QC teams, if it is performed at all. As an alternative to batch testing, variable optimization could be run. With variable optimization the system checks the performance under each of several configurations, then adjusts the variables based on the findings, continuing to locate the best variables for device under any number of different use cases.
In addition to directly helping the development team, Design for Testability is also very helpful for the QC team Validation and Verification. Providing the QC team software command and logging tools enables them to more quickly test the device and analyze the data to confirm whether the device passes validation.
Cons of Design for Testability
Design for Testability can get neglected because it is not as “shiny” as other tasks. There are more tangible reasons why a team may choose not to use Design for Testability.
The first is the cost associated with Design for Testability, it adds some additional up-front design time to plan for testability. It is also needs a continuous amount of overhead to maintain and expand the testability tools and interfaces over the course of the project. Despite the long-term benefits, the up-front cost is one that many projects do not want or cannot afford.
Second, during the early phases of a project it can be relatively easy for the engineering team to quickly put together some basic tools that will allow them to have very basic feedback on the performance of their systems. Although these systems will likely not be as accurate as a more thoroughly designed system, they will likely take less time to make and cost less. For some projects this is preferable during the early phases, especially if the work is intended entirely as a proof of concept and is not intended to form the backbone of the future device.
Third, every tool that is added to the system can be misused, every interface can be broken, and every digital connection creates a possible avenue for a cybersecurity attack. While automating the interactions can save significant savings with advantages, it comes with additional work to secure the interfaces from cyberattack. There are many solutions to this problem, but they all require time and effort that add cost.
Conclusion
There are a significant number of benefits available to engineering teams by following Design for Testability. Large projects can improve the project quality and decrease the overall cost. On the other hand, small projects may not find significant benefits or savings. Medium projects are most likely to be in a challenging spot as the pros and the cons are more closely balanced. The real bugbear that interferes with Design for Testability is scope creep. When additional scope is added often the project leads will add the features but forget to add the scope to make them testable or for testing them.
Because every project is different, I cannot provide a hard and fast rule for when a project should follow Design for Testability. I can, however, say that the needs of the project should be used to guide the team on deciding whether Design for Testability should be used.
The plan may not be Design for Testability, but the planning should always consider Design for Testability.
James Sease is a QA Engineer at StarFish Medical with a background in both Mechanical and Software Engineering. He has worked primarily medical and aerospace engineering across several different roles. James enjoys working on automation of devices, tools, and processes.
