Let’s do an experiment. I want you to learn how to develop software. I won’t teach you, though. I’ll only give you access to a desk, a chair and a laptop which is connected to the internet. And then I’ll give you three months time.

Will you succeed?

Given that you’re motivated, I think you will. You certainly have access to all resources. You’d probably start by heading to Google and typing “How to learn software development” and then you’ll take it from there.

The internet has empowered us to access all human knowledge in less than a few seconds.

Wait — that’s incorrect. Here’s the corrected sentence:

The internet has empowered us to access all human knowledge, except how to certify a medical device, in less than a few seconds.

Here’s the thing: With the internet, if you’re a motivated person in front of a computer and want to “learn X”, then you’re going to succeed as long as there are enough good resources to learn “X” on the internet. And by now, the internet offers superb online courses on software development, data science, design, business, maths, physics, chemistry, most languages — anything, really.

But not for medical device regulation and certifying a medical device.

I challenge you to find out how to certify a medical device by looking it up on the internet. We tried this in 2017 and failed. And we are confident that everyone else fails, too. For startups, this is especially frustrating as they have great products and the only missing piece of their go-to-market strategy is the knowledge on how to certify their medical device.

Regulation in itself is not a bad thing; its main goal is ensuring patient safety. Without question, that’s important. Unfortunately, often times regulatory requirements make a company’s productivity grind to a halt. Then, it’s easy to blame the regulatory authorities for being incompetent bureaucrats.

But most companies don’t notice that most regulatory wounds are self-inflicted, mostly due to incompetent consultants, misinterpreting standards and introducing processes which are unnecessarily complex.

And, in our experience, regulatory authorities are, perhaps unexpectedly, pragmatic and constructive when working towards the goal of becoming compliant and ensuring patient safety.

In our series “Certifying a Medical Device as a Startup” we’d like to share our experience getting our software certified as a medical device. Why would we do this in a field which is notorious for its intransparency? Because we believe that healthcare needs better software and we’ll get there by enabling more startups to enter the market without hitting regulatory roadblocks.

In this first part, we’d like to talk about the biggest problems we faced while getting started.

Lack of Transparency

Trustworthy, simple and well-written information on how to certify a medical device is basically non-existent. You’ll get more actionable results when searching “how to build a rocket which flies to Mars” instead.

As a software developer, this came as a surprise to me. Developers are very open in sharing information: They write friendly tutorials and share code which you can download and run on your computer. It can hardly be overstated how tremendously software has improved due to this openness. As the quality of software improved, it started to enter the industry. Nowadays, open-source software plays a large role in pretty much any company. If it hadn’t been for open-source software, many things which we’ve come to appreciate like smartphones, operating systems and the internet wouldn’t exist in their current form.

Sharing knowledge is not a zero-sum game. By sharing knowledge, we enable others to build great things. And right now, healthcare desperately needs some great things — like better software.

Rails has been downloaded some 170 million times from RubyGems since it started being hosted there. By one account I read, more than a million applications built with Rails have been put online. Neither the first nor the second fact has harmed me in any meaningful way.
(DHH, Open Source Beyond the Market)

However, openness is not found in the space of medical devices. The incentives are easily explained:

• Companies which fail in getting regulatory approval disappear.
• Companies which succeed in getting regulatory approval hoard their knowledge so that their competitors fail and disappear.
• Consultants who help companies getting regulatory approval don’t share their knowledge as they’d otherwise be out of their job.

The result is that information is heavily guarded and must be purchased indirectly by hiring consultants to teach you. On the bright side, there are some great initiatives, most notable the free blog posts of the Johner Institute in Germany (they also do consulting).

Ambiguity

I recently talked to a developer at a large healthcare software company. I asked how they handled code changes. He answered:

For every code change, we have to print out a form, sign it and put it in a folder.
(Anonymous Developer, Healthcare Software Company)

You’ve got to be kidding me. And where do you get your electricity from? By employing people to ride bicycles connected to generators? Just kidding.

But how do you expect to develop great software if everyone is busy filing paperwork? And more importantly, how do you attract great developers who love building things when you constantly hinder them from doing their work?

Let’s figure out why the healthcare software company from above came up with this clunky process.

When developing a medical device, companies have to comply with certain standards. These standards are very ambiguous. For example, the ISO 62304:2006, section 5.5.5 states:

The manufacturer must verify the software units and document the results.

So this tells us that we have to verify software units and document the results somewhere.

My best guess is that they came up with this statement by assembling an “expert panel” of 100 healthcare companies and letting them come up with their lowest common denominator. Everyone agrees that software has to be verified to some extent. This is good and bad news.

The good news is that the standard tells you what to do and you’re free to choose how to do it. The bad news is that if you don’t understand what the sentence above means, you won’t benefit from the liberty of how to do it.

To further complicate the matter, depending on how you choose to do it, you may be doing it wrong! In the context of the above statement which describes software verification, lots of follow-up questions pop up: Do you write unit tests for software verification? With which test coverage? Using which libraries? What about integration tests?

This is typically where consultants come in, leading us to our next problem.

Incompetent Consultants: Three Things to Look For

The goal of all regulations is ensuring patient safety by achieving high product quality. Surely, there’s regulation ensuring equally high quality of regulatory consultants, right?

Unfortunately, starting a regulatory consultancy is easier than setting up a kebab shop. Anyone can become a regulatory consultant. This results in an ironic situation: An industry which focuses on high quality doesn’t have any checks in place for those people who consult companies on getting there.

And consultants often have a crucial position in companies: Similar to a surgeon conducting a vital operation on a patient, they restructure and create processes at the heart of the company, changing the way things are done forever.

And just like an unqualified surgeon forgetting his scalpel in your body, many consultants implement processes which are outdated, complex and overhead-ridden. And just like a patient, startups are unable to assess the competence of their consultants.

After having worked with multiple consultants, we asked ourselves: What’s different about the good consultants and how can we identify them in the future? For that, we came up with a list of things and questions which you should ask your soon-to-be consultants.

‍

1. Software Expertise
   Medical devices span a broad range — did you know that wheelchairs, contact lenses and x-ray films are also medical devices? Accordingly, regulatory consultants only have experience in a subset of those fields. If you’re developing software like we are, you want a consultant with software expertise. Don’t settle for consultants who generally develop hardware and purport that “we can do software, too”. Imagine losing hours of billable hours explaining what GitHub is to them.
   Question: Are you specialized in software as a medical device? Have you written code yourself? Do you know about version control, git and GitHub?
   Best case: Your consultant has worked as a software developer in the past.
   Acceptable case: Your consultant has some IT experience (college, prior job).
   Worst case: “We do everything, no problem”
2. Successful certification of software of same risk class
   Even if we limit our search to consultants with software experience, this still includes a huge variety of things. From simple apps to desktop applications for viewing x-ray images up to embedded software controlling cardiac pacemakers, the associated risks of your software affect how you structure your software development and what to document. At the minimum, your consultant has successfully certified software of the same risk and software class in the past.
   Question: Have you certified a similar medical device of the same class?
   Best case: Your consultant has successfully certified software of the same classification which is similar to yours.
   Acceptable case: Your consultant has successfully certified software of the same classification which is different from yours.
   Worst case: Your consultant hasn’t certified software of the same classification yet.
3. Regulatory compliance software
   You think the market of regulatory consultants is intransparent? Wait until you start searching for regulatory compliance software. That’s the kind of software which promises to help you structure your documents in a way to stay compliant. Interestingly, opinions towards this software varied wildly among consultants. Of course, don’t forget that no software on this planet will save you from having to understand the regulations first.
   As developers, we like to say there is no best tool but only the right tool for the job. Accordingly, the only correct answer to this question is “whatever works for you”. That could even be Google Sheets or Excel!
   Question: Can you recommend software for managing our regulatory compliance?
   Best answer: Whatever works for you. As a small company, you probably don’t need any. We have some experience with X, Y and Z which I can share.
   Worst answer: You’re not allowed to use any software, do everything on paper!
   Worst answer #2: Yes, I have this friend who sells software X, it’s the best.
   ‍

In summary, regulatory consultants won’t save you from developing a deep understanding of the regulations yourself. The same goes for regulatory software which can help you a lot but only once you’ve understood which features you need. None of them will make work magically go away.

On the plus side, as someone in the German authorities once told me, consultants do have their place. If you find good consultants who enable you to get certified in 6 months instead of 2 years that can be game-changer. Doing things yourself will always mean having to learn them, making more mistakes and being slower.

It all boils down to a trade-off between time and money. In any case, your end goal should be to achieve self-sufficiency by building your in-house regulatory knowledge and limiting expensive consultant hours. So choose wisely!

Crappy software

Becoming regulatory compliant is no easy task. The main requirements are maintaining a Quality Management System, Application Lifecycle Management and Risk Management (ISO 13485, IEC 62304, ISO 14971 and IEC 62366). Adhering to all those standards is complicated — can software solve the problem for us?

It’s a very enticing idea to have software which guides you in becoming regulatory compliant. However, there are two problems:

Repeating my point from above because it can’t be emphasized enough: Using software for regulatory compliance won’t spare you from understanding the regulatory requirements in the first place. It merely helps you keep things organised.

Secondly, such software doesn’t exist. Really? Let’s look at what it needs to do:

• Features: Quality Management System (QMS), Application Lifecycle Management (ALM) and Risk Management
• Good usability
• Fair pricing
• Low maintenance: Cloud-based, no local installation

Currently, no such software exists. There’s software which covers some of the points above:

• Cloud-hosted JIRA with Confluence could cover ALM and QMS but would be missing Risk Management. Also, JIRA is slow and many developers dislike JIRA.
• Polarion covers QMS, ALM and Risk Management but has bad usability and is high-maintenance. The installation manual alone covers 64 pages!
• Other tools which cover only one of the aspects above, e.g. ALM tools.

As in software development, we recommend only getting new tools once you feel the need for them. If your company is small and your product simple, you may be perfectly fine using Excel or Google Sheets.

Later on, once you notice your needs becoming more complex and they can’t be handled by existing tools, why not write your own? You’re a software company after all.

‍

Wrapping Up

As a startup, realising that your product is a medical device may seem like a show-stopper but doesn’t have to be. By sharing our experiences, we hope that startups follow along and bring great healthcare products to market. Let us know if this helped you and follow us for more posts!

‍