Friday, June 30, 2017

Data is the key to assessing medical device efficacy, but it is extremely poorly studied

MedCity News pointed me to a very interesting retrospective study on Gastric Lap Bands, and a Danish reporting system that led to findings on a hernia mesh in an article today. This got me thinking about medical device post-market surveillance and efficacy analysis in general. Based on data from two different countries, on two completely different products, it is easy to understand, how data is key to assessing medical devices once they are in the market. However, problems remain. As Andrew Ibrahim et. al. themselves showed, there are problems, and then there are solutions.

What would be the advantages of the assessments?

Let's get the obvious out of the way first. Good Quality Assessments can lead to a lot of benefits:

1. Patients will be the winners. If it is discovered earlier, as it could have been in the case of the Gastric Band, that a device is not working, then it could change the standard of care.

2. Both the manufacturer and the competitors can improve their devices. Or, abandon going down the wrong path. Either way, patients benefit again.

3. Medicare and Medicaid, and thus the US Government and consequently patients and all tax payers would benefit from wasting money unnecessarily. The JAMA Study indicated that of the $470mn spent on gastric bands, 47.6%, or $22.4mn was spent on re-operations and removals of the bands. That is quite a wastage.

Reducing waste, increasing standard of care, improving or withdrawing devices, these are all great great motivations to make sure that medical device efficacy assessments be done thoroughly and routinely. However, this would be the time to take the discussion back to the problems facing this effort.

Problems with Medical Device Post-Market Data Collection

1. Reporting is Voluntary

Obviously, it would be counterproductive to force data collection on over-burdened medical facilities and practitioners. However, are there incentives that can make reporting more attractive to practitioners? Like a rewards program with all carrots and no sticks. What if facilities doing more and complete reporting got awards, and priority when it came to reimbursements and such?

2. Incomplete reporting plagues the system

As with the above problem, the FDA does receive reports, but most of them incomplete. Should the device manufacturer be burdened with opening up a CAPA and sending agents to the reporting entity to complete the reports? I know some companies train their employees to think of a CAPA as the black plague, but if you have a good system and train your personnel properly, it is a great tool. Perhaps, organizations should take this on themselves, without being forced to do so.

3. The FDA doesn't know about Device Usage

Ibrahim et. al. report this, but I think the more curious question, does CMS know? Do private insurers know? It is a bit concerning that post-market monitoring doesn't call for regular reporting on data usage statistics! And yes, because no one knows how much of a device is used, and on what cohorts, no one knows. This is truly appalling!

4. Does the FDA know how to analyze the data?

This one is not on the paper or the write up, and is not exactly a problem, but do the FDA and/or the CMS or medical practitioners themselves know how to judge when a device is slipping. When do you worry? Let us say you have all the data and you find that there is a 5% failure or re-operation/removal rate. Then? Or at 10%? Or, like in the gastric band's case, at nearly 50% at one point?

5. What about Patient Reported Outcomes (PROs)?

Patient Reported Outcomes, especially of pain, discomfort and other important Oncological treatment related factors are all the rage in Oncology. This doesn't seem to be the case with any medical devices in any other fields.

The Key Issue

As Data Scientists (formerly Statisticians, but that is no longer sexy) point out, just having the data is not enough, you need to know how to analyze it, and when to set the alarms off! Sadly, it would appear, neither the FDA nor the nearly dozen or so agencies tasked with disseminating healthcare in one form or the other have the required data, or how to analyze it. This is a really alarming problem! The term "information processing" was coined by Watson in the '20s and here we are, learning about how poorly information gathering is done, in 2017, from a JAMA paper. Hyperbole? NO!

These issues must be looked at with a sense of urgency and I am not sure the Government is solely equipped to do this.

The Solution

1. Overburdened or not, perhaps, medical practioners need to take reporting more seriously. Without reporting, frankly, the data itself is lost and patients suffer.

2. Medical Device Manufacturers can benefit from knowing how a device is performing. They should voluntarily engage in superior data collection practices and share that with Doctors and agencies alike. Yes, sometimes this leads to a loss of market, but many times, they can benefit from the good will, iterate designs and generally benefit the public, which is the stated Vision of most, if not all medical device organizations.

3. The UDI is no panacea, and yet, it can lead to greater efficiency in data collection. The FDA should reduce barriers for adoption, reward successes and organize conferences and gather inputs to see why failures occur and how to fix them.

4. Patients and other stakeholders should be recruited in helping monitor device efficacy. This is an important step and we are still not there yet.

5. Going beyond the current device classification, the FDA, and Medical Organizations such as JAMA and others should categorize risk, and try to pinpoint at what rate of failure a device's use should be reassessed based on its need vs performance and other factors.

6. Better analytics and more research, like the one Ibrahim et. al. engaged in, is needed, not just retrospectively, but also, on an ongoing basis.

If you can think of other solutions, do let me know!

Subscribe and Support, Please!

Did you enjoy this post? Please subscribe for more updates, using the sidebar. Have ideas or blog posts you'd like to see here? Contact me at yamanoor at gmail dot com.


1. The MedCity News Article:

2. The JAMA Study Summary:

3. The JAMA "Viewpoint":

No comments: