Pages: [1]   Go Down
Author Topic: Urgent need for the critical reappraisal of clinical effectiveness  (Read 2952 times)
0 Members and 1 Guest are viewing this topic.
Hero Member

Karma: +24/-0
Offline Offline

Posts: 59593

« on: August 05, 2014, 12:17:30 AM »


Letter to the Editor

Vagus nerve stimulation: Urgent need for the critical reappraisal of clinical effectiveness

Vagus nerve stimulation (VNS) received approval for medical use by the European authorities in 1994 and by the US Food and Drug Administration in 1997. Today, about fifteen years later, VNS may fairly be regarded as the third pillar of epilepsy treatment, besides antiepileptic drugs (AEDs) and epilepsy surgery.1 More than 60,000 stimulators had been implanted for the treatment of epilepsy by November 2011 (Cyberonics Inc., personal communication) and an increasing number of patients are now getting reimplanted after battery-end-of-service of the pulse generator. In Germany, VNS treatment is completely reimbursed by the public health insurance, including the cost for devices, implantation surgery, all necessary follow-up visits, and re-implantations. However, the local budgets at epilepsy centres and thus the number of affordable devices per year is limited and reimplantations have to be scheduled within this budget.

But is there compelling evidence for the clinical effectiveness of VNS?

In this edition of Seizure Garcı´a-Navarrete et al. report 18- months follow-up data from a sample of patients treated with VNS and stable AEDs.2 The authors report that 27/43 patients (63%) experienced a greater than 50% reduction of their monthly seizure frequency (seizure responders). This finding is completely in line with the great number of reports on VNS from the last 15 years1 including a similar study on patients receiving unchanged medication by Labar.3 Based on this evidence, Garcı´a-Navarrete et al. conclude that VNS is an ‘‘effective therapy in the long-term control of medication-resistant seizures’’. I have some serious concerns about this conclusion, and this editorial comment is motivated by the observation that this new paper is rather symptomatic of the entire VNS literature.

First, the outcome was not entirely positive. The device had to be explanted or switched off in 5/43 implanted patients (12%) for different reasons (e.g. infection); the treatment was ineffective in 11/43 patients (26%, non-responders); no patient experienced complete and sustained release from seizures; and side-effects were reported by 22/43 patients (51%), including such severe adverse effects as dysphagia in one instance. Again, this dataset resembles the findings of former studies.1 Taken together, VNS realistically offers odds of roughly 50:50 for the individual patient to experience a moderate (but most probably not substantial) improvement of seizure status. This means that every other VNS patient cannot be expected to experience any therapeutic benefit. Nevertheless, the patient has to bear the complete burden of surgical risks, often irreversible diagnostic limitations (such as reduced feasibility of high-field magnetic resonance imaging), and stimulation-related adverse effects. From a medico-ethical perspective, an elective invasive treatment with a partly irreversible burden for the patient and such a high probability of treatment failure hardly appears recommendable, or even justifiable.

Second, Garcı´a-Navarrete et al. were not in the position to draw any conclusion about the efficacy of VNS from their data. For logical reasons, single-arm studies do generally not support causal attribution of changes occurring over time to a given medical intervention, especially not to one particular part of a more complex treatment package. Obviously, things are always changing and more likely to improve if they were particularly unsatisfactory when the treatment was applied (regression toward the mean). In contrast to many other studies, Garcı´a-Navarrete et al.2 managed to exclude medication changes as a confounding factor, but this does not satisfy the logical requirement of a control group for the evaluation of the effectiveness of an intervention.

Unfortunately, adequately controlled studies on VNS are lacking. This does not stop the authors of countless single-arm studies from drawing positive conclusions about the efficacy of VNS from their data which is, of course, a logical fallacy. For example, Helmers et al.7 recently performed an uncontrolled health-economic study and concluded that ‘‘VNS is associated with decreased resource utilization and epilepsy-related clinical events and net cost savings after 1.5 years’’. However, no such associations can be established in the absence of a control group.

Third, in the VNS literature efficacy is often mixed-up with clinical effectiveness. Many authors, like Garcı´a-Navarrete et al., actually want to recommend VNS for clinical use when they argue for the ‘‘efficacy’’ of VNS. However, efficacy of VNS has been shown in two pre-marketing studies.4,5 In these double-blind randomized controlled trials patients receiving ‘‘true’’ VNS were compared to patients receiving sham VNS (i.e. low on/off time ratio condition) to identify the specific effect of electrically stimulating the vagus in contrast to the (placebo) effect of only implanting a medical device. However, while proving efficacy is certainly an essential prerequisite for clinical effectiveness it is by no means sufficient. As VNS therapy is an add-on to best drug treatment (BDT), the clinical effectiveness of VNS must obviously be shown in terms of therapeutic benefits which can be achieved by adding VNS to BDT as compared to not adding VNS. To the best of my knowledge, only Sherman et al. published a small but controlled study on the seizure outcome in pediatric patients under BDT plus VNS as compared to BDT only.6 These authors found no specific benefits of adding VNS to BDT.

What makes matters worse is that the majority of systematic reviews on VNS in the treatment of epilepsies have drawn similarly positive conclusions about the ‘‘efficacy’’ (read: clinical effective- ness) instead of unambiguously stating the rather embarrassing lack of adequately controlled studies. For example, Connor et al.1 recently concluded that ‘‘vagal nerve stimulation is a safe and effective alternative for adult and pediatric populations with epilepsy refractory to medical and other surgical management’’. Again, there is no evidence supporting this conclusion and the Sherman et al.6study was not even mentioned. Furthermore, it is arguable whether an implanted medical device which has to be switched off or explanted in more than 10% of the patients should be classified as safe.

In the absence of adequately controlled studies, I suggest using the data from Callaghan et al.8 as a kind of interim control measure for the effects of VNS. This suggestion could also be applied to other innovative non-pharmacological add-on treatments which are currently explored without adequate control conditions (e.g. deep brain stimulation, DBS). Callaghan et al. followed a large sample of therapy refractory non-surgical patients (i.e. eligible candidates for VNS/DBS) over several years and reported surprisingly good outcomes of BDT, i.e. flexible lege artis drug treatment. For example, the annual rate of patients who became seizure-free was about 5%. Thus, even in severe epilepsy, substantial improvement due to spontaneous remission, new drugs or more effective drug combinations remains possible and the efficacy of BDT should not be underestimated.

Unfortunately, the mean therapeutic changes occurring during VNS treatment (i.e. 3–5% of patients achieving seizure cessation, 40–50% experiencing a greater than 50% improvement in seizure frequency1) remain slightly below the effects of BDT reported by Callaghan et al.8 However, Garcı´a-Navarrete et al.2 and the authors of so many single-arm studies on VNS do not consider the possibility that their improved VNS patients might have been even better off without implantation. Most pertinently, we do not know whether keeping drug regimens stable over months and years for an unbiased evaluation of VNS prevents VNS patients from utilizing the full potential of state-of-the-art BDT.

To be honest, given the current outcome data on BDT plus VNS and BDT only, I am not particularly hopefully that future studies will reliably show the kind of sustained additional therapeutic effects of adding VNS to BDT which could more than compensate for the costs, risks, diagnostic limitations, and side-effects associated with VNS. Fifteen years after approval, we are facing the possibility that adding VNS to BDT is not actually associated with any clinically relevant therapeutic benefit for the patients (despite confirmed efficacy). As the necessary studies on therapeutic superiority are still missing and the clinical effectiveness of add-on VNS over BDT has not been shown so far, the recommendation of VNS to patients is currently not supported by appropriate evidence.

I know that many epileptologists can report unexpected therapeutic improvements in individual patients after VNS implantation. However, this is no scientific way of establishing clinical effectiveness. Otherwise, low-cost therapies like homeopathy and acupuncture must also be approved and reimbursed based on anecdotal reports of intriguing ‘‘healings’’. Patients certainly deserve evidence-based treatments, especially the most severely affected among them. As non-efficacy can never be empirically proven for logical reasons, positive evidence from adequate studies is essential for the acceptance of therapeutic efficacy and clinical effectiveness.

The end of the VNS story has not yet been written but it already seems to provide good reasons to re-consider the established procedures of scientific (i.e. independent and critical) therapy evaluation and approval in our field. I generally suggest that single-arm studies of a newly approved innovative treatment should only be accepted for publication shortly after approval. Such studies may confirm and document the clinical feasibility of the new treatment.  However, one or two years after approval, only adequately controlled studies should be accepted for review and publication to avoid logical fallacies, causal misattribution and the suggestive mix-up of therapeutic efficacy and clinical effectiveness.


1. Connor Jr DE, Nixon M, Nanda A, Guthikonda B. Vagal nerve stimulation for the
treatment of medically refractory epilepsy: a review of the current literature.
Neurosurgical Focus 2012;32(3):E12.
2. Garcı´a-Navarrete E, Torres CV, Gallego I, Navas M, Pastor J, Sola RG. Long-term
results of vagal nerve stimulation for adults with medication-resistant epilepsy,
on unchanged antiepileptic medication. Seizure 2012.
3. Labar D. Vagus nerve stimulation for 1 year in 269 patients on unchanged
antiepileptic drugs. Seizure 2004;13(6):392–8.
4. George R, Sonnen A, Upton A, Salinsky M, Ristanovic R, Bergen D, et al. A
randomized controlled trial of chronic vagus nerve-stimulation for treatment
of medically intractable seizures. Neurology 1995;45:224–30.
5. Handforth A, DeGiorgio CM, Schachter SC, Uthman BM, Naritoku DK, Tecoma ES,
et al. Vagus nerve stimulation therapy for partial-onset seizures – a randomized
active-control trial. Neurology 1998;51:48–55.
6. Sherman EMS, Connolly MB, Slick DJ, Eyrl KL, Steinbok P, Farrell K. Quality of life
and seizure outcome after vagus nerve stimulation in children with intractable
epilepsy. Journal of Child Neurology 2008;23:991–8.
7. Helmers SL, Duh MS, Gue´ rin A, Sarda SP, Samuelson TM, Bunker MT, et al. Clinical
and economic impact of vagus nerve stimulation therapy in patients with drugresistant
epilepsy. Epilepsy and Behavior 2011;22(2):370–5.
8. Callaghan B, Schlesinger M, Rodemer W, Pollard J, Hesdorffer D, Allen Hauser W,
et al. Remission and relapse in a drug-resistant epilepsy population followed
prospectively. Epilepsia 2001;52:619–26.

Christian Hoppe*
University of Bonn, Department of Epileptology,
Sigmund-Freud-Strasse 25, 53105 Bonn, Germany
« Last Edit: August 05, 2014, 11:31:47 AM by dennis100 » Logged
Pages: [1]   Go Up
Jump to: