Embry: Actual data from PREMiSe suggests CCSVI is of value

[ScutFarkus]

Active MS is one of the most impossible states of this disease to predict, timing the 'regular' relapsers so that you can treat when in the worst inflammatory cycle is an interesting ploy!

It is typical for MS clinical trials to have recent disease activity as an enrollment criteria. The reason is simple: you want there to be as few differences as possible among the subjects in your test, and you cannot measure changes in disease rate if there is no activity. So this isn't a "ploy" at all; it's typical and reasonable.

If the testing was done to find out why there are nil responders to treatment Internationally occurring, I would have 'assumed' would have been a stated purpose in the design of the trial.

I don't know what this sentence means.

To test if undertreating could in future be classified as less than 50% flow improvement from/at PTA appears to be the purpose of phase 2 in reality?

No, you can't redefine the "purpose" based on results. We have no evidence to suggest that the purpose, or intent, of the study was anything different that they stated. All we can say from the results is that their CCSVI intervention wasn't very successful.

In phase 1 treating and achieving greater than 75% flow improvement was a success!
So why cherry pick the two phases PwMS?

I don't think you understand what I mean by "cherry pick". Embry (and others who try comparing the Phase 1 results to Phase 2 results now) are guilty of cherry picking. This is when you look at a bunch of data, and pick out the parts that support a given conclusion. The way to avoid cherry picking is to define the specific measures (e.g. what data you'll look at) before you collect and analyze the data.

It appears to be purposeful and the outcome, predictable for that phase, to not improve the flow.

I keep coming back to 'why' and what pressures were happening when the trial was designed by BNAC?

Something purposeful by manipulation and attempted stealth happened in this trial, was it to disillusion the public, was it to see what they could get away with, was it about exposing bias, was it about profit at any cost?

I think people are way, way too quick to claim fraud and bias on these forums. Those are very serious accusations, and the only "proof" being presented is the disappointing study results. Sure, fraud happens (e.g. Wakefield and his bogus claims that vaccines are related to autism), but it's rare. Bias (unintentional or otherwise) is likely somewhat more common, which is why it's important for studies to be independently replicated. What we have here is one study, where the authors conclude that CCSVI treatment should only be undertaken as part of a clinical trial. I believe Zamboni himself makes exactly the same recommendation.

/Scut

[Squeakycat]

What we have here is one study, where the authors conclude that CCSVI treatment should only be undertaken as part of a clinical trial. I believe Zamboni himself makes exactly the same recommendation.

/Scut

Agree completely with your criticism of Embry for cherry picking and the issues you are raising with the comments.

But, they did come to one other conclusion and that was that PTVA makes MS worse.

I don't think anyone disagrees with the idea that it would be great if people were only treated in a clinical trial, other than the minor problem that there are not many clinical trials.

To me, the issue is not fraud, perhaps it is bias, but more importantly and regardless of why, it seems to simply be an invalid conclusion based on what we know now: There was no improvement in blood flow of those treated.

By definition, the whole point of treatment is to improve blood flow. If blood flow doesn't improve, it seems to me that the only valid conclusion is that the treatment failed, not that it made MS worse.

I also wonder what happens if the outlier person in the treatment arm who ended up with 8 lesions out of a total of 12 for the other 18 participants is thrown out. Does that change the conclusion that treatment, adequate or inadequate, causes MS symptoms to get worse. The related question is whether treatment caused an impairment in this person's blood flow.

[NZer1]

The progression of RRMS is 'flexible and erratic' and there was an attempt by the selection process to influence the outcome of the trial stages, this is obvious and wasn't noted as a purpose of the study, WHY?

The conclusions don't match the data, again WHY?

The presentations to different conferences and public groups varies, WHY?

The PTA treating method to achieve different outcome percentages of flow occurred to different groups of PwMS, but was not noted in the purpose of the study, WHY?

Did the study follow the stated purpose YES or NO ? WHY?

The announcements to Media were different from individuals from BNAC involved in the study, WHY?

Are BNAC united in the data that has been produced and is there agreement in the conclusions, WHY?

Some things cannot be overlooked and need to be acknowledged.

ScutFarkus, I did understand you comments and opinion, I was presenting mine, sorry to hear you didn't understand mine.

;)
Nigel

[ScutFarkus]

What we have here is one study, where the authors conclude that CCSVI treatment should only be undertaken as part of a clinical trial. I believe Zamboni himself makes exactly the same recommendation.

/Scut

Agree completely with your criticism of Embry for cherry picking and the issues you are raising with the comments.

But, they did come to one other conclusion and that was that PTVA makes MS worse.

I think you're overstating their conclusion a bit. What they said in the conclusion section of the paper was "...more sizable change in venous outflow was associated with increased disease activity
primarily noted on MRI." Maybe this is too fine a point, but "was associated with" is technically quite different from "makes", because the first is a statement of an observed correlation, while the second implies causality.

It's like the difference between "umbrella usage is associated with rain" and "umbrella usage makes it rain". One is a simple statement of fact, the other is silly.

I don't think anyone disagrees with the idea that it would be great if people were only treated in a clinical trial, other than the minor problem that there are not many clinical trials.

To be honest, I don't fully agree with the repeated insistence that people only have treatments like this if they're part of a trial. I personally think a well-informed person should be allowed to pay for whatever treatments they want, so long as they're responsible for the consequences. Things get less clear once health insurance gets added to the mix, but in general I'm a fan of choice.

To me, the issue is not fraud, perhaps it is bias, but more importantly and regardless of why, it seems to simply be an invalid conclusion based on what we know now: There was no improvement in blood flow of those treated.

But that's not the point. Assuming you accept that the subjects were treated in good faith, this (limited) data suggests that the treatment performed may not be successful in many cases, and furthermore might be associated with increased disease activity. I think we can agree that surgery tends to be rather stressful, and many people think stress can trigger disease activity, so it's plausible that there is causality here, even if the authors don't claim that.

By definition, the whole point of treatment is to improve blood flow. If blood flow doesn't improve, it seems to me that the only valid conclusion is that the treatment failed, not that it made MS worse.

If you take your car to the shop to fix the air conditioner, and after you get it back neither the heater nor the air conditioner works, is it valid to conclude that the mechanic broke your heater, or is it only valid to conclude that the air conditioner wasn't fixed?

I also wonder what happens if the outlier person in the treatment arm who ended up with 8 lesions out of a total of 12 for the other 18 participants is thrown out. Does that change the conclusion that treatment, adequate or inadequate, causes MS symptoms to get worse. The related question is whether treatment caused an impairment in this person's blood flow.

That is indeed an interesting question, but unfortunately tossing out that person's data and drawing new conclusions is pretty much the definition of "cherry picking". That said, it would be interesting, and if nothing else this is a reminder of how hard it is to do MS studies with small sample sizes (because outliers can be waaay out there).

/Scut

[Squeakycat]

I think you're overstating their conclusion a bit. What they said in the conclusion section of the paper was "...more sizable change in venous outflow was associated with increased disease activity primarily noted on MRI." Maybe this is too fine a point, but "was associated with" is technically quite different from "makes", because the first is a statement of an observed correlation, while the second implies causality.

It's like the difference between "umbrella usage is associated with rain" and "umbrella usage makes it rain". One is a simple statement of fact, the other is silly.

I don't think it is an overstatement in the context of the video and press release and certainly not in terms of how those pronouncement were then spun by those writing about the study.

To be honest, I don't fully agree with the repeated insistence that people only have treatments like this if they're part of a trial. I personally think a well-informed person should be allowed to pay for whatever treatments they want, so long as they're responsible for the consequences. Things get less clear once health insurance gets added to the mix, but in general I'm a fan of choice.

Okay. I guess you are one of those SillyCon Valley libertarians. My wife has been treated three times outside of a clinical trial, but we did make an attempt to get into one.

But that's not the point. Assuming you accept that the subjects were treated in good faith, this (limited) data suggests that the treatment performed may not be successful in many cases, and furthermore might be associated with increased disease activity. I think we can agree that surgery tends to be rather stressful, and many people think stress can trigger disease activity, so it's plausible that there is causality here, even if the authors don't claim that.

I think we have to assume the study was done in good faith and if there had not been improvement in the VHISS scores in the Phase 1 study, could accept that sometimes "treatment preformed may not be successful." But here, it appears that it was universally successful in Phase 1 and failed almost completely in Phase 2. I won't jump from that to conspiracy theories, but do think it is something that has to be addressed, no?

That is indeed an interesting question, but unfortunately tossing out that person's data and drawing new conclusions is pretty much the definition of "cherry picking". That said, it would be interesting, and if nothing else this is a reminder of how hard it is to do MS studies with small sample sizes (because outliers can be waaay out there).

I wasn't suggesting that we throw out the data and draw new conclusions, though that might have been an appropriate a priori consideration of the study regardless of which way it fell, but I think it is appropriate to look at means in the context of the variation, not as absolute values. It seems that we often learn more from the 3rd deviation than from the mean.

[ScutFarkus]

I think you're overstating their conclusion a bit.

I don't think it is an overstatement in the context of the video and press release and certainly not in terms of how those pronouncement were then spun by those writing about the study.

OK, I can buy that. I'll admit I haven't actually watched the video, and I find the whole idea of "pre-releasing" study results to the public before they've been peer-reviewed to be dubious at best.

Okay. I guess you are one of those SillyCon Valley libertarians. My wife has been treated three times outside of a clinical trial, but we did make an attempt to get into one.

Wow, you nailed it!

But that's not the point. Assuming you accept that the subjects were treated in good faith, this (limited) data suggests that the treatment performed may not be successful in many cases, and furthermore might be associated with increased disease activity. I think we can agree that surgery tends to be rather stressful, and many people think stress can trigger disease activity, so it's plausible that there is causality here, even if the authors don't claim that.

I think we have to assume the study was done in good faith and if there had not been improvement in the VHISS scores in the Phase 1 study, could accept that sometimes "treatment preformed may not be successful." But here, it appears that it was universally successful in Phase 1 and failed almost completely in Phase 2. I won't jump from that to conspiracy theories, but do think it is something that has to be addressed, no?

Yes, I agree that stands out as something in need of further investigation. However, since (as was mentioned earlier) the Phase 1 and Phase 2 groups had very different enrollment criteria, it's reasonable there would be some general differences. But it's hard to see how that would explain such dramatic differences in success rate, unless active MS actually causes VHISS worsening...

Anyway, I think we're both in pretty strong agreement that Embry was off base in his accusations and conclusions, and that this study raises several questions. Glad to have a discussion with someone so reasonable!

/Scut

[NZer1]

If you look at the Psychology in this BNAC trial it opens up the picture of what has transpired.

If there was a preconceived outcome, an assumption, then the study design 'should have' the outcome in mind. The design of the study will have the required bias and the outcome 'should' be as assumed.
In this trial the outcome didn't match the bias and the reporting appears preconceived, naturally the public response has been about honesty most of all because of the incongruence at all levels.

When the results don't match the process what can be done?

Bias or Truth? Politics or Research? Profit or Loss?


Nigel

[ScutFarkus]

If you look at the Psychology in this BNAC trial it opens up the picture of what has transpired.

If there was a preconceived outcome, an assumption, then the study design 'should have' the outcome in mind. The design of the study will have the required bias and the outcome 'should' be as assumed.
In this trial the outcome didn't match the bias and the reporting appears preconceived, naturally the public response has been about honesty most of all because of the incongruence at all levels.

When the results don't match the process what can be done?

Bias or Truth? Politics or Research? Profit or Loss?

Can you provide specific supporting examples for your claims? It's very tough to respond to general claims like "results don't match the process" because I don't see where that happened, and you aren't sharing any specific examples. This is the same reason I didn't respond to your prior post in this thread.

It is perfectly fine for a study to have an outcome in mind, in the sense that many (most?) experiments serve to test a hypothesis. A hypothesis is of the form "if we do X, then Y will happen", so Y is the outcome that's "expected". So your experiment is designed to see if the data supports your hypothesis. If you do the experiment properly and the results don't support the hypothesis, the experiment was still a success because it served its purpose of testing the hypothesis. You can't do a real test if you only accept results that match your hypothesis!

Now, what typically happens in real life is that negative results are disappointing, especially if the hypothesis is something you hoped was true, so you comb over the experiment and resulting data, hoping to find some anomaly, flaw or shred of supporting evidence that lets you explain away the poor results. This is fine as far as it goes, so long as perceived problems are fixed by performing a new experiment with a better/stronger/different design, rather than by trying to analyze away the "flaws" after the fact.

/Scut

[cheerleader]

It's actually not that complicated. The poster contains the answer.

The pwMS who were treated in phase 1 received a venous return of > or equal to 75% after 6 months.
The pwMS who were treated in phase 2 (the blinded arm) received treatment which allowed them a venous return of no more than 50%.
However, only the phase 2 results were reported to the press.

The people who were treated in phase 1 had EDSS reduction, reduction in relapses and symptom relief.
When CCSVI is correctly treated, and venous return > or equal to 75% is acheived, there is healing.
Venoplasty for CCSVI is very effective when done correctly. PREMiSe phase 1 showed that clearly.
It also showed that when venoplasty is insufficient, there is no benefit, and may be harmful.


from the poster:
• In phase 1 (Figure 1), there was significant improvement of VHISS (p<0.0001) over 6-months that resulted in >75% restoration of the venous outflow compared to baseline.

• In phase 2, improvement was observed also in treatment (p=0.02) and sham (p=0.04) arms at month 1 but did not reach >75% restoration of the venous outflow compared to baseline. No differences in VHISS improvement were detected between phase 2 treated and sham groups (p=0.894).

Here, read it for yourselves. Don't take my word for it. It really doesn't matter what we believe or what our bias might be. It matters what the research shows. When venous return is not adequately corrected, there is no benefit. When it is, there is benefit.
http://www.hubbardfoundation.org/CCSVI_ ... apers.html

But that's not what the video or press release discussed.
that's the problem,
cheer

[ScutFarkus]

It's actually not that complicated. The poster contains the answer.

The pwMS who were treated in phase 1 received a venous return of > or equal to 75% after 6 months.
The pwMS who were treated in phase 2 (the blinded arm) received treatment which allowed them a venous return of no more than 50%.
However, only the phase 2 results were reported to the press.

There's no mystery here. The Phase 1 test ended long ago (2010, correct?) and was designed purely to assess safety (and in fact they DID report the successful result of that study back in 2010, since it was what allowed them to continue to Phase 2, see http://www.ubneurosurgery.neuroscienced ... cpid=27174).

The people who were treated in phase 1 had EDSS reduction, reduction in relapses and symptom relief.
When CCSVI is correctly treated, and venous return > or equal to 75% is acheived, there is healing.
Venoplasty for CCSVI is very effective when done correctly. PREMiSe phase 1 showed that clearly.
It also showed that when venoplasty is insufficient, there is no benefit, and may be harmful.

You cannot draw those conclusions from the Phase 1 trial. It was neither blinded nor randomized, nor was the enrollment controlled for similar levels of disease activity, etc. As far as symptom improvement, it is no better than the various anecdotal reports that fill these forums.

The Phase 2 trial represents the first true test of intervention, since it was properly blinded and placebo-controlled. Sadly, the results were negative, possibly due to the ineffectiveness of the actual intervention. But we don't really know why it failed, only that it did fail to show improvement after treatment.

But that's not what the video or press release discussed.
that's the problem

That's also not what the experiment showed. The Phase 2 trial really was the important news here, not two-year old confirmation that the surgery is relatively safe.

/Scut

[NZer1]

If Dr Zivadinov was to present results and conclusions in his own words and with his personal advisers/team helping,

AND if Dr Siddiqui presented independently his results and conclusions in his own words and with his personal advisers/team helping,

Would the documents match?

What is the issue if incongruence occurs in a research study?

The data does not lie it is presented, may not be complete though.

If the two documents did not come to the same conclusion what would you look for to understand the reason?

If an independent, qualified team of Researchers read and documented a conclusion would that be different from other BNAC documents which are based on the same data?

ScutFarkus I don't think you and I could discuss this, I am finding that you are fixed in your opinion and any other viewpoint will not matter to you.

Joan thanks for comment above, clear and accurate imo

;)
Nigel

[cheerleader]

from the poster:
• In phase 1 (Figure 1), there was significant improvement of VHISS (p<0.0001) over 6-months that resulted in >75% restoration of the venous outflow compared to baseline.

• In phase 2, improvement was observed also in treatment (p=0.02) and sham (p=0.04) arms at month 1 but did not reach >75% restoration of the venous outflow compared to baseline. No differences in VHISS improvement were detected between phase 2 treated and sham groups (p=0.894).

This is all that matters. Restoration of venous outflow compared to baseline and improved VHISS. It is a scientific measurement. It is a quantifiable measurement. And it was not acheived in phase 2.

cheer

[Squeakycat]

from the poster:
• In phase 1 (Figure 1), there was significant improvement of VHISS (p<0.0001) over 6-months that resulted in >75% restoration of the venous outflow compared to baseline.

• In phase 2, improvement was observed also in treatment (p=0.02) and sham (p=0.04) arms at month 1 but did not reach >75% restoration of the venous outflow compared to baseline. No differences in VHISS improvement were detected between phase 2 treated and sham groups (p=0.894).

This is all that matters. Restoration of venous outflow compared to baseline and improved VHISS. It is a scientific measurement. It is a quantifiable measurement. And it was not acheived in phase 2.

cheer

Let me see if I can clarify this issue.

Scut is simply pointing out that you cannot SCIENTIFICALLY compare the results of Phase 1 to Phase 2.

The reason is simple. They are apples and oranges. It is completely academic that they are "scientific" or "quantifiable" they are different.

Phase 2 was a random, blind trial and had inclusion criteria that were different than those in Phase 1. They are two different studies and you can't equate and compare them.

That's not to say that you can't raise questions about the Phase 2 trial on the basis of what was found in the Phase 1 trial, but to be SCIENTIFICALLY correct, we need to raise those questions WITHIN the context of the Phase 2 trial itself, not some other study that was structured differently.

So we can ask, why there was no improvement in VHISS in those treated in Phase 2. We can ask how the VHISS can be the same between those treated for stenosis and those who were in the sham arm of the Phase 2 trial since the whole point of treatment is to improve blood flow.

These are perfectly valid questions because they are within the context of a single trial. They could be compared with another random, blinded trial with the same inclusion criteria, but not with other trials that are differently structured.

Is that clear? Nigel, this isn't being close-minded, it is simply following the rules of the game. It also doesn't mean that we can't raise questions about the results in the context of the trial itself.

It is perfectly valid to ask whether treatment itself was a failure given that there was little post-intervention difference in VHISS between those who were "treated" and those who underwent the sham procedure.

It is perfectly valid to say that this suggests that the treatment itself failed since there was no difference in blood flow between the groups IN THE PHASE 2 STUDY. And since there was no difference in blood flow, why would you expect any difference in outcome?

Look at all the different studies of CCSVI being done by people who don't use the same protocol that Zamboni used.

They say nothing about what Zamboni found because they used a different technique to measure what he measured.

The only valid conclusion from their studies is that if you use a different technique to evaluate CCSVI, you will get a different result.

But their studies do not disprove what Zamboni found. They only prove that using a different approach gives you different results.

It isn't being close-minded to insist that we apply the same rules for evaluating research when it goes against what you hope for as when it confirms what you hope will be the outcome.

[ScutFarkus]

This is all that matters. Restoration of venous outflow compared to baseline and improved VHISS. It is a scientific measurement. It is a quantifiable measurement. And it was not acheived in phase 2.

cheer

Let me see if I can clarify this issue.

Very well clarified! I agree with Squeakycat's summary.

/Scut

[NZer1]

Thanks all,
I do understand the outcomes, my point has always been the same, WHY design a trial the way they have?

I wonder why in Phase 2 there was a constraint on the treatment that it was only to achieve a maximum of 50% improvement, no more?

The purpose of this percentage regarding treatment has not been explained so that creates questions regarding the purpose of the entire trial.

The safety research concept as a stand alone Phase 1 and not inclusive part of the entire trial appears to be an expensive waste of money available and potential for finding a deeper understanding, when the task is performed anyway.
If 75% gives one outcome, why was 75% chosen and not any other figure to experiment with the safety concept?

And in Phase 2 50% tops is used with differently selected patients from Phase 1, it gives a purposeful impression OR it gives an impression that separate theories were being tested.
Were the researchers united in both the trial stages or was there complete separation of Phase 1 and Phase 2 projects?

Could the trial have been planned differently and achieved more CCSVI understandings if there had been more progression of treatment stages with the same criteria for patient selection and then the safety finding would have been based on boarder knowledge of degrees of risk with percentage of resizing, blinded or not.

The safety understanding would have evolved with trial and outcome in Phase 2 rather than selecting a set figure Phase 1 with no other criteria and saying one percentage was safe on 10 subjects that were picked by 'xyz' screening?

The 50% PTA maximum resizing (from anecdotal feedback from other researchers) would not achieve an outcome that was sufficient to measure with the tools/methods eg.EDSS being used, why use 50% in this part of the trial when you are also going to the expense of comparing against controls.
By selecting the participants differently in Phase 1 and Phase 2 it disables being able to compare Phase 1 against Phase 2 in almost every way, this makes the trial a split focus that cannot be purposeful or economic!

What percentage and criteria will be chosen for Phase 3, will it match Phase 2 in some regard and what is the purpose of the patient picking criteria with a disease with an unknown disease progression?
Without a Phase or Stage approach how will that effect the cost going of the entire trial, this is why I question the in-congruency of the process, not the outcome interpretations. My mind is wired for Spatial thinking, and said to be very high.
Interpretations will be sorted over time, the challenge is to perform a logical purposeful design for the entire trial. Plan!!!! Time!!!! Costs!!!! Minimise Waste!!!! Maximise Learning Potential!!!!

My favourite question,
WHY? WHY? WHY?

;)Nigel