CETVD again
Posted: Sun May 15, 2011 4:47 pm
I have been on holiday and so I missed the previous discussion of this. Apologies to those who do not want to discuss it again.
Like many of us I was very disappointed with Zivadinov’s recently published paper and the change in his views since the webinar last year. There is quite a difference between what he said then and what he is saying now, although as we all know the data he reported have not changed.
Having had the opportunity to read the paper I think I can now explain the difference. It is also now clear to me that the arguments which Zivadinov uses to support his published conclusions are flawed.
1) ‘Sensitivity and Specificity’
Last year at the webinar Zivadinov said that the difference between his results and Zamboni’s was due to his testing techniques not being good enough. He talked about having improved that technique since doing the CETVD (part 1) study and about using better equipment. He said that the results he was subsequently getting were better.
That view is hardly mentioned in the published report and new comments relating to sensitivity and specificity have appeared. The disappearance of one and the appearance of the other is not a coincidence.
The basic thrust of CETVD was to look at how well CCSVI correlated with MS. Zamboni used standard tests of correlation in his study and so did Zivadinov. But Zivadinov then went on to apply further tests which is where the sensitivity and specificity came in. These tests are commonly used to assess how well a diagnostic procedure detects a particular disease or condition. Specificity is a measure of how well a procedure detects just the disease you are trying to detect and no other while sensitivity measures how well you detect the presence of a disease when it is present and not when it is absent.
Zivadinov has basically asked the question here ‘How well does the presence of CCSVI predict the presence of MS (and nothing else).’ Part of the reason he did that was because he was interested in the idea that vascular problems might affect other neurological conditions which is quite acceptable.
But there is a flaw here. For the claims of sensitivity and specificity (re predicting MS) to be valid Zivadinov has first to assume that the procedure he employed to test for the presence of CCSVI is itself 100% accurate. If for example the procedure detects CCSVI 80% of the time and the correlation with MS is also 80% then the results would not conflict with a 100% correlation between CCSVI and MS. If you do not first know the sensitivity and specificity of your procedure in testing for CCSVI then you cannot conclude anything about the same test applied to MS. So Zivadinov is not justified in publishing the sensitivity and specificity claims in the way he has.
To put this another way: there is a discrepancy between Zivadinov’s results and Zamboni’s (which were 100% sensitive and specific.) Last year that discrepancy was being ascribed to deficiencies in the testing procedure. This year the discrepancy is being presented as proof of a lack of correlation between CCSVI and MS. While this view is not unreasonable it is by no means the only plausible interpretation of the data. As such this view has every reason to be included (amongst others) in the discussion but has no reason to be in the conclusion to the exclusion of any other explanation nor does it have any reason to be in the title. Surprisingly Zivadinov says himself in the discussion:
“The sensitivity, specificity, PPV, NPV, and OR figures were substantially lower in the CTEVD study compared to the initial CCSVI studies.2,4 The exact reasons for these differences are not clear.”
As we all know in the other studies which Zivadinov quotes which purported to use the same testing procedure there were wildly differing results. That could not be the case if the testing procedure was reliable and accurate. So there seems to be sufficient justification for the view that the testing procedure is not accurate enough to support Zivadinov’s conclusion. It seems to me just as likely that the sensitivity and specificity figures which Zivadinov reported are due to errors in the testing procedure and not a true measure of lack of correlation between CCSVI and MS. Until other studies add further support to either view Zivadinovs’s conclusion (in respect of sensitivity and specificity re MS) is clearly unsafe.
2) “MS causes CCSVI”
Unfortunately Zivadinov’s reasoning in concluding that “Our findings point against CCSVI having a primary causative role in the development of MS” is not strong either. This is his argument:
“The findings from the CTEVD study argue against such a hypothesis because:
1) 61.9% of patients with CIS at first symptom onset and 43.9% of patients with MS did not present with CCSVI;
2) 42.3% of patients with OND and 22.7% of HC presented with CCSVI. Given the composition of our OND group that was biased toward immune mediated or inflammatory diseases, we cannot exclude
that prevalence of CCSVI may be increased in patients with OND presenting with similar diseases.
3)Moreover, there was no CCSVI prevalence difference between genetically related and unrelated HC, which would argue against a genetic origin of the CCSVI hypothesis.
4)The prevalence of CCSVI was higher in progressive forms of MS, being the highest in relapsing SPMS (89.4%), followed by nonrelapsing SPMS and PPMS.
5)RRMS and CIS showed the lowest prevalence.
These findings suggest that CCSVI may be a consequence rather than a cause of MS”
The numbering 3,4 and 5 above is my own. Zivadinov stops at 2. I have five reservations with what has been said above and I am also not sure it is clear what Zivadinov means by ‘primary.’
1) 43.9% not having CCSVI but having MS is obviously OK as evidence that CCSVI is not the sole cause of MS. But is quite compatible with CCSVI being involved along with other factors. The CIS prevalence is even more problematic. CIS is defined in the UK (by my neuro) as ‘not MS.’ Obviously it is quite plausible that some cases of CIS will turn into MS but Zivadinov says nothing about what proportion that is. He seems to be assuming all CIS leads to MS. I tried to find an estimate by googling and found on the MSTRUST website it said ‘less than 50% of CIS is likely to turn into MS. So about 40% then? Zivadinov found 38% with CCSVI in the CIS group. This seems a fairly good fit to me. An obvious conclusion is that Zivadinov detected those people in the CIS group who would go on to develop MS and did so with good accuracy. That points against the idea that ‘early MS’ (CIS) has a lower prevalence than ‘late MS.’
2) Again 22.7% of healthy controls does point to CCSVI not being the sole cause of MS. But I struggle with what Zivadinov says about OND : “we cannot exclude that prevalence of CCSVI may be increased in patients with OND presenting with similar diseases.” Clearly there is a substantial prevalence of CCSVI in OND cases. But what is he inferring from that re: CCSVI and MS? Is he saying that CCSVI may be causative in other diseases as well as MS? That is obviously OK but does that imply anything about the relationship between CCSVI and MS? If Zamboni had presented evidence about not finding CCSVI in OND then I could see some relevance. If he was discussing specificity it would make some sense too. But as far as I can see he is not doing either of those things. So I am clearly missing whatever the link is he is trying to make. So I cannot see how this evidence supports his argument.
3) Genetic differences. The word ‘genetic’ is being used differently here from when Zamboni used ‘congenital’ in relation to venous abnormalities. Zamboni used it to mean ‘present at birth.’ ‘Birth defects’ can be caused by a number of different factors which are not necessarily ‘genetically determined’ (i.e. coded for in the genes). Vitamin D deficiency being an obvious one. Factors in the embryo’s environment during gestation seem to me just as likely to be the cause of CCSVI. So I don’t think Zivadinov is really addressing the issue. If we were talking about ‘inherited differences,’ most people accept that there is a genetic component in MS but you can have monozygotic twins where one has MS and other doesn’t. So just because subjects are genetically related does not mean both will have MS and the same argument seems to be true for CCSVI. Unless there is something I am misunderstanding I cannot see how this argument is relevant.
4) The argument in 4 and 5 is again that CCSVI prevalence increases as MS progresses. I agree this argument does have some merit. But there are a number of contrary findings here. – As I have already said the evidence from CIS doesn’t seem to work. I am not sure either about relapsing and non-relapsing SPMS. If relapsing SPMS is a stage in between RRMS and non-relapsing SPMS then his figures are the wrong way round. Non-relapsing SPMS should show a greater prevalence than relapsing SPMS and it doesn’t. Paediatric evidence would obviously be very relevant here but he does not mention it until later. When he does mention it he only says that it suggests that aging is not a factor. He says that because the paediatric prevalence is not significantly different to adult prevalence. Hm, doesn’t that point against his conclusion that CCSVI prevalence increases with disease progression? Actually the paediatric group is so small (10) nothing much can be concluded from it. If that was all Zivadinov said I would not be critical. But in fact he fails to cite it in his argument about increasing prevalence but then chooses to use it to support an argument against age being a factor. It is hard not to see that as simply selecting the evidence to suit his bias.
Clearly there is evidence here that CCSVI is not the sole cause of MS. If that is what Zivadinov means by ‘primary’ cause then I agree but I don’t like his choice of words. If he means ‘primary’ in the sense of ‘preceding’ or ‘prior’ which I think he does here, then his argument is much less convincing. The argument is certainly worth discussing but it is not strong enough to be even a tentative conclusion. If the other evidence about causation which Embry and others have cited were included in the discussion it would be clear that firm conclusions could not be justified. There is reasonable evidence pointing both ways. ‘Aging’ as a factor for example sounds very plausible. It is a factor in other neurodegenerative diseases but he hardly addresses it.
The multi-factorial nature of MS is also something which is not mentioned. Zivadinov is well aware that MS is likely to have multiple causative factors and it is much more sensible to discuss CCSVI within that context. But he doesn’t. If he did he would have to accept the possibility that the relationship between CCSVI and MS may be more complicated than the two alternatives he has considered here. It is not very intelligent to only consider that either CCSVI causes MS or MS causes CCSVI. I am sure in reality Zivadinov has much more sense than that but strangely it does not show here.
The last thing that bothers me about this paper is the sudden changes from what is said in the discussion (which sounds more reasonable) and what is in the conclusion and title (which states tentative speculation as conclusions). It is as if one person wrote the discussion and another person wrote the conclusion and title. They don’t quite agree with each other. He states the objective of the study as: “ To determine prevalence of CCSVI in a large cohort of patients with MS, clinically isolated syndrome (CIS), other neurologic diseases (OND), and healthy controls (HC), using specific proposed echo-color Doppler (ECD) criteria.” You would expect then that his conclusion would address that objective.
His actual conclusions (plural) are: “Our findings are consistent with an increased prevalence of CCSVI in MS but with modest sensitivity/specificity. Our findings point against CCSVI having a primary causative role in the development of MS.” So there are now two conclusions and one is qualified while there was only one stated objective. The major conclusion according to the data should be that he had demonstrated a significant correlation between CCSVI and MS. This relates directly to the data and the stated objective Yet he doesn’t say that. He says his findings are merely ‘consistent with’ an increased prevalence. This is a weak way of stating an actually clear and secure correlation. But he then goes on to qualify this with (unsafe) remarks about modest sensitivity/specificity making it even weaker. The second conclusion does not follow directly from the data or the stated objective but from the discussion. Yet that conclusion is unqualified and is stated as ‘pointing against’ CCSVI having a primary role in the development of MS. ‘Pointing’ is a synonym of ‘indicates’ and is certainly stronger than ‘is consistent with’ yet this ‘conclusion’ is actually speculation! In the discussion he actually says ‘These findings suggest that CCSVI may be a consequence rather than a cause of MS’ followed by ‘The role of CCSVI in contributing to or being a consequence of MS progression cannot be excluded and should be further investigated.’ To me there is a clear disconnect between the tentative views expressed in the discussion and the conclusion and title.
This also seems to reflect the differing statements that Zivadinov has made in public where he says that nothing is proven and we need more evidence. I am left very suspicious that there is a political influence here overlaying the science.
Like many of us I was very disappointed with Zivadinov’s recently published paper and the change in his views since the webinar last year. There is quite a difference between what he said then and what he is saying now, although as we all know the data he reported have not changed.
Having had the opportunity to read the paper I think I can now explain the difference. It is also now clear to me that the arguments which Zivadinov uses to support his published conclusions are flawed.
1) ‘Sensitivity and Specificity’
Last year at the webinar Zivadinov said that the difference between his results and Zamboni’s was due to his testing techniques not being good enough. He talked about having improved that technique since doing the CETVD (part 1) study and about using better equipment. He said that the results he was subsequently getting were better.
That view is hardly mentioned in the published report and new comments relating to sensitivity and specificity have appeared. The disappearance of one and the appearance of the other is not a coincidence.
The basic thrust of CETVD was to look at how well CCSVI correlated with MS. Zamboni used standard tests of correlation in his study and so did Zivadinov. But Zivadinov then went on to apply further tests which is where the sensitivity and specificity came in. These tests are commonly used to assess how well a diagnostic procedure detects a particular disease or condition. Specificity is a measure of how well a procedure detects just the disease you are trying to detect and no other while sensitivity measures how well you detect the presence of a disease when it is present and not when it is absent.
Zivadinov has basically asked the question here ‘How well does the presence of CCSVI predict the presence of MS (and nothing else).’ Part of the reason he did that was because he was interested in the idea that vascular problems might affect other neurological conditions which is quite acceptable.
But there is a flaw here. For the claims of sensitivity and specificity (re predicting MS) to be valid Zivadinov has first to assume that the procedure he employed to test for the presence of CCSVI is itself 100% accurate. If for example the procedure detects CCSVI 80% of the time and the correlation with MS is also 80% then the results would not conflict with a 100% correlation between CCSVI and MS. If you do not first know the sensitivity and specificity of your procedure in testing for CCSVI then you cannot conclude anything about the same test applied to MS. So Zivadinov is not justified in publishing the sensitivity and specificity claims in the way he has.
To put this another way: there is a discrepancy between Zivadinov’s results and Zamboni’s (which were 100% sensitive and specific.) Last year that discrepancy was being ascribed to deficiencies in the testing procedure. This year the discrepancy is being presented as proof of a lack of correlation between CCSVI and MS. While this view is not unreasonable it is by no means the only plausible interpretation of the data. As such this view has every reason to be included (amongst others) in the discussion but has no reason to be in the conclusion to the exclusion of any other explanation nor does it have any reason to be in the title. Surprisingly Zivadinov says himself in the discussion:
“The sensitivity, specificity, PPV, NPV, and OR figures were substantially lower in the CTEVD study compared to the initial CCSVI studies.2,4 The exact reasons for these differences are not clear.”
As we all know in the other studies which Zivadinov quotes which purported to use the same testing procedure there were wildly differing results. That could not be the case if the testing procedure was reliable and accurate. So there seems to be sufficient justification for the view that the testing procedure is not accurate enough to support Zivadinov’s conclusion. It seems to me just as likely that the sensitivity and specificity figures which Zivadinov reported are due to errors in the testing procedure and not a true measure of lack of correlation between CCSVI and MS. Until other studies add further support to either view Zivadinovs’s conclusion (in respect of sensitivity and specificity re MS) is clearly unsafe.
2) “MS causes CCSVI”
Unfortunately Zivadinov’s reasoning in concluding that “Our findings point against CCSVI having a primary causative role in the development of MS” is not strong either. This is his argument:
“The findings from the CTEVD study argue against such a hypothesis because:
1) 61.9% of patients with CIS at first symptom onset and 43.9% of patients with MS did not present with CCSVI;
2) 42.3% of patients with OND and 22.7% of HC presented with CCSVI. Given the composition of our OND group that was biased toward immune mediated or inflammatory diseases, we cannot exclude
that prevalence of CCSVI may be increased in patients with OND presenting with similar diseases.
3)Moreover, there was no CCSVI prevalence difference between genetically related and unrelated HC, which would argue against a genetic origin of the CCSVI hypothesis.
4)The prevalence of CCSVI was higher in progressive forms of MS, being the highest in relapsing SPMS (89.4%), followed by nonrelapsing SPMS and PPMS.
5)RRMS and CIS showed the lowest prevalence.
These findings suggest that CCSVI may be a consequence rather than a cause of MS”
The numbering 3,4 and 5 above is my own. Zivadinov stops at 2. I have five reservations with what has been said above and I am also not sure it is clear what Zivadinov means by ‘primary.’
1) 43.9% not having CCSVI but having MS is obviously OK as evidence that CCSVI is not the sole cause of MS. But is quite compatible with CCSVI being involved along with other factors. The CIS prevalence is even more problematic. CIS is defined in the UK (by my neuro) as ‘not MS.’ Obviously it is quite plausible that some cases of CIS will turn into MS but Zivadinov says nothing about what proportion that is. He seems to be assuming all CIS leads to MS. I tried to find an estimate by googling and found on the MSTRUST website it said ‘less than 50% of CIS is likely to turn into MS. So about 40% then? Zivadinov found 38% with CCSVI in the CIS group. This seems a fairly good fit to me. An obvious conclusion is that Zivadinov detected those people in the CIS group who would go on to develop MS and did so with good accuracy. That points against the idea that ‘early MS’ (CIS) has a lower prevalence than ‘late MS.’
2) Again 22.7% of healthy controls does point to CCSVI not being the sole cause of MS. But I struggle with what Zivadinov says about OND : “we cannot exclude that prevalence of CCSVI may be increased in patients with OND presenting with similar diseases.” Clearly there is a substantial prevalence of CCSVI in OND cases. But what is he inferring from that re: CCSVI and MS? Is he saying that CCSVI may be causative in other diseases as well as MS? That is obviously OK but does that imply anything about the relationship between CCSVI and MS? If Zamboni had presented evidence about not finding CCSVI in OND then I could see some relevance. If he was discussing specificity it would make some sense too. But as far as I can see he is not doing either of those things. So I am clearly missing whatever the link is he is trying to make. So I cannot see how this evidence supports his argument.
3) Genetic differences. The word ‘genetic’ is being used differently here from when Zamboni used ‘congenital’ in relation to venous abnormalities. Zamboni used it to mean ‘present at birth.’ ‘Birth defects’ can be caused by a number of different factors which are not necessarily ‘genetically determined’ (i.e. coded for in the genes). Vitamin D deficiency being an obvious one. Factors in the embryo’s environment during gestation seem to me just as likely to be the cause of CCSVI. So I don’t think Zivadinov is really addressing the issue. If we were talking about ‘inherited differences,’ most people accept that there is a genetic component in MS but you can have monozygotic twins where one has MS and other doesn’t. So just because subjects are genetically related does not mean both will have MS and the same argument seems to be true for CCSVI. Unless there is something I am misunderstanding I cannot see how this argument is relevant.
4) The argument in 4 and 5 is again that CCSVI prevalence increases as MS progresses. I agree this argument does have some merit. But there are a number of contrary findings here. – As I have already said the evidence from CIS doesn’t seem to work. I am not sure either about relapsing and non-relapsing SPMS. If relapsing SPMS is a stage in between RRMS and non-relapsing SPMS then his figures are the wrong way round. Non-relapsing SPMS should show a greater prevalence than relapsing SPMS and it doesn’t. Paediatric evidence would obviously be very relevant here but he does not mention it until later. When he does mention it he only says that it suggests that aging is not a factor. He says that because the paediatric prevalence is not significantly different to adult prevalence. Hm, doesn’t that point against his conclusion that CCSVI prevalence increases with disease progression? Actually the paediatric group is so small (10) nothing much can be concluded from it. If that was all Zivadinov said I would not be critical. But in fact he fails to cite it in his argument about increasing prevalence but then chooses to use it to support an argument against age being a factor. It is hard not to see that as simply selecting the evidence to suit his bias.
Clearly there is evidence here that CCSVI is not the sole cause of MS. If that is what Zivadinov means by ‘primary’ cause then I agree but I don’t like his choice of words. If he means ‘primary’ in the sense of ‘preceding’ or ‘prior’ which I think he does here, then his argument is much less convincing. The argument is certainly worth discussing but it is not strong enough to be even a tentative conclusion. If the other evidence about causation which Embry and others have cited were included in the discussion it would be clear that firm conclusions could not be justified. There is reasonable evidence pointing both ways. ‘Aging’ as a factor for example sounds very plausible. It is a factor in other neurodegenerative diseases but he hardly addresses it.
The multi-factorial nature of MS is also something which is not mentioned. Zivadinov is well aware that MS is likely to have multiple causative factors and it is much more sensible to discuss CCSVI within that context. But he doesn’t. If he did he would have to accept the possibility that the relationship between CCSVI and MS may be more complicated than the two alternatives he has considered here. It is not very intelligent to only consider that either CCSVI causes MS or MS causes CCSVI. I am sure in reality Zivadinov has much more sense than that but strangely it does not show here.
The last thing that bothers me about this paper is the sudden changes from what is said in the discussion (which sounds more reasonable) and what is in the conclusion and title (which states tentative speculation as conclusions). It is as if one person wrote the discussion and another person wrote the conclusion and title. They don’t quite agree with each other. He states the objective of the study as: “ To determine prevalence of CCSVI in a large cohort of patients with MS, clinically isolated syndrome (CIS), other neurologic diseases (OND), and healthy controls (HC), using specific proposed echo-color Doppler (ECD) criteria.” You would expect then that his conclusion would address that objective.
His actual conclusions (plural) are: “Our findings are consistent with an increased prevalence of CCSVI in MS but with modest sensitivity/specificity. Our findings point against CCSVI having a primary causative role in the development of MS.” So there are now two conclusions and one is qualified while there was only one stated objective. The major conclusion according to the data should be that he had demonstrated a significant correlation between CCSVI and MS. This relates directly to the data and the stated objective Yet he doesn’t say that. He says his findings are merely ‘consistent with’ an increased prevalence. This is a weak way of stating an actually clear and secure correlation. But he then goes on to qualify this with (unsafe) remarks about modest sensitivity/specificity making it even weaker. The second conclusion does not follow directly from the data or the stated objective but from the discussion. Yet that conclusion is unqualified and is stated as ‘pointing against’ CCSVI having a primary role in the development of MS. ‘Pointing’ is a synonym of ‘indicates’ and is certainly stronger than ‘is consistent with’ yet this ‘conclusion’ is actually speculation! In the discussion he actually says ‘These findings suggest that CCSVI may be a consequence rather than a cause of MS’ followed by ‘The role of CCSVI in contributing to or being a consequence of MS progression cannot be excluded and should be further investigated.’ To me there is a clear disconnect between the tentative views expressed in the discussion and the conclusion and title.
This also seems to reflect the differing statements that Zivadinov has made in public where he says that nothing is proven and we need more evidence. I am left very suspicious that there is a political influence here overlaying the science.
