Under its Evidence-based Practice Program, the Agency for Healthcare Research and Quality (AHRQ) is developing scientific information for other agencies and organizations on which to base clinical guidelines, performance measures, and other quality improvement tools. Contractor institutions review all relevant scientific literature on assigned clinical care topics and produce evidence reports and technology assessments, conduct research on methodologies and the effectiveness of their implementation, and participate in technical assistance activities.
Select for PDF version (727 KB). PDF Help.
Introduction / Research Questions / Key Terms and Definitions / Methods / Literature Sources / Search Strategies / Results and Discussion / Future Research / Availability of Full Report / References
The Social Security Administration (SSA) operates the world's largest and most stringent disability program, processing more than 3.5 million claims each year, with multiple sclerosis (MS) representing the third most common neurological diagnosis cited as the cause for disability.1 The purpose of this project, nominated by SSA and contracted through the Agency for Healthcare Research and Quality (AHRQ), is to determine whether current medical knowledge supports the SSA's stated policies regarding MS. In January 2003, the Duke Evidence-based Practice Center began work on this 13-month task to review evidence from the medical literature for use in updating SSA's listing of impairments for multiple sclerosis (MS) and for revising its disability policy (if indicated).
The seven major research questions addressed during this review are as follows:
Question 1a: What is the reliability of new McDonald criteria (incorporating supplementary information from radiologic and laboratory studies including magnetic resonance imaging [MRI], visual evoked potential [VEP], and cerebrospinal fluid [CSF] analyses) compared with long-term followup diagnosis of clinically definite MS according to the Poser criteria?
Question 1b: What is the inter-rater reliability of diagnosis of MS according to Poser or McDonald criteria among neurologists or between neurologists and non-neurologist physicians?
Question 2: What clinical indicators, including particularly time-course of impairments, predict physical or mental impairment at 12 months?
Question 3a: Among patients with MS, do current disease-modifying treatments result in long-term improvements in physical or mental outcomes compared to placebo or usual care?
Question 3b: Among patients with MS, do treatments aimed at symptom management result in improvements in physical or mental outcomes compared to usual care?
Question 4: Among individuals with MS, what physical, mental, laboratory, or radiographic findings have been associated with inability to work?
Question 5: Among individuals with MS, how does elevated temperature or other environmental factors impair the capacity to work?
Knowledge of the terms used in the SSA disability evaluation process, components of that process, and Medical Listing criteria related to MS is critical to the reader's understanding of this report. To assist in the preparation of the report, SSA provided explanations of terms and processes as currently defined by SSA regulations and rulings. The terms cited below, as well as other terms and processes used by SSA for disability determination, are defined and described in the SSA publication, Disability Evaluation Under Social Security 2003.2
The statutory definition of "Disability" is "the inability to engage in any substantial gainful activity by reason of a medically determinable physical or mental impairment(s) which can be expected to result in death or which has lasted or can be expected to last for a continuous period of not less than 12 months." This definition differs from the clinically used definition of the World Health Organization's International Classification of Impairments, Disabilities and Handicaps,3 which defines disability as "any restriction or lack of ability to perform an activity in a manner or within the range considered normal for a human being." While much of the medical literature uses the latter, broader definition, the reader must be aware that the goals of this report relate to the statutory definition.
The following terms are defined by current (2003) SSA regulations:
The regulations include a Listing of Impairments for each body system that define disability. Often referred to as the "medical listings," this list allows quick disability determinations to be made on the basis of medical criteria alone. The SSA publication, Disability Evaluation Under Social Security 2003,2 under the neurological category of impairments, includes Listing 11.09.
11.09 Multiple Sclerosis with:
Full details on the Medical Listing for multiple sclerosis, including the imbedded references to sections 2, 11, and 12, are available in the above-cited SSA publication.2
"Residual Functional Capacity" is assessed when a claimant is determined to have a "severe" impairment that does not meet or equal the intent of the medical listings. Physical capacity (lifting, carrying, walking, standing, sitting, pedaling, and so on) and mental capacity (cognitive and behavioral, thought processing, concentration, pace, behavior) are assessed in determining residual functional capacity.
To adjudicate claims by individuals with MS for disability benefits, SSA must determine whether the claims file includes information from an acceptable medical source that documents the signs, symptoms, and laboratory findings that are diagnostic of a physical or mental impairment. SSA adjudicators also determine whether the impairment would be expected to more than minimally interfere with the claimant's capacity to carry out basic work activities for at least 12 consecutive months or end in death. If a severe impairment is identified, the adjudicator determines whether the medical findings meet or equal an impairment in the medical listings. If the documented impairment does not meet or equal a listed impairment, the adjudicator must determine the claimant's residual functional capacity and consider vocational factors prior to making a final disability determination.
A systematic and comprehensive search of the medical literature was conducted and was followed by a thorough review and evaluation of the literature determined to be relevant to the major research questions.
The primary sources of literature were MEDLINE® (1966-April 2003), CINAHL® (1983-April 2003), Cochrane Database of Systematic Reviews, and Web of Science. Searches of these databases were supplemented by reviews of reference lists contained in all included articles and in relevant review articles and meta-analyses.
Searches were limited to the English language and to human subjects. For efficacy-of-treatment topics, the searches were also limited to studies with randomized controlled trial designs. In all, there were five major searches:
All searches, including narrowly focused sub-searches, yielded 1,487 potentially relevant citations.
For each question, we developed fairly detailed instructions and decision rules for the screeners' reference. There were very broad inclusion requirements for abstracts: MS study subjects and potential relevance to any of the five questions. For the full-text screening, screeners were asked to record their include/exclude decision, research question assignment, and, if appropriate, exclusion criteria that detailed insufficiencies in study design and clinical substance requirements.
The titles and abstracts of the 1,487 articles were reviewed against the inclusion/exclusion criteria by at least two of five clinical investigators. The full text of each article passing the title-and-abstract screening was retrieved from the library for further review.
At the full-text review stage, each article was independently evaluated by two investigators, who forwarded their decisions to the task order manager for recording and comparison. If indicated, reviewers were asked to reconcile differences of opinion and return a reconciled final decision. If reviewers had difficulty reaching agreement, or submitted indecisive codes, the principal investigator was the arbiter.
Approximately 50 percent of the articles were included after the abstract screening and full-text article review stages.
Data from articles included after full-text screening were abstracted directly into an evidence table template, which served as a data abstraction form. The study's writer/editor began the process with a partial abstraction of each included article. The partial abstraction included descriptions of the study design, interventions, number of subjects at the start of the study, and types of outcomes data to be collected. The partial abstraction form was forwarded to a clinician for completion and then returned to the writer/editor, who checked it for completeness and consistency of information and forwarded it to a second clinician for over-reading. The over-reader returned the table to the writer/editor for a final check of the completeness of the content, editing, and formatting.
At the end of the data abstraction stage and the very close scrutiny of each article, 168 articles were included.
The primary goal of this review was to examine the evidence in the medical literature for data that can guide policy in determining disability in MS patients. Although the literature in general (and certain studies in particular) suffers from limitations, reasonably strong conclusions can still be drawn for most of the seven research questions.
This topic encompassed two questions:
Question 1a: What is the reliability of new McDonald criteria (incorporating supplementary information from radiologic and laboratory studies including MRI, VEP, and CSF analyses) compared with long-term followup diagnosis of clinically definite MS according to the Poser criteria?
Question 1b: What is the inter-rater reliability of diagnosis of MS according to Poser or McDonald criteria among neurologists or between neurologists and non-neurologist physicians?
Analytic approach. Regarding Question 1a, the most important difference between the Poser criteria4 and the new McDonald criteria5 is the addition of MRI findings to the diagnosis of MS, in lieu of the presence of a second attack. Our approach to this question was to identify studies in two categories:
For Question 1b, the relevant diagnostic criteria were the Poser and McDonald criteria plus any other clinical, laboratory, neurological exam, MRI, VEP, CSF, or other data supporting the MS diagnosis. Results had to describe data on agreement or disagreement on the MS diagnosis between evaluating physicians. Agreement statistics could include kappa scores, sensitivity and specificity rates, or other data of the type that could be used to complete a two-by-two table.
Results. The validity of the McDonald criteria is well-supported by two types of evidence:
First, two studies6,7 show that between 73 and 94 percent of patients presenting with CIS who go on to develop clinically definite MS over 1 to 4 years of follow up could be diagnosed with MS according to the McDonald criteria (but would have been undiagnosed under previous Poser criteria). Furthermore, the specificity of the McDonald criteria is reasonably high, ranging from 83 to 87 percent.
Second, many studies8-16 support the MRI component of the McDonald criteria by showing a strong and consistent association between the number of T2 lesions on MRI and the subsequent development of clinically definite MS among patients with CIS or optic neuritis.
Two studies17,18 examined the inter-rater reliability of neurologist-physicians in diagnosing MS according to the Poser criteria; one of these18 also examined inter-rater reliability in diagnosing MS according to the McDonald criteria. We found no data examining inter-rater reliability among non-neurologist clinicians. Overall, there was substantial agreement between observers in classifying MS. Poorer agreement was observed in determining whether a patient had one or more "attacks" of MS and in interpretation of MRI.
Discussion. From the studies identified in the review, the McDonald criteria appear to have substantial evidence for validity and offer the obvious potential advantage of resulting in an earlier diagnosis of MS than the Poser criteria permit. The McDonald criteria have been criticized for their complexity in comparison with previous criteria; however, we found data that demonstrate that these criteria yield a good overall diagnostic reliability, at least as good as the previous Poser criteria. However, data about reliability are available only for neurologists specializing in MS; adoption of the new criteria by clinicians with less expertise could result in deterioration of reliability. Further research on the inter-rater reliability of these criteria in broader clinical settings would be helpful to determine the quality of MS diagnosis.
The research question for this topic was: What clinical indicators, including particularly time-course of impairments, predict physical or mental impairment at 12 months?
Analytic approach. There were four main categories of clinical predictors of particular interest to the analysis:
Our evaluation was limited to those studies with a time course of 12 months (SSA's statutory limit), a timeframe which treating physicians would not ordinarily consider an important decision point. The course of MS has typically been studied over time horizons of many years.
Results. We found relatively little data describing changes in neurological or other impairments over 9 to 24 months; however, we used the data that were available to approximate the 12-month time horizon dictated by statutory requirements. Clinical characteristics have been the best studied, with four reports providing evidence for this review.19-22 Brain23,24 and spinal cord25 MRI have not been shown to be promising. Suggestive evidence is available for laboratory markers26-29 and self-reported quality of life,30 but these indicators will need further study to establish their reliability and utility. While clinical features do not individually provide reliable guidance on prognosis, multivariate predictive models based on relatively easy-to-obtain features may have better performance; such models have not, however, been validated.
Discussion. The ability to predict the future course of MS has been an active area of research; however, most studies examining disease course do so over relatively long time periods (5 to 20 years). The limited predictive ability of some multivariate models has not been validated in populations other than those in which the models were developed; thus, their value for predicting disability has yet to be determined.
Research Question 1a was targeted to current disease-modifying therapies: Among patients with MS, do current disease-modifying treatments result in long-term improvements in physical or mental outcomes compared to placebo or usual care?
Analytic approach. Interventions of interest were all current (2003) disease-modifying immunomodulatory and immunosuppressive treatments. Outcomes of interest were absolute improvements that might result in an individual who is unable to work becoming able to work. The following domains were considered: physical functioning (primarily Expanded Disability Status Score [EDSS]), relapse frequency, cognitive functioning, quality of life, and adverse events.
Results. Most of the data suggest that few patients improve on disease-modifying therapy. Those few who do improve generally do so only in the range of 1.0 point on the EDSS. We found no data regarding improvement in work ability and no data that would correlate a 1.0-point improvement in EDSS with improvement in work ability. The significance of a 1.0-point EDSS improvement varies depending on baseline EDSS score (because the scale is non-linear), but the improvement data available are not generally stratified according to baseline EDSS score. With regard to work ability, the significance of the available data on clinical improvement is unclear. We found no data that quantified individual patient improvement with regard to cognitive function or quality-of-life measures.
Discussion. Our review does not support a conclusion that the current therapies are likely to result in substantial improvement in a significant proportion of patients with MS. This finding is consistent with expert opinion and demonstrated by the design inherent in current clinical trials, that is, the use of lack of decline in EDSS scores as the primary outcome measure. Current therapies are generally regarded as allowing for a modest reduction in progression of MS—particularly in the relapsing-remitting patient population—but are not generally expected to result in significant long-term improvement. Recently, however, combination therapies have begun to be used in the treatment of MS; such combinations of current therapies or new therapies may have greater potential to result in improvements in neurological status.
Symptom management was the focus of Question 3b: Among patients with MS, do treatments aimed at symptom management result in improvements in physical or mental outcomes compared to usual care?
Analytic approach. The effectiveness of symptomatic therapies for spasticity, rehabilitation, urinary management, fatigue, depression, and cognitive impairment was examined. Relevant outcomes were analyzed within six categories:
Results and discussion. Treatment aimed at alleviation of symptomatic manifestations of MS, rather than at the underlying disease, could have an important role in maximizing functioning among people with MS. Among the six areas we investigated, the degree of impairments and the effectiveness of the treatments varied. We found:
In contrast to the previous questions, Question 4 directly linked clinical results with ability to work: Among individuals with MS, what physical, mental, laboratory, or radiographic findings have been associated with inability to work?
Analytic approach. The phrasing of this question predetermined the outcome of interest as ability to work. Findings reported as absolute and relative measures of physical and mental/cognitive function and laboratory and radiographic testing related to work activity were assessed.
Results and discussion. There is a significant gap between what is included in the literature and the type of research evidence required to link objective clinical measures (physical, mental, laboratory, and radiographic findings) with ability to work. Although objective physical and cognitive measures have been developed, their application in the occupational literature is sparse. Furthermore, assessment of how symptoms such as pain and fatigue impact work ability is essentially absent. The reported findings on work ability displayed some consistency across studies. For example, individuals who had higher EDSS levels45,46 or low cognitive function47 were more likely to report not working. However, the strength of association across these studies was not clearly demonstrated, as most reported frequencies or crude estimates of association. Several studies had small sample sizes, which hindered researchers from calculating risk estimates that were adjusted for potential biases such as age, education, level of employer assistance, job type, and desire to work. In addition, most studies considered only physical function or cognitive function, when both can hinder employment.
Similar to the previous question, the focus of Question 5 was the ability to work: Among individuals with MS, how does elevated temperature or other environmental factors impair the capacity to work?
Analytic approach. The evidence sought for this question was on the association of workplace environmental conditions and demands (ambient temperature, individual's body temperature, heat or cold exposure) on the ability of an individual with MS to work. Relative and absolute measures of association were assessed.
Results. With regard to work impairment, limitation, or disability related to temperature conditions, we found remarkably little research that met our inclusion criteria; thus, this question remains mostly unanswered. The one included report confirmed that some MS patients perceive that excessive heat impedes their work capacity.48
Discussion. The evidence provides no basis for generalizations such as maximum appropriate working temperature levels unique to MS patient populations. It is unlikely that medical data in Social Security Disability Insurance (SSDI) application files in the current era will include objective diagnostic test results identifying MS patients who respond adversely to heat challenges. However, subjective patient reports may describe such associations with or without clinician comment or correlation with objective clinical status measures. Although not necessarily founded on randomized controlled trial data, current clinical impression seems to hold that ambient and/or exercise-induced body temperature effects may bear a relationship to MS symptom status in some patients, perhaps more so than is thought to be the case for chronic disease states in general.