Hitting A Moving Target I: Defining and Quantifying CFS: the New Prevalence Estimates by Cort Johnson (2007)

 Reeves, W., Jones, J., Maloney, E., Heim, C., Hoaglin, D., Boneva, R., Morrissey, M. and R. Devlin. 2007. Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural Georgia. Population Health Metrics 5:5

White, P. 2007. How common is chronic fatigue syndrome; how long is a piece of string? Population Health Metrics 5:6 doi:10.1186/1478-7954-5-6

Jason, Leonard. 2007. Problems with the new CDC CFS Prevalence Estimates. IACFS Website. 

Reeves, W., Wagner, D., Nisenbaum, R., Jones, J., Gurbaxani, B., Solomon, L., Papanicolaou, D., Unger, E., Vernon, S. and C. Heim. 2005. Chronic fatigue syndrome – a clinically empirical approach to its definition and study. BMC Medicine 3:19.

Background - We’ve been hearing about this paper for about six months now. We got a little preview of it back in November when for a time CFS prevalence estimates all of a sudden jumped by 400 percent (!). The number was quickly withdrawn but it’s back now and yes, the CDC now estimates that it’s not one but four million adults in the US who have CFS. Throw in the adolescents and kiddies and we’re talking about maybe 7 million people.

Those are some big numbers and they’ve generated some big waves. Within days of its publication it was met with an editorial at the IACFS site (a first), an open response from the CAA (a rare occurrence), and an editorial in the journal it appeared in. The upcoming edition of CAA’s Chronicle will apparently examine this paper in detail. 

Oddly enough, this is not the really important paper; that paper – which established a new definition for CFS -  was published two years ago but the implications of it weren’t really clear. They’re still not perfectly clear but they’re clearer.

A New Definition of CFS – Given there have been at least seven definitions of CFS (the 1988 Holmes, the 1994 International (Fukuda), the 1991 Oxford, the Lloyd, the 2003 Canadian Consensus, the 2007 IACFS Pediatric definition) one might be forgiven for wondering what’s the big deal?  The big deal is that it’s the CDC that’s doing it and historically the CDC definitions have stuck. Except in the UK almost all CFS research studies use the CDC sponsored 1994 International definition to determine who they will study. Since major definitional changes in CFS do not happen often - the last definition lasted for 13 years – this one could alter the face of CFS research for another decade or so.

Why be worried about a definition of CFS? Knowledgeable CFS physicians don’t really need a definition; they know CFS when they see it. But researchers and others need a definition they can rely on to separate CFS patients from others. Researchers prove their cases by using statistical tests to compare the differences between a sick group and a healthy group. If CFS isn’t defined properly and study groups contain both CFS patients and other types of patients then neither type will be well elucidated. Instead of finding clear differences the studies will get a mishmash of results – sometimes significant (positive), sometimes not – a difficulty that has permeated some areas of CFS research. Some researchers believe that progress in CFS research will be slow and plodding until researchers find a way to produce to coherent sample sets, which requires first an adequate definition.

A Culmination. This paper does not stand alone; it is a culmination of several years of efforts by the CDC to do two things; (a) assess the effectiveness of the Fukuda definition of CFS and (b) to try to differentiate CFS patients from people with unexplained fatigue.

The first undertaking was apparently prompted by the difficulty many CFS researchers have had replicating their findings  - a problem that has slowed CFS research efforts and impaired CFS’s  search for legitimacy. The CDC believes that this may be due to something called ‘referral bias’ which occurs when different types of patients predominate in different studies. They believe that the vagueness of the Fukuda definition allows this to happen.

An Important Recommendation. In 2001 the CDC brought together an international consortium of researchers, physicians and CFS experts (ICFSWG) to advise them how the definition should be altered (click here to see group). That group met three times and announced its findings in a 2003 paper. The ICFSWG and the CDC came to three major conclusions:

(1) The Current Definition of CFS is Fatally Flawed: Comparison studies indicated that the International definition does a poor job in picking out the more severely ill ‘CFS patients’ and that the patients it does pick out often don’t qualify for CFS for long. Ultimately Dr. Reeves came to have so little faith in the International definition that he said the current practice of simply stating that individuals meet the 1994 CDC definition is so vague that ‘it is essentially impossible to compare results between studies critically’.  

(2) Symptoms Cannot Be Used to Define CFS: Two studies, one by the CDC (Nisenbaum et. al. 2004) and an independent one (Sullivan et. al. 2005), concluded that the symptoms CFS patients experience are not markedly different from those of patients with unexplained fatigue. The authors of the CDC study noted that in examining the symptom question they were taking on ‘one of the most controversial aspects of defining CFS”. Their study results, however, indicated that ‘CFS is multi-dimensional and that it overlaps with other dimensions of unexplained chronic fatigue”. They concluded that ‘it might not be possible to use symptoms to define CFS’.

(3) CFS Should  Be Defined Using Measures of Fatigue, Disability and Symptom Severity. The ICFSWG proposed using standard measures of fatigue, disability and symptom severity to define CFS. This meant that once all other diseases that could cause a CFS-like condition are excluded a CFS patient would simply be defined by how fatigued, disabled and symptomatic they are. The  ICFSWG also produced the first complete list of exclusionary disorders and conditions.

Benefits: This type of definition would have the considerable benefit of allowing researchers and physicians to really characterize the kind of symptoms and disability present in CFS. This should allow them to create a baseline they can use to determine what kinds of patients they have and how they respond to treatments. It could help break CFS patients into their natural subsets – something that could provide immense help to CFS researchers. Combined with laboratory findings, this new process could determine if certain symptoms such as post-exertional fatigue are associated with certain laboratory abnormalities or if low NK cell function is associated with certain kinds of fatigue. This aspect of the definition is clearly a major step forward.

 CREATING THE DEFINITION

The Reeves group took those findings and in 2005 took the crucial step of translating them into new criteria, i.e. of determining just how disabled, fatigued and symptomatic a person had to be to be deemed to have CFS. They used part of one of the tests recommended by the ICWG, the Medical Outcomes Short Form (SF-36), and added two more,  the Multidimensional Fatigue Inventory (MFI) and the CDC Symptom Inventory.

These tests were chosen because they are widely available and because their normative values have been well established. This means researchers and physicians using these tests can easily tell how impaired CFS patients are compared to patients with other diseases.

The Multidimensional Fatigue Inventory (MFI) assesses the degrees of the following kinds of fatigue present; general fatigue, physical fatigue, mental fatigue, reduced motivation and reduced activity.

The Medical Outcomes Short Form (SF-36) assesses the degree of the following kinds of disability; Physical function, role physical, bodily pain, general health, vitality, social function, role emotional, mental health

The CDC Symptom Inventory – measures the frequency and intensity of the symptoms associated with CFS by the 1994 Fukuda definition (post-exertional fatigue, sore throat, swollen lymph nodes, muscle pain, joint pain, poor concentration, unrefreshing sleep, headaches) plus others (diarrhea, fever, chills, sleeping problems, abdominal pain, sinus/nasal problems, shortness of breath, sensitivity to light, depression). 

 

The New Criteria

 This group created a new definition of CFS by creating cutoff points that ensured, the authors said, that CFS patients were quite disabled; the SF-36 scores were at about 25% of norms or below those associated with individuals who have congestive heart failure.

They determined that to meet the criteria for CFS one has to score in the 25th percentile or below of the established norms on ONE of the following sections of the SF-36:

  •         Role physical

  •         Physical function

  •         Social function

  •         Role emotional

AND score greater than 13 or 10 respectively on ONE the following scales of the MFI:

  •        General fatigue

  •        Reduced activity

 AND one has to have 4 symptoms and score greater than 25 on the Symptom Inventory. People who failed to meet one or more of the cutoff points would be placed in the ‘insufficient symptoms or fatigue category’ (ISF).

CFS Prevalence The CDC then applied these criteria to a new kind of intensive random telephone sampling scheme of people in urban, metropolitan and rural Georgia. Instead of asking whether someone in the household was fatigued they asked if there was someone who was ‘unwell’. Dr. Reeves explained, ‘as we have learned more, we understood that fatigue is not the only thing that bothers people with CFS. People also report, “I hurt, can’t think clearly, feel crappy when I wake up”. So we changed the screening algorithm to get people who are ‘unwell’. It turns out that a substantial number of those with cognition problems have CFS. We cast a bigger and more sensitive net and got a larger number’ (CFSAC Meeting July ’06) They called almost 11,000 numbers and did detailed interviews with people who had a variety of types of unwellness.

At the end they got 292 CFS patients (according to the old definition), 268 chronically unwell patients, 60 people with short-term unwellness and 163 well people to come to the clinic for an examination. At the clinic they examined them more closely, did laboratory tests to determine if they had any diseases that precluded them from being in a CFS research study and then recategorized them using the empirical definition. They analyzed them and came up with new estimates of CFS prevalence.

A New Kind of CFS Patient: It’s been clear for quite some time that this new definition would uncover a new kind of CFS patient. Two years ago CDC researchers found the Empirical Definition increased the number of CFS patients almost three-fold. But it doesn’t simply add patients to the mix it also appears to kick a substantial number of ‘CFS patients’ out of it. According to table two of the present paper about 45% of people formerly classified during a telephone interview as having CFS under the old definition would not have ‘CFS’ under this new definition. Indeed the 2005 Empirical Definition study found ‘little agreement’ between the groups of patients identified by the Fukuda and ED criteria.

A Sicker Group! The patients singled out by the empirical definition were clearly quite ill. The Reeves group noted that this group had “lower scores on all scales except physical function and general health than patients with congestive heart failure.”

Surprisingly (given the increased prevalence figures) the new definition defines a sicker group of people than does the old Fukuda (International) definition. Statistical tests using all eight SF-36, 5 MFI tests and 19 symptom test scores indicated that in general that the most ill patients usually ended up being placed in the CFS category, the next most ill were in the CFS-like category and so on. This was a distinct improvement over the International definition.

Defining CFS or Something Else? But were they CFS patients? Despite the Reeves group success at identifying a sicker group that displays more consistency over time with regards to their illness than the old definition, several researchers have raised substantial questions regarding the new definition.

Dr.Jason, a well-known CFS researcher noted that “It is very possible that this new empirical classification does identify a group of individuals with high levels of fatigue, impairment and symptoms but” and here’s the rub ‘ it might also be identifying a group with high chronic distress and illness rather than CFS as a unique disorder”.   

FOUR QUESTIONS. Four main questions have been raised:

  1. Should emotional disability be given the same weight as physical or social disability in a CFS definition?

  2. Should low activity levels (apart from fatigue) be used to define CFS?

  3.  Is a low (median) cutoff point for fatigue/activity suitable?

  4. Are the symptom criteria set high enough? 

Along with Dr. Jason, two reviewers, Dr. White and Dr. Lloyd, raised these questions in their prepublication comments. 

1. The Emotional Subscale Used is Inappropriate. This definition allows people who believe their distress to be primarily due to  emotional causes to qualify for one of the three major  criteria for CFS. Both White and Jason argue that the emotional subscale should not have been included. Historically, emotional components have not played a major role in defining CFS.

Dr. Jason pointed out that CFS patients typically rate themselves very poorly in ‘physical functioning’ but under this  scheme someone could score fine in ‘physical functioning’ but qualify for CFS by scoring poorly in the emotional category. Dr White, a psychiatrist, pointed out that the CDC’s own studies indicate that ‘role emotional’ does not appear to play a prominent role in CFS. Several other studies have failed to find a strong emotional component to CFS.

Dr. Jason and Dr. White also noted how similar the prevalence figures under the new definition are to the prevalence figures found in the UK using a definition (Oxford) that appears to select for patients with increased rates of psychological disorders. Oxford study patients also have  decreased rates of post-exertional fatigue (63%) and sleep problems (64%). Since these are core components of the Canadian Consensus definition at least a third of these patients would not qualify for CFS under its criteria. It would be difficult to uncover the cause of post-exertional malaise in a study group with the Oxford makeup.

Rationale: Why include the emotional subscale when your own tests do not suggest it is central in CFS? When prodded by Dr. White, Dr. Reeves said ‘we included the emotional subscale to capture the relation between the functional emotional impairment and reduced social and personal activities”. This suggests Dr. Reeves wants to capture that set of unwell people whose emotional problems impair them from partaking in social and personal activities.

Could Dr. Reeves be right in his sense that the emotional component has been underemphasized in CFS? One can hardly argue that CFS does not have an emotional effect; rates of mood disorder do, after all, soar as people come down with CFS and few with CFS would probably argue that they are, physical problems aside, the same person emotionally they were prior to coming down with CFS.  The early ME studies again and again emphasize the emotional turmoil inherent in that disease; a turmoil that appears to go far beyond that associated simply with being ill. While the Canadian Consensus Definition does not make emotional problems a major criterion it does include ‘emotional overload’ in one category and notes the panic, anxiety and emotional problems that can occur in CFS.

An emotional component to CFS would not be surprising given what we are learning about the central nervous system abnormalities in CFS. They appear to be centered in areas of the brain involved in both mood regulation and cognition (as well as autonomic nervous system functioning, metabolism, fatigue and pain). Given that, it might be surprising if, along with cognitive and physical problems, CFS wasn’t also associated with emotional difficulties. Throw in the increasing evidence that infection may be at the heart of some cases of mood disorder and one can easily visualize a scenario that satisfies everyone sitting at the CFS table. In the end it may be that every one, the psychiatrists, the immunologists, the neurologists, etc., will be at least partially right about CFS.

The question, of course, is where do you draw the line? Simply including an emotional component should not, of itself, not dramatically alter the makeup of CFS patients.

By giving emotional distress the ability to fulfill one of the three legs of the new CFS definition, the new definition does, however, elevate the role ‘emotion’ potentially plays to a new level. There is a physicality inherent in CFS that has lead CFS patients to tenaciously assert an organic cause to their disease. Depressed or anxiety disorder patients appear to more readily accept their diagnosis but the physical constraints play a far stronger role in ME/CFS. Clinicians again and again remark on how avid their patients are to get well.

2. Fatigue is not a necessary component of the definition. In the end Dr. White did not believe the emotional subscale question was the major issue. Both he and Dr. Lloyd, the other prepublication reviewer, felt a more important issue involved the fatigue ratings.

Because CFS patients must either report general fatigue or reduced activity this is the first definition of CFS that does not explicitly require that severe fatigue be present. In his review White states “this means it would be possible to meet the fatigue criterion without significant fatigue”. Jason reports this opens the door for people with reduced activity levels not due to fatigue but to emotional problems (depression) to meet the criteria for CFS.

3. The cutoff points for fatigue and activity were not low enough – In order to qualify for being highly fatigued or very inactive a patient simply had to score at or below the median. (The median point is the middle point in a gradient of numbers)

Why define a disease that has historically been characterized, rightly or wrongly, by fatigue, using a median cutoff point?. The Reeves group stated they based their conception of fatigue on the 1994 International definition but it’s hard to believe the authors of that document had in mind a median level of fatigue when they stated “in our conception of the chronic fatigue syndrome, the symptom of fatigue refers to severe mental and physical exhaustion”.

When the editors of the Journal of Population Health Metrics questioned this cutoff point, Dr. Reeves noted that CFS-like patients had to report that they were “were fatigued, had been fatigued for at least six months and that rest did not relieve their fatigue” and that “Fatigue was mandatory for the diagnosis of CFS in this study.”

Yet it’s hard to understand how this could be so. The interview process asked about unwellness not fatigue. Once the different groups (including the non-fatigued group) were brought together and re-classified according to the Empirical Definition criteria, fatigue was no longer a necessary component. According to one table approximately 15% of the ‘new’ CFS patients reported they did not have problems with fatigue during the telephone interview - an amazing finding for a disease called chronic fatigue syndorme.

4. The cutoff points for the symptom severity are not high enough – Dr. Jason pointed out that someone could meet the criteria for CFS who reported that only two of eight symptoms (fatigue, muscle/joint pain, headache, sore throat,  unrefreshing sleep, swollen lymph nodes, problems concentrating), one of moderate and the other of severe severity, were present all the time.

Counterpoint: Several aspects of the definition suggest that the projected flood of very emotionally distressed people into CFS studies may not materialize. The exclusion of patients who have had major depression in the past five years from participating in CFS research studies should filter out a good number of those patients. (The CDC has not been consistent in this matter; the Wichita CDC studies included them, the Empirical definition paper did include them in one part of the study but isolated them, this latest paper excluded them altogether.)

The report that the scores for the different categories are all correlated suggests that people with emotional problems but not physical problems or people with low activity levels but not fatigue problems were not often seen. Thus, while theoretically possible, in practice the definition may select few if any people with purely emotional problems.

THE FUTURE

A Depressing Future? But at the 2007 IACFS conference Dr. Jason reported that a preliminary study found that 40% of patients with major depression qualified for CFS under the new definition. This, of course, suggests that future CFS studies by the CDC could have a greatly increased component of emotionally distressed patients. Dr. Jason suggested that this has already occurred; the recent Heim study that employed the empirical definition found that 62% of the CFS patients had evidence of early childhood abuse of one type or another. This contrasted with a study employing the Fukuda criteria that did not find such high levels of childhood abuse.

This is, of course, the crux of the matter; if the new definition allows a significant number of people who don’t have CFS but do have emotional or other problems into research studies, then focus of CFS research will change dramatically. One can argue that the vagueness of the 1994 definition probably allowed a subset of people with mood disorders or other diseases but not CFS into research studies and that this is one reason for the inconsistent findings that have plagued some aspects of the CFS research for the past 13 years.

More on the New Definition. See the Hitting A Moving Target II for more short essays on the new definition

  • Another ‘Smoke-Filled Room’ - examines the conditions under which the new definition was created.

  • Testing the Definition: the Early Results – some worry that the new definition will lead CFS research to take a more psychological bent. This article examines if it has thus far.

  • Increased Prevalence Rates – What Increased Prevalence Rates?– examines if it was the definition or some other aspect of the study that resulted in the increased prevalence rates.

  • Metrosexual Problems in Georgia – does the greatly increased female prevalence rates in the metropolitan area (12x’s the men) suggest a flaw in the study?

  • A Self Correcting Problem? – are the tools to fix the definition – if it needs fixing –  included in the definition itself. 

  • An Inevitable Conclusion – questions whether the discounting of post exertional malaise in the International (Fukuda) definition in 1994 made a definition based on unwellness an inevitability.