Oversight of Transplant Center Performance Can Impact Patients and Care Providers

/kidneynews/1_3/12/graphic/12f1.jpg

The United States has the highest per capita cost of medical care in the world—medical care consumes 17 percent of the gross domestic product. Yet the United States ranks far from the top in most measures of health.

In recent years, government, insurance companies, business groups, and others have placed great emphasis on improving the quality of medical care because of the perception that physicians, hospitals, and others have done an insufficient job of improving outcomes and reducing costs. These groups want to control the ever-increasing costs and improve the quality of medical care.

Imagine you’re the director of a kidney transplant program. Representatives from the Centers for Medicare and Medicaid Services (CMS) recently conducted a site visit (they are visiting every transplant program in the country). Your program might be in danger of being closed because the outcomes for one-year grafts and patient survival are below expectations.

/kidneynews/1_3/12/graphic/12f2.jpg

A thorough self-examination of your program fails to find any systematic deficiencies. You have good personnel, the hospital supports the program adequately, the follow-up care is good, and your program uses commonly accepted immunosuppression and the latest protocols for evaluating and following your patients. You are even recognized as an innovative center with experimental protocols that bring the advantages of kidney transplantation to patients who have been turned down by other transplant centers.

As if the visit by CMS were not enough, your transplant administrator received a letter from an insurance company that refers many transplant candidates to your center. It does not want to renew its contract with your center because of the below expected results.

Why are your results below expectations? Does your program transplant riskier patients than other programs? The Scientific Registry of Transplant Recipients (SRTR) calculates risk-adjusted outcomes that are used by CMS and insurance companies, so even though you think your patients might be high-risk, that has presumably been taken into account by the risk-adjustment model.

Many initiatives have been instituted to improve the quality of care. Some are joint efforts by several groups. Among these initiatives are the National Quality Forum, the National Healthcare Quality Report (Agency for Health Research and Quality), National Quality Measures Clearinghouse, National Surgical Quality Initiative Program, and the Surgical Care Improvement Project. Other groups concerned with quality are the Leapfrog Group, the Integrated Healthcare Association, Hospital Quality Initiative (CMS), Hospital Quality Alliance, American Medical Association-Physician Consortium for Performance Improvement, Ambulatory Care Quality Alliance, Institute for Healthcare Improvement, and the Surgical Quality Alliance (American College of Surgeons).

The Organ Procurement and Transplant Network (OPTN), using SRTR data, and CMS oversee quality in transplantation. These initiatives may establish performance measures or “report cards” for physicians and hospitals.

Performance measures can have profound effects on medical care and providers. The goal, of course, is to improve the quality of care and to reduce costs—to achieve better value. Physicians and hospitals that do not meet certain performance measures can be denied insurance contracts, can have their poor performance made available to the public, and can even be closed. Certainly, no one can deny that we should all have a goal of improving patient outcomes, the quality of medical care, and reducing costs.

Risks to risk adjustment

In order to account for differing patient characteristics, models that measure performance adjust for risk. Risk adjustment is a statistical technique using patient variables to make comparisons valid. It is supposed to level the playing field. But there are risks to risk adjustment. Models may omit important variables, data collection may be incomplete, or data collection forms may not be filled out correctly. Even the models used for risk adjustment can vary, so that a provider may meet performance measures calculated with one model but not with another (1).

Every transplant center in the United States must report the results of every transplant to the SRTR. This reporting includes many patient and donor characteristics. Using a Cox regression model, the SRTR calculates the graft and patient survival for each organ for each transplant center. The SRTR also calculates the expected outcomes using national donor and patient characteristics.

The SRTR uses a two-sided t-test to compare the transplant center’s observed outcomes with the expected outcomes. CMS, on the other hand, uses a one-sided t-test for comparison. A transplant center’s outcome is more likely to fall below the expected outcome using a one-sided t-test and is thus more likely to be flagged as a poor performer. Because of the way in which outcomes are calculated, there will always be some transplant centers that have graft or patient survivals below expected results. Therefore, it is possible that CMS could close some transplant centers each time it has a round of transplant center evaluations.

The SRTR risk stratification model

Even though the SRTR employs excellent statisticians and uses numerous donor and patient variables that are submitted by the transplant centers themselves, its determination of poor center performance has been criticized. Despite the approximately 55 variables the SRTR model uses, the model does not include some important determinants of patient outcome, such as the degree of cardiac or coronary disease and social class, although it would be difficult to get such data for all patients.

Other important patient characteristics may also be missed in the SRTR model, including presence of other disease states, genetics, patient support networks, and subjective characteristics of patients that experienced caregivers may recognize to be important for patient compliance and overall prognosis. If the OPTN required that transplant centers collect and provide this information, the SRTR would be able to place cardiac and social class variables into its model.

The potential impact of factors not known is reflected in the relatively low predictive values of models in kidney transplantation. The c-statistic, a measure of model discrimination, is significantly lower in transplantation than for other disease outcomes. The c-statistic varies from 0.5, which is no predictive ability, to 1.0 for a model that is 100 percent predictive. For kidney transplant survival statistics, the c-statistic is 0.67 for graft survival and 0.72 for patient survival, generally regarded as quite low. This number compares to 0.86 for deaths after myocardial infarction, 0.83 for coronary artery bypass grafting, 0.85 for pneumonia, and 0.87 for stroke (1).

This low number for the predictive ability of the model for transplant outcomes strongly suggests there is a lot that we do not understand about determinants of transplant outcomes. In fact, transplantation models fail to account for the reasons the majority of grafts fail or patients die (2). Thus transplant outcomes and performance evaluations are related in a significant way to factors that may not reflect quality of care. Furthermore, government regulators, insurance companies, patients, and even transplant professionals may assume that these assessments are completely reliable and interpret them accordingly (2).

It is difficult to argue with the concept of evaluating transplant center performance, but the evaluation tool should be accurate and reliable. The complex nature of transplantation renders quality assessments problematic. We must work to ensure that the goal of high quality for all patients is not compromised by the assessment tools used to evaluate quality (2).

What transplant centers can do

Given that measures of poor performance—whether valid or not—may lead to severe outcomes for the transplant program, either elimination of insurance contracts or even closure of the program, what can transplant programs do to protect themselves?

First, transplant centers should ensure that there are no problematic systems issues. Do they have the right personnel who can select appropriate candidates, perform the surgery, and manage the patients after transplantation with sufficient skill? Are the protocols for immunosuppression and other medications, candidate evaluation, and posttransplant follow-up appropriate? Does the hospital provide sufficient support for the transplant program with staff, facilities, and laboratory and radiological availability?

If the transplant center is convinced there are no systems issues, one of the few remaining choices is to try to select transplant patients who are more likely to have successful outcomes. Despite the SRTR’s risk adjustments, many transplant surgeons believe that the model nevertheless does not control for important variables. For example, one of the risk-adjustment parameters is age. However, the model includes all patients who are 65 years old and older in one group. Most transplant surgeons probably believe that patients in the 70-year-old age group are at higher risk of dying than patients between 65 and 69 years old and may choose to not transplant patients in this age group. In fact, this is already happening. Some transplant centers have decreased the age of acceptable transplant candidates.

Cardiac disease is generally regarded as one of the most important risk factors for patients undergoing kidney transplantation. Certainly, the risk of a 60-year-old man with no history of heart problems, a normal cardiac stress test, and no coronary artery disease is much less than that of a 60-year-old man who has had two myocardial infarctions and four-vessel coronary artery bypass grafting. Yet the SRTR model does not include cardiac disease in its risk adjustment model.

Social class (generally measured by education level and income) is an important determinant of outcome (3). The SRTR does not include measures of social class—other than whether or not the patient has private insurance—in its risk adjustment model. This is not to be critical of the SRTR. After all, it can only include the data it has in its model.

Many transplant centers are undertaking innovative procedures and techniques to transplant certain patients who might not otherwise have an opportunity for transplantation. Other centers may have experimental protocols testing new drug regimens or other treatment protocols that may pose higher risk for their patients. But because of possible CMS sanctions, some transplant centers may be forced to restrict the introduction of new and innovative treatment regimens—regimens that could ultimately improve the outcomes of transplantation.

Ethical issues a concern

Transplant centers that might receive poor performance scorecards and possibly have insurance contracts withheld or be threatened with closure by CMS will likely try to reduce these possibilities by transplanting only patients they perceive as having a lower risk of graft loss or death. This is already happening at many transplant centers.

Personnel at these centers are not convinced that the SRTR risk adjustment model adequately adjusts for risk and believe that their patients are different from centers that have better outcomes. We have shown that transplant centers with the highest candidate mortality rates have lower kidney transplant graft and patient survival after transplantation, even after risk adjustment (4).

Ethical issues are raised by the possible refusal of transplant centers to transplant certain patients who would do better with a transplant than without one, but who might have comparatively poor outcomes and thus threaten the transplant center with losing insurance contracts or even with closure.

The culture of medicine dictates that the physician put the interests of the patient before other interests. The transplant candidate comes to the transplant center with the understanding that the best clinical decision will be made for her. Transplantation increases the longevity of patients with renal failure compared to dialysis, and the quality of life is also greater with a transplant. Yet transplant centers may refuse to list and transplant some patients perceived to be at high risk in order to preserve their very existence.

Another strategy transplant centers could use to improve their results is to avoid using kidneys that are thought to be associated with lower graft survivals. Although the SRTR risk-adjustment model incorporates many donor characteristics, some, such as biopsy findings, are not accounted for. Transplant centers may also feel other important donor variables are not included in the risk-adjustment model. This refusal to use some kidneys might result in fewer patients being transplanted.

Almost certainly no program wants to be closed or rendered nonviable because of a loss of insured patients. Furthermore, if the program is closed, it may be that other potential transplant recipients will either have to travel great distances to receive transplants or not receive access to transplantation at all. This outcome particularly threatens the poor, who may find the additional costs of travel beyond their means.

Strategies for preserving the transplant center, then, will have the effect of limiting access to care for some patients. And that may not necessarily be entirely bad. Such strategies may result in kidneys being transplanted into individuals who may live longer. Thus life-years after transplantation may be increased. In fact, the United Network for Organ Sharing is currently considering instituting a new allocation system that does just that by favoring younger recipients in the allocation of kidneys.

But if transplant centers also restrict the kidneys they are willing to transplant to only those they believe to be the best quality, the unintended outcome may be that fewer transplants will be performed.

This trend may already be happening. In 2007, for the first time ever, the number of deceased organ transplants decreased in the United States by 1 percent, even though the number of deceased donors increased by 0.8 percent. It appears that the trend for a reduced number of deceased donor transplants will hold for 2008 as well.

References

1. 

Iezzoni LI: The risks of risk adjustment. J Am Med Assoc 1997; 278:1600–1607.

2. 

Schold JD, Howard RJ: Prediction models assessing transplant center performance: can a little knowledge be a dangerous thing? Am J Transpl 2006; 6:245–246.

3. 

Howard RJ: The challenging triangle: Balancing outcomes, transplant numbers, and costs. Am J Transpl 2007; 7:2443–2445.

4. 

Schold JD, Srinivas TR, Howard RJ, Jamieson IR, Meier-Kriesche HU: The association of candidate mortality rates with kidney transplant outcomes and center performance evaluations. Transpl 2008; 85:1–3.