Here are answers to some fundamental questions about sampling and extrapolation, which are key to auditing and overpayment findings (see story, p. 1).[1] They were provided by auditor/statistician Bruce Truitt, a former faculty member of the Medicaid Integrity Institute in Columbia, South Carolina, who said these concepts are often misunderstood. Contact him at brucetruitt@gmail.com.
PROGRAM INTEGRITY SAMPLING AND EXTRAPOLATION CONVENTIONAL WISDOM IS AT TIMES NOT SO WISE
Here we explore some “conventional wisdom” and indicate areas that have been addressed in specific cited cases.
Can Statistical Sampling Be Used?
The seminal case is Chaves County Home Health Servs. v. Sullivan, in which the court judicially approved sample adjudication despite the lack of specific reference to it in the Medicare Act. See also Michigan Dept. of Educ. v. United States, Mile High Therapy Centers, Inc. v. Bowen, United States v. Smushkevich, Illinois Physicians Union v. Miller.
In Ratanasen v. Cal. Dept. of Health Servs., the court rejected the objection that reaching a true overpayment required examining each file on its own and found that a simple random sampling approach to calculating liability was valid.
In Georgia v. Califano, the court found that “Projections of the nature of a large population through review of a small number of its components has been recognized as a valid audit technique and approved by federal courts,” citing New Jersey Welfare Rights Organization v. Cahill and Rosado v. Wyman. Similar ruling arose in United States v. DeCosmo.
The Georgia case also concluded that “Audit on an individual claim-by-claim basis of the many thousands of claims submitted each month by each state would be a practical impossibility as well as unnecessary.”
Is the Medicare Program Integrity Manual Binding?
Guidance and Memoranda apply but are not binding. “ALJs [administrative law judges] and the MAC [Medicare administrative contractor] are not bound by Local Coverage Determinations, Local Medical Review Policies, or CMS program guidance, such as program memoranda, and manual instructions, but will give substantial deference to these policies if they are applicable to a particular case” (42 C.F.R. § 405.1062-1063).
Does the Administrative Law Judge Have to Consider the Entire Sample?
Yes, the Administrative Law Judge must consider the entire example. “When an appeal from the QIC involves an overpayment issue and the QIC used a statistical sample in reaching its reconsideration, the ALJ must base his or her decision on a review of the entire statistical sample used” (42 C.F.R. § 405.1064).
Do ‘Generally Accepted Statistical Principles and Procedures’ Exist?
The courts have not adopted specific methodological guidelines. The Medicare Appeals Council and Federal Courts have held that there is no formal recognition of “generally accepted statistical principles and procedures” (Michael King, M.D. and Kinston Medical Specialists, P.A. Cigna Government Services Claim for Part B Benefits, Alpine Home Care, Cahaba Government Benefit Administrators Claim for Part A Benefits, and Pruchniewski v. Leavitt).
How Important Is Sample Size or a Sample’s Percentage of the Population?
The courts have held that no statistical floor for sample size exists (Webb v. Shalala) and that sampling a percentage of the population is irrelevant, noting use of a .4% sample in (Michigan
Dept. of Education). Moreover, in Pruchniewski v. Leavitt, the judge rejected plaintiff arguments that a sample size of 30 was too small to be reliable, that a sample size of 320 was necessary, and that a sample of 320 would have produced an estimated overpayment that was below the lower limit of the 90% confidence level calculated by the carrier.
Courts have further noted that confidence intervals account for imprecision from a smaller sample size (Border Ambulance Service, LLC TrailBlazer Health Enterprises Claim for Part B Benefits and Transyd Enterprises LLC Trailblazer Health Enterprises LLC Claim for Part B Benefits). American Institute of Certified Public Accountants (AICPA) adds that all random samples are “representative” and that representativeness relates to selection method and has nothing to do with sample size.
How Important Is Precision?
The MAC and Federal courts have hesitated to set aside statistical sampling and extrapolation in response to claims that the overpayment was imprecise without also showing that it was arbitrary and capricious, especially demand amounts are at the lower confidence limit (Foot and Ankle Associations of NC, PLLC, AdvanceMed Claim for Part B Benefits, John Sanders, M.D. CIGNA Government Services Claim for Part B Benefits, John v. Sebelius, and Pruchniewski v. Leavitt).
Must Population and Sample Means or Proportions Be Statistically the Same?
State of New York v. Rite Aid of New York, Inc. found that failure to match the population dollar mean and sample dollar mean is neither necessary nor likely in a valid estimate. Also rejected was comparing the proportion of patients-to-claims in the sample to the proportion of patients-to-claims in the universe.
Is Stratification Required?
While CMS (and AICPA) recognize that stratification may afford greater precision, the MAC and courts have held that failure to stratify does not disqualify the results, especially absent demonstration that a different stratification would have made a significant difference in the overpayment estimation (Diana Carneal, OTR, D/B/A The Muscle Manager Western Integrity Center (PSC) Claim for Part B Benefits, Pruchniewski v. Leavitt, and Ratanasen v. State of California). In HCA v. Kansas, failure to stratify was ruled not to affect the reliability of the sample. State of New York v. Rite Aid of New York, Inc. noted that, while stratification might give a more precise estimate, it is not required for a valid estimate.
