“It’s a model, that means it’s not real.”  S.T. Rao 

1.       The Meaning of Scientific Reliability as it Relates to Modeling 

           Environmental professionals using contaminant transport equations to determine causation often repeat the hoary phrase that their opinions are reliable to a “reasonable degree of scientific certainty.”  The significance of this phrase is questionable unless the underlying opinion can be validated by statistical analysis. It is generally accepted in the statistical science community that results can be considered scientifically accurate (reliable) when at a 95% “confidence interval.” Typically, computer modeling occurs without the aid of statistical analysis. How then can the results of a computer simulation of contaminant transport in groundwater satisfy the meaning of “scientifically reliable”? Does the meaning of “scientifically reliable” when applied to modeling become murky when there is no meaningful attempt at quantification? Finally, why should the use of “scientifically reliable “be any less rigorously applied to opinions in environmental cases than to cases involving expert opinions about medical causation.

             Often, these questions are not the focus of the inquiry about the reliability of the opinion. Instead judges have a tendency to allow the use of groundwater flow models so long as they are “generally accepted by the environmental science community” and thus can pass muster under the evidential standard of Daubert (and analogous state court cases) which was intended to prevent the proliferation of opinions based upon questionable (junk) science. Since most groundwater flow models have been extensively peer reviewed, a court is unlikely to sustain an objection to an opinion based on a recognized model.

             Under Federal Evidence Rule 702 (and similar state law requirements), the court must consider whether or not a scientific opinion will assist the trier of fact. It would seem obvious that the trier of fact has a need to know how accurate a computer simulation is in order to be aided in its deliberations. Therefore, the proponent of an opinion using a computer simulation should be required to determine for example the effect of human measuring error and variations in local conditions upon such variables as soil characteristics and groundwater flow directions in order for that opinion to be judicially recognized as scientifically reliable.  

2.         The Reverse Modeling Conundrum 

            The scientific reliability of computer simulations becomes extremely important when groundwater fate and transport models using historical data on the spread of contamination are used in reverse to identify the date of the release.  One researcher described the problems inherent in reverse modeling thusly: “Finding the source location and the time history of the solute in groundwater can be categorized as a problem of time inversion.  This means that we have to solve the governing equations backward in time.  Modeling contaminant transport using reverse time is an ill-posed problem since the process, being dispersive, is irreversible.  Because of this, the solutions have discontinuous dependence on data and are sensitive to errors in the data.”[i] Because there are no hydrological footprints leading back to the origin of the spill, the results of the simulation are highly dependent on the accuracy of the current data. In addition, the limitations inherent in the model and the ability of the user to adapt it to site specific conditions need to be recognized in achieving scientific reliability.   

 3.        Reverse Modeling Expertise  

     Because the results of reverse modeling are so susceptible to errors from many sources, special attention needs to be paid to the qualifications of the model user. Groundwater modeling is a specialized field within the science of hydrogeology.  “All models require the talents of a skilled model user, a tailor, to design hydrogeologically valid boundaries and initial conditions and select meaningful values for the model parameters.”[ii]  “The application of a groundwater model to a field problem is characterized by many uncertainties and…mistakes in model application are common.”[iii]  It is therefore the generally accepted position in the scientific community that the user of a groundwater flow model must be skilled in its use in order to produce valid simulations of groundwater flow conditions and patterns.

        Educational institutions have trained many talented individuals in the environmental sciences who have become experts in soil and groundwater investigation and remediation. Groundwater modeling may even have been part of their curriculum. Education and experience in soil and groundwater contamination and remediation alone does not make one a qualified groundwater modeling expert. While courts have recognized either education or experience as the basis for qualifying a witness as an expert, a potential expert in a highly specialized field such as reverse computer modeling of groundwater flow would likely be required to have both.           

4.         Model Selection 

Selection of the model most suitable for the site conditions and the intended purpose is equally important in achieving defensible results. The recognized categories of groundwater models include screening level models and aquifer simulators.  Models can also be obtained according to the type of contaminant being spread such as gasoline/hydrocarbons and chlorinated solvents. These can further be classified as either analytical models or numerical models.     

Analytical models rely upon simplifying assumptions about the solute transport system in order to determine the future spread of the contaminant. An analytical model such as BIOCHLOR can only be used where the data establishes that there is a uniform groundwater flow direction. An analytical model would not be used if there are complex flow patterns, boundary conditions, vertical movement, tidal influences, differential natural attenuation rates, and other variables which are non uniform.  Practically, the lack of data to support the simplifying assumptions should be the basis to exclude an analytical model in many real life situations unless the results are non-critical. Even when the simplifying assumptions can be supported, the model still requires sound professional judgment and experience in its application to field situations. The advantages of analytical models are that they are easy to use and do not require extensive data sets. They are therefore most often applied in remediation studies not in reverse modeling or age dating situations.   

Numerical models are more complex than analytical models, require much more site data but involve far fewer simplifying assumptions to use. They can be used to simulate more complex hydrogeologic systems or contaminant transport being affected by multiple reactions for which rates or properties may vary spatially.  Numerical models can also be used to model heterogeneous and anisotropic hydrogeologic systems as well as situations of transient flow such as hydrogeological formations under stress or boundary conditions that are controlling groundwater flow change over time.  Numerical model codes are also more flexible in allowing for the simulation of multiple scenarios. In general they can provide more accurate results than an analytical model. 

              In a cost recovery action that asserts liability for prior discharge, validation of the model becomes the next threshold issue after the qualifications of the user. Any opinion based on computer simulation should contain extremely detailed and reliably accurate results that closely match site conditions in order to satisfy a Daubert type of review.  The lack of sufficient data should not preclude the use of a full-blown numerical aquifer simulator model which would be best suited to the needs of this type of action. An experienced model user could construct various scenarios to test the range of site values and then have these results compared with field tests.      

5.          Variable Inputs as Sources of Model Uncertainty and Error           

            One of the main problems encountered in model calibration is accurately measuring the distance from the source area to the leading edge of the groundwater contaminant plume.  Often the contaminant may not have reached the furthest monitoring well while it has reached the next closest well, which can be twenty or thirty yards away.  Based upon where the modeler locates the leading edge between these two monitoring wells, the release date can be many years or decades different. 

              In addition to calibrating the model to the site, the modeling results should also be subjected to a sensitivity analysis in order to see how small changes in certain variable inputs and combinations of changes affect the predicted result.  The modeler cannot know the accuracy of the modeling simulation results unless one has subjected the simulation results to a sensitivity analysis of the key input variables.     Finally, another well-known problem with modeling in general is known as the “non-uniqueness” of the solution.  Non-uniqueness refers to the fact that changing the variable inputs to the model can produce the same result.    

7.       Lessons from the Bench.    

          In spite of most judges’ routine acceptance of models which are recognized as reliable in the environmental science community, some courts have been willing to reject their results due to the expert’s lack of specialized qualifications and inappropriate model testing and validation. One such case is Dura Automotive vs. CTS Corp.[iv]  which addresses the distinction between a mere acquaintance with groundwater modeling and expert qualifications. The expert witness in Dura Automotive admitted that he was not an expert in groundwater flow models but instead had relied on the work of others for his opinion.  Judge Posner observed that the “Daubert test must be applied with due regard for the specialization of modern science.  A scientist, however well-credentialed he may be, is not permitted to be the mouthpiece of a scientist of a different specialty. That would not be responsible science.”[v] Judge Posner specifically focused on the problems inherent in reverse groundwater modeling:  “Had Dura merely wanted to use [the groundwater flow model] to determine the current capture zone of the Elkhart well field, we might we have a different case; such use might be routine.  Dura wanted to use these models to determine the capture zone twenty years ago.  The affidavits make clear that adapting models to that use required a host of discretionary expert judgments for the [professional groundwater modelers], not [those of the proffered expert witness] to make.”[vi] 

               Another case, Plainview Water District vs. Exxon Mobil Corp[vii] focused on the procedures necessary to produce a scientifically reliable result. The case involved the use of MODFLOW to predict whether a municipal water supply well would become contaminated by the gasoline-additive methyl tertbutyl alcohol (MTBE).  The plaintiff’s groundwater modeling expert opined that the town’s water supply wells were in imminent threat of contamination from MTBE. Judge Davis’ comprehensive and exhaustive factual analysis summed up the requirements for the reliable use of computerized groundwater transport and fate models as follows: 

“It is axiomatic that computer models are only as reliable as the data inputtedby the modeler. A model and the inputted data may be manipulated to reach a particular result that the modeler is seeking and therein lays the fallibility ofa model.  The benefits derived from using a model are directly related to theaccuracy of the data inputted.  Mere speculation and selective use of certaindata serves no purpose in achieving a forecast of reality.  In order for a modelto be valid, realistic data both past and present must be used.  Even in thebest of circumstances, a model is only an estimate and the accuracy of theestimate depends to a considerable extent on the data selected for use in thecomputer model, the quality of that data and, of course, the skill of the modeler.”[viii]           

The court observed that “the accuracy and reliability of [groundwater modeling] results depends, in large part, on the quality of the data that are input into the model by the modeler.  It is generally accepted in the field of groundwater modeling that to ensure reliable and accurate results a model must be both calibrated and validated…It is also generally accepted practice to perform sensitivity analysis, which is a method of identifying the parameters in the model that have the greatest impact on the results of the model.”[ix] Judge Davis specifically found that the MODFLOW groundwater transport and fate model is generally accepted in the scientific community.[x] He also found that the MODFLOW model had not been calibrated or validated.[xi]  Judge Davis accepted the defendant’s argument that a comparison of modeled groundwater elevations and computer predicted elevations had a 46% error rate.  Judge Davis concluded that “it is generally accepted that a model like the one created by [the plaintiff] should not have an error rate higher than 10 percent.”[xii]          The Plainview case is a primer for lawyers and environmental science professionals on evaluating a computer simulation of contamination fate for judicial decision making.  The results of reverse modeling would be subjected to an even closer judicial scrutiny                                                                                                                                                                                                         Conclusion 

      The use of computerized groundwater fate and transport models for predictive purposes is well-established. When established models are used to determine a release date, causation and liability in environmental cost recovery litigation, their results should not be taken for granted as valid but must meet a much higher standard for reliability. The results from using a sophisticated aquifer simulation model to determine a release date are likely to be within 10 to 15 years of the claimed release date.  Unless such results can be shown to be of 90% or 95% accurate, as a threshold for the reliability of groundwater modeling, the phrase “reasonable degree of scientific certainty” will become meaningless and junk science will have entered the court room.  Only if an objective standard is developed can modeling results be helpful to the trier of fact under the F.R.E. 702.  Modeling results that are only more likely than not to be accurate do not meet any kind of scientific standard and the modeling is just more data.

          The use of computer modeling for age dating presents particular challenges to the litigation team and the judicial system. It requires that all of the elements be present, from a skilled professional modeler, to the correct model and inputs and a demonstration of reliability through calibration and sensitivity analyses. Courts should be vigilant before admitting reverse groundwater modeling results and expert opinions based on them to prove causation or to assign liability in environmental cost recovery litigation until each element has been demonstrated. 

© 2008 James Sherman, Esq. and Edward A. Bertele, Esq.  All rights reserved.