Benchmark Dose Software

Timeline: External Reviews of Benchmark Dose Software (BMDS) Models

Because BMDS is the Agency's tool for estimating risk assessments, the validity and reliability of its statistical models are of paramount importance. This page provides links to peer reviews and expert summaries of the BMDS applications and its models as they were developed and eventually released. 

However, if you are interested in specific changes to the BMDS models and the user interface, you can review the BMDS development timeline.

On this page:

1995 Initial Guidelines on the Benchmark Dose (BMD) Approach

EPA's Risk Assessment Forum (RAF) publishes the initial guidelines on the use of the benchmark dose (BMD)HelpBMDAn exposure due to a dose of a substance associated with a specified low incidence of risk, generally in the range of 1% to 10%, of a health effect; or the dose associated with a specified measure or change of a biological effect. approach for assessing noncancer health risk.

Top of Page

2000 Peer Review Workshop on the Draft Technical Guidance

An external peer review workshop on the Draft Benchmark Dose Technical Guidance was held in Washington, DC.

Top of Page

2000 External Review of BMDS Version 1.1b and EPA Responses

Numerous comments were received by 24 reviewers of the original BMDS beta test version (version 1.1b). Reviewers were impressed with its capabilities, potential and overall user friendliness. Reviewer comments covered all aspects of the software, including the interface, model reliability, model output, hardware compatibility and documentation.

Top of Page

2007 External Review of the Exponential Model

In September, 2007, a new Exponential model was externally reviewed by experts with reviews generally positive and confirmed that the functioning of the computer code had been rigorously tested.  The finalized version of the Exponential models that appeared in BMDS took these comments into account.

Top of Page

2007 External Review of New Quantal Models

New quantal dose-response models having alternative background parameters were externally reviewed by experts in June, 2007. These models were included as part of the BMDS 2.0 beta. Reviews were generally positive and confimed that the functioning of the computer code had been rigorously tested. The reviewers' comments, EPA's responses, and the model development and testing report are available for review.

The new models modified existing BMDS quantal models. They were developed by modifying existing source codes, changing the code only as necessary. Adding these new models to the BMDS model suite expanded the range of alternatives available for fitting dose-response curves using generally accepted types of models.

At the time, BMDS's quantal models had a background term that was directly additive to the response probability. In this case, five new models implemented an alternative form of background term, one that is additive to the dose: multistage, cancer, log-probit, gamma, and Weibull. Models with background terms additive to dose were proposed and analysed in peer-reviewed papers by leading experts over 20 years ago, so they are not new to risk analysis. One type of model, log-logistic, was not provided in a version with background additive to dose, because of technical difficulties causing unreliable convergence on solutions.

Two quantal models, probit and logistic, that previously had no explicit background response parameter, became available in that form.

Therefore, with the exception of the log-logistic, all the BMDS quantal models available in BMDS 2.0 beta had two versions, differing only in the background term. The new quantal models kept the names used in BMDS 1.4.1c. New versions with an alternative background parameter were distinctively named to identify them clearly, e.g., "Weibull-BgDose" or "Probit-BgResp".

Top of Page

2007 External Review of the Multistage Weibull Time-to-Tumor Model

In June, 2007, a new time-to-tumor multistage Weibull model ("MSW") was externally reviewed by experts, with reviews generally positive and confirmed that the functioning of the computer code had been rigorously tested. The reviewers' comments are provided with EPA's responses (finalized in 2008, after revisions to the program and documentation that were followed by more testing).

Top of Page

2008 External Review of the Concentration x Time (ten Berge) Model

In June, 2008, a new Concentration x Time (ten Berge) model was externally reviewed by experts in modeling and statistics. The intentions of EPA with respect to this model were as follows:
  • To replicate for use in BMDS the CxT model that Wil ten Berge originally implemented in Visual Basic in the C program language.
  • To develop comprehensive documentation for the model's use in BMDS.

Thus, the primary charge to the external reviewers was to determine whether these intentions were adequately met. While providing comments on how the software might be improved, the reviewers' comments indicated that the first intention was generally met and offered a number of improvements to the model documentation. The ten Berge model included in the current BMDS version take into account these comments.

Top of Page

2011 External Review of the MS_Combo Model

In early 2011, the new MS_Combo model was externally reviewed by three experts in dose-response modeling and documented in a MS_Combo Peer Review Report(21 pp, 118 K, 2011-03-03) . The primary purpose of this review was to ensure that the results of the draft EPA MS_Combo model calculates correct, accurate, and statistically valid estimates of the maximum likelihood and lower bound on a dose corresponding to fixed risk level, for a combination of two or more tumor types. The peer reviewers were specifically asked to evaluate MS_Combo with respect to clarity of the documentation and model output, and the adequacy of testing methods and test results. The reviewers generally agreed that the model produced statistically valid results, but made several recommendations regarding enhancements that could facilitate or expand its practical application, and how the documentation and outputs could be improved with respect to clarity. EPA revised the MS_Combo software and documentation in response to these comments, incorporating most of the suggested revisions. Some suggested revisions that were considered important but not critical (e.g., reporting the entire estimation-error distribution for each estimated BMD) were postponed for implementation in a future version of MS_Combo. A technical background document for the MS_Combo program contains details necessary to understand the program's assumptions and application.

Top of Page