Timeline: External Reviews of Benchmark Dose Software (BMDS) Models
Because BMDS is the Agency's tool for estimating risk assessments, the validity and reliability of its statistical models are of paramount importance. This page provides links to peer reviews and expert summaries of the BMDS applications and its models as they were developed and eventually released.
However, if you are interested in specific changes to the BMDS models and the user interface, you can review the BMDS development timeline.
- 1995 Initial Guidelines on the Benchmark Dose (BMD) Approach
- 2000 Peer Review Workshop on the Draft Technical Guidance
- 2000 External Review of BMDS Version 1.1b and EPA Responses
- 2007 External Review of the Exponential Model
- 2007 External Review of New Quantal Models
- 2007 External Review of the Multistage Weibull Time-to-Tumor Model
- 2008 External Review of the Concentration x Time (ten Berge) Model
- 2011 External Review of the MS_Combo Model
1995 Initial Guidelines on the Benchmark Dose (BMD) Approach
EPA's Risk Assessment Forum (RAF) publishes the initial guidelines on the use of the benchmark dose (BMD)
BMDAn exposure due to a dose of a substance associated with a specified low incidence of risk, generally in the range of 1% to 10%, of a health effect; or the dose associated with a specified measure or change of a biological effect. approach for assessing noncancer health risk.
2000 Peer Review Workshop on the Draft Technical Guidance
An external peer review workshop on the Draft Benchmark Dose Technical Guidance was held in Washington, DC.
2000 External Review of BMDS Version 1.1b and EPA Responses
Numerous comments were received by 24 reviewers of the original BMDS beta test version (version 1.1b). Reviewers were impressed with its capabilities, potential and overall user friendliness. Reviewer comments covered all aspects of the software, including the interface, model reliability, model output, hardware compatibility and documentation.
2007 External Review of the Exponential Model
In September, 2007, a new Exponential model was externally reviewed by experts with reviews generally positive and confirmed that the functioning of the computer code had been rigorously tested. The finalized version of the Exponential models that appeared in BMDS took these comments into account.
2007 External Review of New Quantal Models
New quantal dose-response models having alternative background parameters were externally reviewed by experts in June, 2007. These models were included as part of the BMDS 2.0 beta. Reviews were generally positive and confimed that the functioning of the computer code had been rigorously tested. The reviewers' comments, EPA's responses, and the model development and testing report are available for review.
The new models modified existing BMDS quantal models. They were developed by modifying existing source codes, changing the code only as necessary. Adding these new models to the BMDS model suite expanded the range of alternatives available for fitting dose-response curves using generally accepted types of models.
At the time, BMDS's quantal models had a background term that was directly additive to the response probability. In this case, five new models implemented an alternative form of background term, one that is additive to the dose: multistage, cancer, log-probit, gamma, and Weibull. Models with background terms additive to dose were proposed and analysed in peer-reviewed papers by leading experts over 20 years ago, so they are not new to risk analysis. One type of model, log-logistic, was not provided in a version with background additive to dose, because of technical difficulties causing unreliable convergence on solutions.
Two quantal models, probit and logistic, that previously had no explicit background response parameter, became available in that form.
Therefore, with the exception of the log-logistic, all the BMDS quantal models available in BMDS 2.0 beta had two versions, differing only in the background term. The new quantal models kept the names used in BMDS 1.4.1c. New versions with an alternative background parameter were distinctively named to identify them clearly, e.g., "Weibull-BgDose" or "Probit-BgResp".
2007 External Review of the Multistage Weibull Time-to-Tumor Model
In June, 2007, a new time-to-tumor multistage Weibull model ("MSW") was externally reviewed by experts, with reviews generally positive and confirmed that the functioning of the computer code had been rigorously tested. The reviewers' comments are provided with EPA's responses (finalized in 2008, after revisions to the program and documentation that were followed by more testing).
2008 External Review of the Concentration x Time (ten Berge) Model
- To replicate for use in BMDS the CxT model that Wil ten Berge originally implemented in Visual Basic in the C program language.
- To develop comprehensive documentation for the model's use in BMDS.
Thus, the primary charge to the external reviewers was to determine whether these intentions were adequately met. While providing comments on how the software might be improved, the reviewers' comments indicated that the first intention was generally met and offered a number of improvements to the model documentation. The ten Berge model included in the current BMDS version take into account these comments.
2011 External Review of the MS_Combo Model
In early 2011, the new MS_Combo model was externally reviewed by three experts in dose-response modeling and documented in a MS_Combo Peer Review Report(21 pp, 118 K, 2011-03-03) . The primary purpose of this review was to ensure that the results of the draft EPA MS_Combo model calculates correct, accurate, and statistically valid estimates of the maximum likelihood and lower bound on a dose corresponding to fixed risk level, for a combination of two or more tumor types. The peer reviewers were specifically asked to evaluate MS_Combo with respect to clarity of the documentation and model output, and the adequacy of testing methods and test results. The reviewers generally agreed that the model produced statistically valid results, but made several recommendations regarding enhancements that could facilitate or expand its practical application, and how the documentation and outputs could be improved with respect to clarity. EPA revised the MS_Combo software and documentation in response to these comments, incorporating most of the suggested revisions. Some suggested revisions that were considered important but not critical (e.g., reporting the entire estimation-error distribution for each estimated BMD) were postponed for implementation in a future version of MS_Combo. A technical background document for the MS_Combo program contains details necessary to understand the program's assumptions and application.