LLNA Report Appendix E: Evaluation Guidance to the Peer Review Panel
Previous Appendix Next Appendix
Return to the LLNA Report
Return to the NICEATM-ICCVAM Document Search Page
Return to the ICCVAM Web Site
Evaluation Guidance to the Peer Review Panel
A. Instructions for Peer Review Panel Members
The Peer Review Panel was charged with developing a consensus on the usefulness of the proposed LLNA test method (appendix D) as an alternative for the currently accepted guinea pig assay. In reaching this determination, the panel was asked to evaluate all of the available information in the submission in accordance with the published criteria for validation and acceptance of toxicological test methods (NIEHS, 1997). The Peer Review Panel was charged with preparing a written report that summarized the extent to which each of these criteria were addressed, and that addressed the acceptability of this method as a substitute for the guinea pig assay.
An outline of the major items addressed in the Peer Review Panel report is provided below in "B. Points for Evaluation." Specific questions and considerations were added by the Interagency Immunotoxicity Working Group to ensure that the assessment provided adequate information to facilitate agency decisions on the regulatory acceptability of the method.
One primary and at least two secondary reviewers were designated for each section by the NIEHS Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM) in consultation with the Peer Review Panel Chair. These individuals were requested to prepare draft written responses for their assigned sections. All reviewers were encouraged to familiarize themselves with the entire set of questions and to comment on any or all sections. All reviewers were asked to complete the summary conclusions section.
In conducting this review, the primary focus of the Peer Review Panel was to evaluate the information supporting the usefulness of the proposed LLNA Test Method Protocol (LLNA ICCVAM Submission). Based on the information provided in the Submission, the panel was asked to determine if the LLNA is an acceptable alternative to standard guinea pig assays for identifying human contact allergens. Two overall questions that they were asked to address were:
Has the LLNA been evaluated sufficiently and is its performance satisfactory to support its adoption as a stand-alone alternative?
Does the LLNA offer advantages with respect to animal welfare considerations (refinement, reduction, and replacement)?
The focus of the Peer Review Panel evaluation was on the utility of the LLNA, as described in the proposed Test Method Protocol, for detecting possible human contact allergens. The Panel was made aware that modifications to the proposed LLNA protocol have been made or were under development (e.g., ex-vivo use of radiolabeled thymidine, use of nonradioactive methods) which were outside the scope of this evaluation. However, the Panel was asked to submit suggestions for future evaluations or workshops to review proposed test method revisions.
B. Points for Evaluation
1. Summary Conclusions
Based on the information provided:
a. Compared with current methods [e.g., the guinea pig maximization test (GPMT)], could this method be used to provide equivalent or better prediction of human allergic contact dermatitis?
b. Does the LLNA adequately identify the lack of potential of chemicals to induce human allergic contact dermatitis? If applicable, specify those circumstances (e.g., specific chemicals/chemical classes) where the LLNA, or test results from the LLNA, would be considered either (i) inadequate or (ii) equal to or better than current methods for concluding that the test article is not a contact sensitizer.
c. Does the LLNA adequately identify the potential of chemicals to induce human allergic contact dermatitis? If applicable, specify those circumstances (e.g., specific chemicals/chemical classes) where the LLNA, or test results from the LLNA, would be considered either (i) inadequate or (ii) equal to or better than current methods for concluding that the test article is a contact sensitizer.
d. Discuss conditions/limitations/restrictions that may affect the intended use of the LLNA, and that are justified based upon the presence or lack of scientific evidence.
e. Discuss advantages of the proposed LLNA, as compared to the standard guinea pig methods.
f. Has there been adequate consideration and appropriate incorporation of animal use refinement, reduction, and replacement alternatives? Will the LLNA reduce the number of animals required or refine the procedure to eliminate pain or distress compared with the reference tests?
2. Test Method Description (see Appendix D, LLNA Protocol)
a. Are the test method and protocol described in sufficient detail, including the scientific and mechanistic basis of the test, range of applications, endpoints, numbers of replicates, need for dose-response curves, and acceptable variations in the protocol?
1) Is the protocol used to generate the supporting submission data in agreement with the proposed protocol (Section II. D.)? If not, discuss the adequacy of the rationale provided for changes incorporated in the proposed protocol.
2) Evaluate the appropriateness of the dose selection procedure. Discuss the need for determination of dermal irritation (e.g., as done for the guinea pig test) or acute toxicity data prior to conducting the actual test.
3) Evaluate the appropriateness of the number of dose groups recommended as necessary for an adequate study.
b. Comment on the adequacy and completeness of the test method protocol, including:
1) Description of the material and equipment needed to conduct the test. Is the number of mice per dose group appropriate? Is the age range appropriate? Is the designated gender and strain appropriate?
2) Description of what is measured and how it is used.
3) Description of data analysis, evaluation, and decision criteria (i.e., a >3-fold stimulation factor) used to identify substances as: 1) a positive skin sensitizer, and 2) a negative skin sensitizer.
c. Are there appropriate provisions for the use of positive, negative, and irritation control chemicals?
d. Discuss the role of a dose response relationship in interpreting the results of this assay.
e. What are the strengths and/or limitations of the LLNA and are they described adequately, including the usefulness for testing mixtures, extracts, and metals?
f. Are there editorial/technical corrections necessary for the proposed protocol?
3. Test Method Data Quality
Is there evidence of sufficient quality assurance/quality control [i.e., were experiments conducted and data collected and maintained in accordance with Good Laboratory Practice (GLP) standards and procedures; in the "spirit" of GLPs (e.g., GLP standards without audits)]? If not, is there clear indication from the technical data that there was adequate record-keeping or data collection.
a. Is there an assurance provided that indicates there was adherence to the protocol during the validation studies? Are deviations from the standard protocol clearly described and justified?
b. If changes were made to the test method protocol during the validation studies, is the rationale for the changes provided, are data clearly identified to indicate which protocol was used, and are the potential impact of these changes on evaluation of the test method presented?
c. Was a data audit conducted by a Quality Assurance Unit? If so, is the data quality satisfactory based on the audit results (e.g., adequate adherence to protocols, record-keeping following GLPs)?
4. Test Method Performance
a. Are the data provided in sufficient detail for you to evaluate the results and conclusions obtained with the LLNA?
b. Comment on the adequacy of the methods used to evaluate the performance of the test method. Are results of the LLNA and the reference test(s) compared and evaluated appropriately?
c. Comment on the adequacy of the numbers of chemicals/products selected to evaluate the performance (end result) of the method for each chemical/product class. Are there limitations in application of this assay to specific chemical/product classes?
d. Are sufficient data provided to adequately evaluate the performance of the method for its proposed use?
e. Comment on the sensitivity, specificity, concordance, false positive rate, and false negative rates for the chemical/product classes that the method is proposed to be used for.
1) To what extent does the method correctly predict negative effects for some or all chemicals/products?
2) To what extent does the method correctly predict positive effects correctly for some or all classes? Does it consistently over or under predict toxicity compared with the current test method?
f. Are the sensitivity, specificity, concordance, and false positive and negative rates acceptable for the chemical/product classes tested?
g. Are the conclusions on the usefulness of this method scientifically sound?
1) Are results of the LLNA clinically relevant and is the test predictive for human contact allergens?
2) Is the utility of the method clearly established for regulatory use in hazard assessment of chemicals as potential contact sensitizers?
5. Determination of Test Method Reliability (Repeatability/Reproducibility)
Are intra- and inter-laboratory reproducibility adequately evaluated?
a. Comment on the adequacy of the evaluation of intralaboratory repeatability and reproducibility of the test method, and the data used to define and describe the level of intralaboratory variability.
b. Comment on the adequacy of the evaluation of interlaboratory reproducibility of the test method, and the data used to define and describe the level of interlaboratory variation.
1) Consider the range of vehicle control data within and across laboratories in the validation studies. Do these differences affect data quality (reproducibility, sensitivity, etc)?
c. Was the reproducibility of the test method evaluated on a series of appropriate reference chemicals or products, and do these adequately represent the types of substances for which the test method is proposed to be used?
d. Are the results obtained with the LLNA sufficiently repeatable and reproducible?
e. Comment on the reproducibility and reliability of the LLNA as compared to standard guinea pig assays.
6. Other Scientific Reviews
Comment on and compare the conclusions published in independent peer-reviewed reports or other independent scientific reviews of the test method, compared to the conclusions reached in this report, and comment on any other ongoing evaluations of this method.
7. Other Considerations
a. Can the test method be readily transferred among properly equipped and staffed laboratories; that is:
1) Is it relatively insensitive to minor changes in protocol (e.g., the acceptable temperature range for reagents and for the location where the test will be conducted)?
2) Are the level of training and expertise required to conduct the test reasonable?
3) Are the necessary equipment and supplies relatively easy to obtain?
b. Is the method cost-effective, relative to the cost of conducting the currently accepted test methods for hypersensitivity?
c. Is the time needed to conduct the test reasonable?
d. Is there any other information that should be added to the report, published or unpublished?
e. Has there been adequate consideration and appropriate incorporation of animal use refinement, reduction, and replacement alternatives? Will the LLNA reduce the number of animals required or refine the procedure to reduce or eliminate pain or distress compared with the reference tests?
C. Related Issues
1. Although this evaluation is for a specific LLNA protocol proposed as an alternative for currently used guinea pig tests, what other endpoints or test methods would you like to see evaluated by ICCVAM in the future?
2. Are there ideas for potential workshops and validation efforts that you think that ICCVAM or others should support in this area of contact hypersensitivity?
Reference:
NIEHS (National Institute of Environmental Health Sciences). 1997. Validation and regulatory acceptence of toxicological test methods: A report of the ad hoc Interagency Coordinating Committee on the Validation of Alternative Methods. NIH Publication No. 97-3981. NIEHS, Research Triangle Park, NC.
Previous Appendix Next Appendix
Return to the LLNA Report
Return to the NICEATM-ICCVAM Document Search Page
Return to the ICCVAM Web Site
|