Dissertations, Theses, and Capstone Projects

Date of Degree

9-2023

Document Type

Dissertation

Degree Name

Ph.D.

Program

Criminal Justice

Advisor

Jeff Mellow

Committee Members

Jeff Mellow

Mechtild Prinz

Kevin Wolff

Lynn Garcia

Max Houck

Subject Categories

Criminology and Criminal Justice | Forensic Science and Technology

Keywords

forensic science, quality management, nonconformities, accreditation, forensic science commissions, oversight

Abstract

Forensic science oversight in the U.S. largely relies upon voluntary third-party forensic laboratory accreditation programs. Without a national system of regulation and given the highly fragmented local systems of control, few forensic science service providers (FSSPs) are subject to regulatory oversight beyond their third-party accreditors. Texas is unique in its establishment of a robust statewide oversight system and a strong governmental culture of transparency, permitting this study of forensic quality management. This study consisted of two parts. The first part of this dissertation characterized and analyzed quality incident reports (QIRs) published by the Texas Department of Public Safety Crime Laboratory System (DPS Crime Labs). The second part of this dissertation identified, characterized, and analyzed the range of disclosures and complaints received by the Texas Forensic Science Commission (TFSC), its responses thereto, and evaluated whether oversight provided by the TFSC differed from oversight by the American National Standards Institute (ANSI) National Accreditation Board (ANAB) accreditation. This dissertation project will contribute to extending the body of research on forensic science quality management and oversight by exploring the following research questions:

Part 1: QIR Study

  • RQ1: What are the characteristics of QIRs produced by the DPS Crime Labs from post-Quality Action Plan (QAP) revision QIRs to May 2021?
  • RQ2: What aspects of QIRs predict a significant disclosure to oversight bodies?
  • RQ3: What did the QAP revision reveal about the theories of forensic science quality management infrastructure when evaluating System DPS Crime Labs QIRs produced before and after the QAP revision?

Part 2: TFSC Study

  • RQ4: What are the characteristics of complaints and self-disclosures investigated by the TFSC? Are complaints and self-disclosures significantly different?
  • RQ5: What factors of complaints and self-disclosures predict action by TFSC?
  • RQ6: What factors of self-disclosures predict action by ANAB?
  • RQ7: What do investigations of self-disclosures and complaints to the TFSC reveal about how the theories of forensic science quality management infrastructure operate in these contexts?

Methods: This dissertation used publicly available data from DPS Crime Labs and the TFSC to address these research questions. The quantitative portion of the QIR study used QIRs produced by the DPS Crime Labs from 2016-2020 (n=1,203) after a revision was made to the Quality Action Plan (post-QAP revision). Contextual content analysis (CCA) was conducted on the QIRs to extract data and code variables. Exploratory data analysis was used to characterize the quality incidents and logistic regression was used to test for predictors of significant disclosures. The qualitative portion of the QIR study used DPS Crime Labs’ System-level location QIRs (n=146) from before (pre-QAP) and after (post-QAP) the QAP revision. Qualitative content analysis (QCA) and triangulation were used to better understand how theories of forensic science quality management infrastructure were expressed through the production of QIRs before and after the QAP revision.

The quantitative portion of the TFSC Study used complaints (n=207) and self-disclosures (n=98) filed with the TFSC and the responses to these complaints and self-disclosures produced by TFSC and ANAB between 2016-2020. CCA was conducted on documents stored on the TFSC website as well as documentation obtained from the TFSC upon request to extract data and code variables. Exploratory data analysis was used to characterize the complaints and self-disclosures and logistic regression was used to test for predictors of (1) TFSC taking further action on complaints and self-disclosures and (2) ANAB taking further action on self-disclosures. For the qualitative portion of the TFSC study, a subset of self-disclosures was selected for further QCA and triangulation to better understand how the theories of forensic science quality management infrastructure might be expressed in the TFSC’s response to these quality incidents. Five self-disclosures in which forensic evidence was lost, missing, or destroyed and in which ANAB chose to review the quality incidents at a future date were selected for review.

Findings: In the QIR Study, the exploratory data analysis found that among forensic science practices, quality incidents occurred most frequently in biology/DNA, seized drugs, and evidence coordination. Among types of nonconformity, evidence processing/storage was the second most frequent type of nonconformity after testing/equipment. Evidence coordination also comprised 38.9% of significant disclosures, far more than any other forensic science practice. The full model, containing all variables significantly associated with significant disclosure when control variables were accounted for, produced three significant predictors—violation of discipline-specific standards, QAP conducted, and severity-level of the quality incident for major compared to minor severity-levels. The qualitative analysis of System-level QIRs found the QIRs in the two different periods (pre- and post-QAP) were so distinct that few opportunities were available to fully compare how DPS Crime Labs implemented the theories of forensic science quality management infrastructure. Notably, the qualitative analysis found the QIRs exhibited an unanticipated intersection of the culture of anticipation, repair, and disclosure when decisions about how and whether to send a corrected report were conditioned upon legal rather than scientific outcomes.

In the TFSC Study, the vast majority of complaints and self-disclosures were dismissed by the TFSC with 5% of complaints and 10% of self-disclosures accepted for further action. In contrast, 99% of Self-Disclosures were dismissed by or lacked action by ANAB. When complaints were dismissed, the rationale for more than half of dismissals exceeded the scope of what the TFSC was permitted to review. The forensic science practice that was the most frequent subject of complaints was biology/DNA while the most frequent forensic science practice that was the subject of self-disclosures was seized drugs. The full model, containing all variables significantly associated with TFSC disposition when control variables were accounted for, produced two significant predictors—type of complainant when an individual person was compared to all other complainants and type of allegation when negligence and/or misconduct was compared to all other allegations. Since there was a nearly complete separation in the outcome variable ANAB disposition, predictors of this outcome could not be analyzed. The qualitative analysis of the five self-disclosure cases depicted a stark contrast between the visible and active TFSC oversight and the undetectable nature of ANAB accreditation responses.

Discussion and Implications: Given the focus of forensic science reform efforts on pattern evidence disciplines (friction ridge, firearm/toolmarks, trace evidence) and concerns regarding their scientific foundations, the QIR study provided evidence that more attention and resources may need to be focused on the collection, processing, and chain of custody of forensic evidence in FSSPs. Although pattern evidence disciplines comprised a small fraction of the QIRs in the study, this data does not necessarily mean that pattern evidence disciplines produced fewer nonconformities or errors. Rather, QIRs may not be the right tool for detecting the accuracy or quality of pattern evidence testing, and other strategies like blind control testing or evidence line-ups may be more effective. The accreditation process proved to be an important quality management strategy in FSSPs but is not positioned to provide the kind of oversight that a regulatory body like the TFSC can. TFSC offered a transparent and publicly accessible forum for discussing and understanding problems that may occur in FSSPs. TFSC is also positioned to act more quickly and investigate disclosures more comprehensively than the accreditation body.

Accreditation is essential and necessary to quality management, but state forensic science commissions produce accountability and transparency that accreditation cannot. Both robust accreditation and state forensic science commission oversight are necessary for reliable and accountable forensic science. Currently, state forensic science oversight bodies range in their level of regulatory power, public accessibility, transparency, and composition. As states across the country contemplate forensic science commissions, especially the question of whether one is needed given the accreditation status of FSSPs in their state, this study can offer insight into the benefits and limitations of accreditation as well as the degree to which state forensic science commissions can support more accurate and more just forensic science.

Download

Share

COinS