Zimmermann, Manuela, Baillie, Mark, Kormaksson, Matthias, Ohlssen, David and Sechidis, Kostas (2024) All that glitters is not gold: Type-I error controlled variable selection from clinical trial data. Clinical Pharmacology & Therapeutics, 115 (4). pp. 774-785. ISSN 0009-92361532-6535

Abstract

Beyond their primary purpose of establishing causal effects, clinical trial data can be used to identify prognostic measures of disease or biomarkers that predict treatment efficacy. Such endeavors can be interpreted as variable selection problems for which a plethora of machine learning algorithms has recently been developed. However, these algorithms are generally designed to optimize predictions and often only provide the measures used for variable selection, such as importance scores, as a by-product. Thus, without known operating characteristics or a mechanism for error control, these approaches contribute to the current replicability crisis. In the context of clinical development, this lack of control for false discoveries (type-I errors) can result in unnecessary research efforts, increased patient burden, and avoidable costs. Here, we review a recently proposed model-agnostic wrapper framework, the knockoff framework, which offers a robust approach to variable selection with guaranteed type-I error control. We explore the operating characteristics of various knockoff based variable selection methods under broad settings relevant to the analysis of clinical trial data, raising awareness for practical considerations. Furthermore, we introduce a novel knockoff generation method that addresses two main limitations of previously suggested methods relevant for clinical development settings, empirically obtaining tighter bounds on type-I error control and gaining an order of magnitude in computational efficiency in mixed data settings.

Item Type:	Article
Keywords:	type-I error control, biomarker discovery, machine learning, prognostic variables, knockoff framework
Date Deposited:	25 Jun 2024 00:46
Last Modified:	25 Jun 2024 00:46
URI:	https://oak.novartis.com/id/eprint/51434