Black Box FDR

06/08/2018
by   Wesley Tansey, et al.
0

Analyzing large-scale, multi-experiment studies requires scientists to test each experimental outcome for statistical significance and then assess the results as a whole. We present Black Box FDR (BB-FDR), an empirical-Bayes method for analyzing multi-experiment studies when many covariates are gathered per experiment. BB-FDR learns a series of black box predictive models to boost power and control the false discovery rate (FDR) at two stages of study analysis. In Stage 1, it uses a deep neural network prior to report which experiments yielded significant outcomes. In Stage 2, a separate black box model of each covariate is used to select features that have significant predictive power across all experiments. In benchmarks, BB-FDR outperforms competing state-of-the-art methods in both stages of analysis. We apply BB-FDR to two real studies on cancer drug efficacy. For both studies, BB-FDR increases the proportion of significant outcomes discovered and selects variables that reveal key genomic drivers of drug sensitivity and resistance in cancer.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/01/2018

The Holdout Randomization Test: Principled and Easy Black Box Feature Selection

We consider the problem of feature selection using black box predictive ...
research
06/04/2019

Covariate-Powered Empirical Bayes Estimation

We study methods for simultaneous analysis of many noisy experiments in ...
research
07/31/2020

Deep Direct Likelihood Knockoffs

Predictive modeling often uses black box machine learning methods, such ...
research
09/28/2022

Shining a Light on Forensic Black-Box Studies

Forensic science plays a critical role in the American criminal justice ...
research
10/06/2021

Unifying Likelihood-free Inference with Black-box Sequence Design and Beyond

Black-box optimization formulations for biological sequence design have ...
research
09/23/2022

Quantile-constrained Wasserstein projections for robust interpretability of numerical and machine learning models

Robustness studies of black-box models is recognized as a necessary task...

Please sign up or login with your details

Forgot password? Click here to reset