PuSH - Publication Server of Helmholtz Zentrum München

Ellenbach, N. ; Boulesteix, A.L.* ; Bischl, B.* ; Unger, K. ; Hornung, R.*

Improved outcome prediction across data sources through robust parameter tuning.

J. Classif. 38, 212–231 (2020)
Publ. Version/Full Text Research data DOI
Open Access Gold (Paid Option)
Creative Commons Lizenzvertrag
In many application areas, prediction rules trained based on high-dimensional data are subsequently applied to make predictions for observations from other sources, but they do not always perform well in this setting. This is because data sets from different sources can feature (slightly) differing distributions, even if they come from similar populations. In the context of high-dimensional data and beyond, most prediction methods involve one or several tuning parameters. Their values are commonly chosen by maximizing the cross-validated prediction performance on the training data. This procedure, however, implicitly presumes that the data to which the prediction rule will be ultimately applied, follow the same distribution as the training data. If this is not the case, less complex prediction rules that slightly underfit the training data may be preferable. Indeed, a tuning parameter does not only control the degree of adjustment of a prediction rule to the training data, but also, more generally, the degree of adjustment to thedistribution ofthe training data. On the basis of this idea, in this paper we compare various approaches including new procedures for choosing tuning parameter values that lead to better generalizing prediction rules than those obtained based on cross-validation. Most of these approaches use an external validation data set. In our extensive comparison study based on a large collection of 15 transcriptomic data sets, tuning on external data and robust tuning with a tuned robustness parameter are the two approaches leading to better generalizing prediction rules.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Journal article
Document type Scientific Article
Corresponding Author
Keywords Prediction ; Robust Modeling ; Tuning Parameter Value Optimization ; Batch Effects; External Validation; Regression; Selection; Models
ISSN (print) / ISBN 0176-4268
e-ISSN 1432-1343
Journal Journal of Classification
Quellenangaben Volume: 38, Issue: , Pages: 212–231 Article Number: , Supplement: ,
Publisher Springer
Publishing Place New York
Non-patent literature Publications
Reviewing status Peer reviewed
Institute(s) Research Unit Radiation Cytogenetics (ZYTO)