Entropy Methods on Finding Optimal Linear Combinations with an Application to Biomarkers

Identifying an optimal linear combination of continuous variables is a key objective in various fields of research, such as medicine. This manuscript explores the use of information-theoretical approaches used to establish these linear combinations. Coefficients obtained from logistic regression can be used to construct such a linear combination, and this approach has been commonly adopted in the literature for comparison purposes. The main contribution of this work is to propose novel ways of determining these linear combination coefficients by optimizing information-theoretical objective functions. Biomarkers are usually continuous measurements utilized to diagnose if a patient has the underlying disease. Certain disease contexts may lack high diagnostic power biomarkers, making their optimal combination a critical area of interest. We apply the above-mentioned novel methods to the problem of a combination of biomarkers. We assess the performance of our proposed methods against combinations derived from logistic regression coefficients, by comparing area under the ROC curve (AUC) values and other metrics in a broad simulation and a real life data application.