Quadratic mutual information feature selection

Davor Sluga (Author), Uroš Lotrič (Author)

Abstract

We propose a novel feature selection method based on quadratic mutual information which has its roots in Cauchy–Schwarz divergence and Renyi entropy. The method uses the direct estimation of quadratic mutual information from data samples using Gaussian kernel functions, and can detect second order non-linear relations. Its main advantages are: (i) unified analysis of discrete and continuous data, excluding any discretization; and (ii) its parameter-free design. The effectiveness of the proposed method is demonstrated through an extensive comparison with mutual information feature selection (MIFS), minimum redundancy maximum relevance (MRMR), and joint mutual information (JMI) on classification and regression problem domains. The experiments show that proposed method performs comparably to the other methods when applied to classification problems, except it is considerably faster. In the case of regression, it compares favourably to the others, but is slower.

Keywords

izbira značilk;informacijsko-teoretične mere;kvadratna medsebojna informacija;Cauchy-Schwarzova divergenca;feature selection;information-theoretic measures;quadratic mutual information;Cauchy-Schwarz divergence;

Data

Language:	English
Year of publishing:	2017
Typology:	1.01 - Original Scientific Article
Organization:	UL FRI - Faculty of Computer and Information Science
UDC:	004
COBISS:	1537405123
ISSN:	1099-4300
Views:	177
Downloads:	55
Average score:	0 (0 votes)
Metadata:

Other data

Secondary language:	Slovenian
Secondary keywords:	izbira značilk;informacijsko-teoretične mere;kvadratna medsebojna informacija;Cauchy-Schwarzova divergenca;
Type (COBISS):	Article
Pages:	str. 1-16
Volume:	ǂVol. ǂ19
Issue:	ǂiss. ǂ4
Chronology:	2017
DOI:	10.3390/e19040157
ID:	13534367