Information Theory and Machine Learning

• Other Authors: Zheng, Lizhong (Editor), Tian, Chao (Editor)
• Format: Electronic Book Chapter
• Language: English
• Published: MDPI - Multidisciplinary Digital Publishing Institute 2022
• Subjects: Technology: general issues; History of engineering & technology; supervised classification; independent and non-identically distributed features; analytical error probability; empirical risk; generalization error; K-means clustering; model compression; population risk; rate distortion theory; vector quantization; overfitting; information criteria; entropy; model-based clustering; merging mixture components; component overlap; interpretability; time series prediction; finite state machines; hidden Markov models; recurrent neural networks; reservoir computers; long short-term memory; deep neural network; information theory; local information geometry; feature extraction; spiking neural network; meta-learning; information theoretic learning; minimum error entropy; artificial general intelligence; closed-loop transcription; linear discriminative representation; rate reduction; minimax game; fairness; HGR maximal correlation; independence criterion; separation criterion; pattern dictionary; atypicality; Lempel-Ziv algorithm; lossless compression; anomaly detection; information-theoretic bounds; distribution and federated learning
• Online Access: DOAB: download the publication; DOAB: description of the publication

The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be...