Lossy Information Exchange and Instantaneous Communications.

摘要

In this project, we studied a new theoretical framework for information transmission with the focus on extracting only a part of the information. This formulation is particularly useful when we apply information theory to data analysis problems, where the goal is different from the full and reliable information recovery needed in the classical communications problems. We developed a geometric structure for the space of probability distributions, and a new method to decompose the information carried by the observed data based on that. This formulation gives a general setup to understand dimension reduction as lossy information processing procedures and gives a new operational meaning of information metrics, in the context of data analytics. We can quantitatively describe the information efficiency, computation complexity, and sample complexity to learn the model in one picture. We also make connections to the existing results on dimensional reduction. Based on this framework, we developed new algorithms for dimension reduction and non-linear feature selection. We proved a number of optimality results for these new constructions, and applied the algorithm in real data analysis tasks.