Quantum information science is an exciting, broad, rapidly progressing, cross-disciplinary field, which makes it both attractive and hard to enter. This tutorial will first answer the three essential questions that any newcomer needs to know: How is quantum information represented? How is quantum information processed? How is classical information extracted from quantum states? We will then introduce the fundamental quantum algorithms and protocols that illustrate quantum computing advantages. We will examine how basic information and coding theoretic notions generalize to quantum. The tutorial will conclude with examples that demonstrate the power of quantum correlations. No prior knowledge of quantum mechanics is assumed.

 

Emina Soljanin is a professor of Electrical and Computer Engineering at Rutgers. Before moving to Rutgers in January 2016, she was a (Distinguished) Member of Technical Staff for 21 years in various incarnations of the Mathematical Sciences Research Center of Bell Labs. Her broad interests and expertise include distributed computing and quantum information science. She is an IEEE Fellow, an outstanding alumnus of the Texas A&M School of Engineering, the 2011 Padovani Lecturer, a 2016/17 Distinguished Lecturer, and the 2019 IEEE Information Theory Society President.

Bridging Information Theory and Machine Learning: A Loss Function Perspective

Lalitha Sankar (Arizona State University)

Machine learning has dramatically enhanced the role of automated decision making across a variety of domains. There are three ingredients that are at the heart of designing of sound ML algorithms: data, learning architectures, and loss functions. In this tutorial, we focus on loss functions and the role of information theory in understanding the choice of loss functions in learning. To this end, we introduce alpha-loss as a parameterized class of loss functions that resulted from operationally motivating information-theoretic measures. Tuning the parameter alpha from 0 to infinity allows continuous interpolation between known and oft-used losses: log-loss (alpha=1), exponential loss (alpha=1/2), and 0-1 loss (alpha=infinity).

In the first third of the tutorial, we will discuss fundamental properties of alpha-loss: its information-theoretic interpretations, consistency and generalization guarantees, and the behavior of its optimization landscape in the logistic model as viewed through the lens of Strict-Local-Quasi-Convexity.

In the second part of the tutorial, we will highlight the robustness of this loss family to a variety of data corruptions (referred to as “twists”) including benign and adversarial feature and/or label noise. We do so using a new robustness framework called Twist-Properness. We will prove that the alpha-loss is more robust to symmetric label noise with a fixed hyperparameter than the oft-used log-loss. In addition, we also present an algorithmic contribution: a new robust boost algorithm to “properly” boost the alpha-loss called PILBoost (Pseudo-Inverse Link Boost). Indeed, for several datasets and twists, we show that PILBoost is more robust than XGBoost, which is a state-of-the-art boosting algorithm.

Finally, we will conclude by highlighting how the core information-theoretic properties of this loss function class allow it to unify a range of generative adversarial network (GAN) models. Here, we will show that a large class of GANs from the original (oft-called vanilla GAN) GAN to f-GANs to Wasserstein and other IPM GANs are captured by using alpha-loss to write the value function of GANs, and thus, present a mechanism to enable meaningful comparisons of GANs. Throughout the tutorial, the technical results will be accompanied by results on publicly available large datasets and deep learning models.

Lalitha Sankar is an Associate Professor in the School of Electrical, Computer, and Energy Engineering at Arizona State University. She received her doctorate from Rutgers University, her master’s from the University of Maryland, and her bachelor’s degree from the Indian Institute of Technology, Bombay. Her research at present is at the intersection of information science and learning theory including their applications to the electric grid. She received the NSF CAREER award in 2014. She currently leads both an NSF HDR institute on data analytics for the electric grid and an NSF-and Google-funded effort on predicting infectious disease exposure risk in a secure and privacy-preserving manner.