老师好，
今电工系顾晓东邀请美国普林斯顿大学S.Y. Kung教授来复旦访问，并做学术报告,欢迎感兴趣的老师和同学参加。
题目：MIND-Net: A Novel Learning Network
报告人： S.Y. Kung, Professor，Princeton University, Life Fellow of IEEE
时间：2018年6月25日（周一） 上午 10:00～11:30
地点：邯郸校区科学楼530会议室
联系人：顾晓东
Title: MIND-Net: A Novel Learning Network
S.Y. Kung, Professor , Princeton University
ABSTACT
The success of deep neural networks (DNN) hinges upon the rich nonlinear space embedded in their nonlinear hidden neuron layers. As to the weakness, the prevalent concerns over deep learning include two major fronts: one analytical and one structural. In this talk, we introduce a learning model called of MIND-Net with a new learning paradigm to Monotonically INcrease the Discriminative power (quantified by DI) of the classifying networks. It offers a learning tool to efficiently tackle both the analytical and structural concerns over deep learning networks.
From the analytical perspective, the ad hoc nature of deep learning renders its success at the mercy of trial-and-errors. To rectify this problem, we advocate a methodic learning paradigm, MIND-Net, which is computationally efficient in training the networks and yet mathematically feasible to analyze. MIND-Net hinges upon the use of an effective optimization metric, called Discriminant Information (DI). It will be used as a surrogate of the popular metrics such as 0-1 loss or prediction accuracy. Mathematically, DI is equivalent or closely related to Gauss’ LSE, Fisher’s FDR, and Shannon’s Mutual Information. We shall explain why is that higher DI means higher linear separability, i.e. higher DI means that the data are more discriminable. In fact, it can be shown that, both theoretically and empirically, a high DI score usually implies a high prediction accuracy.
In the structural front, the curse of depth it is widely recognized as a cause of serious concern. Fortunately, many solutions have been proposed to effectively combat or alleviate such a curse. Likewise, in our case, MIND-Net offers yet another cost-effective solution by circumventing the depth problem altogether via a new notion (or trick) of Omni-present Supervision(OS), i.e. teachers hidden a “Trojan-horse” being transported (along with the training data) from the input to each of the hidden layers. Opening up the Trojan-horse at any hidden-layer, we can have direct access to the teacher’s information for free, in the sense that no BP is incurred. In short, it amount to learning with no-propagation. By harnessing the teacher information, we will be able to construct a new and slender “inheritance layer” to summarize all the discriminant information amassed by the previous layer. Moreover, by horizontally augmenting the inheritance layer with additional randomized nodes and applying back-propagation (BP) learning, the discriminant power of to the newly augmented network will be further enhanced.
In our experiments, the MIND-Net was applied to synthetic and real-world datasets, e.g. CIFAR-10 dataset based on feature extracted from different layers of ResNets. The results generally support our theoretical prediction and yield some performance improvements
BIOGRAPHY
S.Y. Kung, Life Fellow of IEEE, is a Professor at Department of Electrical Engineering in Princeton University. His research areas include machine learning, data mining, systematic design of (deep-learning) neural networks, statistical estimation, VLSI array processors, signal and multimedia information processing, and most recently compressive privacy. He was a founding member of several Technical Committees (TC) of the IEEE Signal Processing Society. He was elected to Fellow in 1988 and served as a Member of the Board of Governors of the IEEE Signal Processing Society (1989-1991). He was a recipient of IEEE Signal Processing Society's Technical Achievement Award for the contributions on "parallel processing and neural network algorithms for signal processing" (1992); a Distinguished Lecturer of IEEE Signal Processing Society (1994); a recipient of IEEE Signal Processing Society's Best Paper Award for his publication on principal component neural networks (1996); and a recipient of the IEEE Third Millennium Medal (2000). Since 1990, he has been the Editor-In-Chief of the Journal of VLSI Signal Processing Systems. He served as the first Associate Editor in VLSI Area (1984) and the first Associate Editor in Neural Network (1991) for the IEEE Transactions on Signal Processing. He has authored and co-authored more than 500 technical publications and numerous textbooks including ``VLSI Array Processors'', Prentice-Hall (1988); ``Digital Neural Networks'', Prentice-Hall (1993) ; ``Principal Component Neural Networks'', John-Wiley (1996); ``Biometric Authentication: A Machine Learning Approach'', Prentice-Hall (2004); and ``Kernel Methods and Machine Learning”, Cambridge University Press (2014).
