- Jan 24: Max Vladymyrov - Miguel Carreira-Perpinan
- Jan 31: Miguel Carreira-Perpinan
- Feb 7: Polly Huang (Taiwan University/MIT) - Alberto Cerpa
- Feb 14: Gabriel Elkaim, UCSC - Stefano Carpin
- Feb 21: Anthony Rowe (CMU) - Alberto Cerpa
- Feb 28: Brian Price (Adobe) - Ming-Hsuan Yang
- Mar 7: no seminar
- Mar 14: Max Vladymyrov - Miguel Carreira-Perpinan
- Mar 21: Yu Cheng - Florin Rusu
- Apr 4: ??? - YangQuan Chen [backup: Jimei Yang - Ming-Hsuan Yang]
- Apr 11: Quoc Le (Google) - Miguel Carreira-Perpinan
- Apr 18: Chengjie Qin - Florin Rusu
- Apr 25: Xiaoyu Wang (NEC Research) - Ming-Hsuan Yang
- May 2: Ming Yang (Facebook) - Ming-Hsuan Yang
- May 9: Carlo Camporesi - Marcelo Kallmann
JAN 24 Constructing better and scalable affinity matrices for machine leaning algorithms
ABSTRACT
Many of the graph-based machine learning techniques, such as algorithms for manifold learning, semi-supervised learning and clustering, rely on the choice of the affinity matrix. This matrix provides the information about similarity between the points in the dataset and plays an essential role on the performance of the algorithms. In my talk, I will propose two methods that can be used for construction and scalability of the affinity matrices.
In the first part, I will present an algorithm to construct a better Gaussian affinity matrix. Traditionally, it is computed using a single bandwidth parameter that is not intuitive to set and often does not give good performance. Instead, we consider a model originally proposed by Hinton & Roweis (2003) that sets the bandwidth parameter individually for every point based on the entropy of the neighborhood distribution of the points in the dataset. We study the mathematical properties of these "entropic affinities" and show that they implicitly define a continuously differentiable function in the input space and give bounds for it. We then devise a fast algorithm to compute the bandwidth and affinities to a very high accuracy.
In the second part, I will address the problem of scalability of spectral methods for dimensionality reduction and clustering. Those methods typically construct the affinity matrix from a dataset and compute eigenvectors of a related matrix in order to get the solution. With large datasets, this eigendecomposition is too expensive, and is usually approximated by solving for a smaller graph defined on a subset of the points (landmarks) and then applying the Nystrom formula to estimate the eigenvectors over all points. This has the problem that the affinities between landmarks do not benefit from the remaining points and may poorly represent the data if using few landmarks. We introduce a modified spectral problem that uses all data points by constraining the latent projection of each point to be a local linear function of the landmarks' latent projections. This constructs a new affinity matrix between landmarks that preserves manifold structure even with few landmarks and allows one to reduce the eigenproblem size.
This is joint work with Miguel A. Carreira-Perpinan.
BIOGRAPHY
Max Vladymyrov is a PhD candidate in the EECS Department of UC Merced. He received his BS in Applied Mathematics in 2007 and MS in Computer Science in 2008, both from Kharkiv National University, Ukraine. Since 2009, he has been working towards his PhD degree in machine learning. His research focuses on nonlinear dimensionality reduction and its applications.
JAN 31Training nested functions using auxiliary coordinates
ABSTRACT
Many models in machine learning, computer vision or speech processing have the form of a sequence of nested, parameterized functions, such as a multilayer neural net, an object recognition pipeline, or a "wrapper" for feature selection. Joint estimation of the parameters of all the layers and selection of an optimal architecture is widely considered to be a difficult numerical nonconvex optimization problem, difficult to parallelize for execution in a distributed computation environment, and requiring significant human expert effort, which leads to suboptimal systems in practice. We describe a general mathematical strategy to learn the parameters and, to some extent, the architecture of nested systems, called the method of auxiliary coordinates (MAC). MAC has provable convergence, is easy to implement reusing existing algorithms for single layers, can be parallelized trivially and massively, applies even when parameter derivatives are not available or not desirable (so computing gradients with the chain rule does not apply), and is competitive with state-of-the-art nonlinear optimizers even in the serial computation setting, often providing reasonable models within a few iterations.
If time permits, I will illustrate how to use MAC to derive training algorithms for a range of problems, such as deep nets, best-subset feature selection, joint dictionary and classifier learning, supervised dimensionality reduction, and others.
This is joint work with Weiran Wang.
BIOGRAPHY
Miguel Á. Carreira-Perpiñán is an associate professor in Electrical Engineering and Computer Science at the University of California, Merced. He received the degree of "licenciado en informática" (MSc in computer science) from the Technical University of Madrid in 1995 and a PhD in computer science from the University of Sheffield in 2001. Prior to joining UC Merced, he did postdoctoral work at Georgetown University (in computational neuroscience) and the University of Toronto (in machine learning), and was an assistant professor at the Oregon Graduate Institute (Oregon Health & Science University). He is the recipient of an NSF CAREER award, a Google Faculty Research Award and a best student paper award at Interspeech. He is an associate editor for the IEEE Transactions on Pattern Analysis and Machine Intelligence and an area chair for NIPS. His research interests lie in machine learning, in particular unsupervised learning problems such as dimensionality reduction, clustering and denoising, with an emphasis on optimization aspects, and with applications to speech processing (e.g. articulatory inversion and model adaptation), computer vision, sensor networks and other areas.
FEB 7 User-Centric Measurement, Modeling, and Control of Skype/SILK VoIP Calls
Abstract
As the proportion of the multimedia traffic over the Internet rises and the world economy recovers slowly, the issue of streaming Voice/Video content content cost-effectively is ever more pressing. The key question to address here is how to satisfy more (paying) users given limited resources. Users switch to other providers/services because they can't hear/see the content well, not because they detect the fine changes in network loss, delay or jitter, so called Quality of Service (QoS). Over the years, the Internet engineers, although getting very good at designing for QoS, have overlooked the fact that users might not perceive fine changes in QoS, nor the subtle quantitative difference of QoS metrics to the overall user experience.
Towards a user-friendly, therefore economically healthy, Internet, we see the need to measure, understand, and redesign various control mechanisms for quality of user experience (QoE), in addition to the QoS. Using Skype/SILK VoIP service as an example, we show how we (1) measure QoE of calls delivered of different QoS, (2) derive models that translate from QoS to QoE, and (3) exploit the model for a design that pleases the users more.
Bio
As a person, Polly Huang is a nerd and a geek. She enjoys observing systems of complexity and loves the process of finding out their true nature. Professionally, she is a professor at National Taiwan University and a visiting scientist at MIT. She swings back and forth among three research interests -- multimedia networking, sensor networking, and mobile computing. Much of the work has been reported in the 100 technical papers and 9 US patents she's co-authored. In 2014, she'll serve as a TPC member of 4 major network and system conferences -- Mobisys'14, Sigcomm'14, Mobicom'14, and Ubicomp'14.
FEB 14The SLUGS Autopilot: A Flexible, Low-Cost Autopilot for R&D Applications
Speaker: Gabriel Hugh Elkaim, Professor, Autonomous Systems Lab, Computer Engineering, UC Santa Cruz
Abstract:
The Santa Cruz Low-Cost UAV GNC Subsystem (SLUGS) autopilot was developed at the University of California Santa Cruz over the past five years. It is a versatile and flexible autopilot capable of controlling a small unmanned system. It is tightly integrated with MATLAB/Simulink, and allows for a simple and easy transition from pure simulation to Hardwar-in-the-Loop (HIL) simulation to flight code. The hardware's main processing units are two low-cost dsPIC33F DSCs, one handling sensor input and the other managing the navigation and control loops. The sensor DSC implements a complementary attitude and position filter. An interprocessor communication protocol delivers fused attitude and position estimates to the control DSC. The control DSC implements low-level platform stabilization using PID loops, and higher level waypoint following based on a line-of-sight guidance law. Data is logged and available for replay post flight on both the ground station software, and also within MATLAB/Simulink. The hardware was installed on a low-cost single engine electric RC aircraft, and has been demonstrated to be capable of sustained autonomous flight. Several estimation topologies have been tested and developed using the system. The SLUGS is general, and can be adapted to multiple autonomous platforms such as helicopters, quadrotors, twin engine aircraft, ground, and marine surface vehicles.
The SLUGS implements a new guidance law that extends the line-of-sight guidance law previously developed by Park et al. Several improvements are presented that allow operation in the real world. A stability analysis accounts for the dynamic response of the bank angle which leads to the definition of regions of instability. Another extension accounts for situations where the cross-track error is larger than the look-ahead distance. Another modifies the look-ahead distance so that the transient response is independent of ground speed. Yet another extension defines a “homing” mode in which the UAV flies to a goal point without a defined path, commonly used as a “return-to-base,” either as a safety measure or as an end-of-mission order. Since there is no constraint that the goal point be stationary, we demonstrate that the new law can be used to follow a moving target whose location is known, such as a mobile ground control station. Simulations with a 6 degree of freedom aircraft model demonstrate these features, and experimental flight data show the same algorithm operating in all modes of flight.
Bio:
Gabriel Elkaim received MS and PhD degrees in Aeronautics and Astronautics from Stanford University in 1995 and 2002. He joined the Computer Engineering Faculty at UC Santa Cruz in 2003. At UCSC he established the Autonomous Systems Lab, and he is founding a faculty of the recently established Robotics Engineering degree program. His research is primarily on applied control systems with a focus on Autonomous and Embedded Systems. His research interests includes Guidance, Navigation and Control (GNC), GPS research, Sensor Fusion, Attitude Estimation, System Identification, and Robust Software Design for Real-Time Reactive Systems.
FEB 21 Sensor Andrew: Sensing and Connecting the Physical World
Speaker: Anthony Rowe, CMU
Abstract:
We are beginning to see an information technology transformation away from computers dependent on human operators for all of their data. Fueled by advances in low-cost, low-power computing and communication platforms, systems are emerging where devices interact directly with each other and the physical environment. This new autonomy and scale has the potential to revolutionize application domains ranging from critical infrastructure monitoring, health care, transportation, and defense systems to mobile robotics, manufacturing, smart buildings and citywide energy optimization. The Sensor Andrew project at Carnegie Mellon University is an ongoing effort to help developers manage and create applications for these types of large-scale Internet connected sensor-actuator networks. The goals of Sensor Andrew are to support ubiquitous large-scale monitoring and control of infrastructure in a way that is extensible, easy-to-use, and provides security while maintaining privacy. At its core, Sensor Andrew uses a federated publish-subscribe model running on top of the eXtensible Messaging and Presence Protocol (XMPP) developed originally for real-time chat. The historical data system provides a cloud-to-edge dispatching layer on top of an efficient multi-resolution time series DB such that high-resolution data can be collected on embedded edge-routers while cached aggregates are stored in the cloud.
In this talk, I will discuss the system architecture as well as the associated schemas used to both describe and deliver transducer data. I will then walk through an example of how to create, register and visualize data from a transducer using the Sensor Andrew web portal. Finally, I will discuss an ongoing building energy management project on campus that highlights challenges related to plug-and-play and programmability.
Bio:
Anthony Rowe
FEB 28 Image Matting
Abstract:
Image matting is the process of computing an accurate selection or mask of an object in a image. This allows effects to be applied locally to an object, or allows the object to be extracted from the image and composited in a new image. In this talk, I will go over the basics of matting as well as recent research by myself and collaborators. I hope to present the material not only from an academic point of view, but also show the practical importance from an industrial point of view as well.
Bio:
Dr. Brian Price is a Research Scientist at Adobe Research specializing in computer vision. He received his PhD degree in Computer Science from Brigham Young University in 2010. His research interests include image and video segmentation and matting, semantic segmentation, saliency, interactive vision methods, stereo, and other computer vision fields, and well as broad general interest in computer graphics and machine learning.
MAR 7
No seminar
MAR 14 Linear-time training of nonlinear low-dimensional embeddings using N-Body approximation algorithms
ABSTRACT
High-dimensional data representation is an important problem in many different areas of science. Nowadays, it is becoming crucial to interpret the data of varying dimensionality correctly. Dimensionality reduction methods process the data in order to help the visualization, complexity reduction, or a search for latent representation of the original problem. The algorithms of nonlinear dimensionality reduction (also known as manifold learning) preserve the structure of the high-dimensional data better than linear or spectral methods, however they suffer from expensive quadratic runtime on the number of points, which limits their practical use to small-scale dataset.
In my talk, I will start by reviewing most popular nonlinear dimensionality reduction methods and analyze their computational cost. I will show that these methods can be seen as a part of more general N-Body problems framework, where the main bottleneck consists in computing the mutual interactions between N points. In a particular case when desired low-dimensional representation is low, these interactions can be approximated by fast methods, such as Barnes-Hut and Fast Multipole Methods, that reduce the quadratic computation to much more benign O(N log(N)) or even linear cost. I will provide a extensive review of two of these methods, study the effect, in theory and experiment, of these approximations and show that the expected error is related to the mean curvature of the objective function.
Finally, I'll propose a simple, yet effective, strategy that indicates that gradually increasing the accuracy level of the approximations over iterations leads to a faster training. When combined with standard optimizers, such as gradient descent or L-BFGS, the resulting algorithm beats exact methods by two to three orders of magnitude and allows to scale up the application of the methods to datasets with million or more points.
This is joint work with Miguel A. Carreira-Perpinan, to appear at AISTATS 2014.
BIOGRAPHY
Max Vladymyrov is a PhD candidate in the EECS Department of UC Merced. He received his BS in Applied Mathematics in 2007 and MS in Computer Science in 2008, both from Kharkiv National University, Ukraine. Since 2009, he has been working towards his PhD degree in machine learning. His research focuses on nonlinear dimensionality reduction and its applications.
APR 25 Generic Object Detection: Features and Models
ABSTRACT
Object detection is a challenging task due to deformation, multiple viewpoints, rich sub-categories. There have been several breakthroughs in recent years which significantly improve the generic object detection accuracy. These works include Deformable Part-based Models, detection with selective search, detection with a deep convolutional neural network. In this talk, we discuss about these approaches and introduce the Regionlets framework. It is a flexible framework with data-driven handling of those challenges in object detection. We also show how a deep convolutional neural network is incorporated into the Regionlets detection framework. The Regionlets approach achieves the state-of-the-art performance: 49.3% mean average precision on the PASCAL VOC 2007 dataset.
BIOGRAPHY
Dr. Xiaoyu Wang is a Research Scientist in NEC Laboratories America. He obtained his Ph.D. in ECE and M.A. in Statistics from University of Missouri in 2012. Before that, he was a software engineer in a startup company CREARO, leading R&D of wireless surveillance products. At the same time, he was a research assistant in University of Science and Technology of China (USTC) from 2006 to 2008. He obtained his Bachelor in Electrical Engineering and Information Science from USTC in 2006. Dr. Wang's research is mainly focused on computer vision with specific interest in object detection and large scale image retrieval, machine learning with specific interest in deep learning, boosting and transfer learning. He is the Runner-up Winner of the ImageNet Large Scale Visual Object Recognition Challenge in 2013 (ILSVRC2013). He is currently appointed the vice president of USTC Silicon Valley.
