Abstract: This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthezing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.

@InProceedings{Li_2016_CVPR,author = {Li, Chuan and Wand, Michael},title = {Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hao Chen, Xiaojuan Qi, Lequan Yu, Pheng-Ann Heng

Abstract: The morphology of glands has been used routinely by pathologists to assess the malignancy degree of adenocarcinomas. Accurate segmentation of glands from histology images is a crucial step to obtain reliable morphological statistics for quantitative diagnosis. In this paper, we proposed an efficient deep contour-aware network (DCAN) to solve this challenging problem under a unified multi-task learning framework. In the proposed network, multi-level contextual features from the hierarchical architecture are explored with auxiliary supervision for accurate gland segmentation. When incorporated with multi-task regularization during the training, the discriminative capability of intermediate features can be further improved. Moreover, our network can not only output accurate probability maps of glands, but also depict clear contours simultaneously for separating clustered objects, which further boosts the gland segmentation performance. This unified framework can be efficient when applied to large-scale histopathological data without resorting to additional steps to generate contours based on low-level cues for post-separating. Our method won the 2015 MICCAI Gland Segmentation Challenge out of 13 competitive teams, surpassing all the other methods by a significant margin.

@InProceedings{Chen_2016_CVPR,author = {Chen, Hao and Qi, Xiaojuan and Yu, Lequan and Heng, Pheng-Ann},title = {DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hoo-Chang Shin, Kirk Roberts, Le Lu, Dina Demner-Fushman, Jianhua Yao, Ronald M. Summers

Abstract: Despite the recent advances in automatically describing image contents, their applications have been mostly limited to image caption datasets containing natural images (e.g., Flickr 30k, MSCOCO). In this paper, we present a deep learning model to efficiently detect a disease from an image and annotate its contexts (e.g., location, severity and the affected organs). We employ a publicly available radiology dataset of chest x-rays and their reports, and use its image annotations to mine disease names to train convolutional neural networks (CNNs). In doing so, we adopt various regularization techniques to circumvent the large normal-vs-diseased cases bias. Recurrent neural networks (RNNs) are then trained to describe the contexts of a detected disease, based on the deep CNN features. Moreover, we introduce a novel approach to use the weights of the already trained pair of CNN/RNN on the domain-specific image/text dataset, to infer the joint image/text contexts for composite image labeling. Significantly improved image annotation results are demonstrated using the recurrent neural cascade model by taking the joint image/text contexts into account.

@InProceedings{Shin_2016_CVPR,author = {Shin, Hoo-Chang and Roberts, Kirk and Lu, Le and Demner-Fushman, Dina and Yao, Jianhua and Summers, Ronald M.},title = {Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: Deformations of surfaces with the same intrinsic shape can often be described accurately by a conformal model. A major focus of computational conformal geometry is the estimation of the conformal mapping that aligns a given pair of object surfaces. The uniformization theorem en- ables this task to be acccomplished in a canonical 2D do- main, wherein the surfaces can be aligned using a M obius transformation. Current algorithms for estimating M obius transformations, however, often cannot provide satisfactory alignment or are computationally too costly. This paper in- troduces a novel globally optimal algorithm for estimating Mobius transformations to align surfaces that are topologi- cal discs. Unlike previous methods, the proposed algorithm deterministically calculates the best transformation, with- out requiring good initializations. Further, our algorithm is also much faster than previous techniques in practice. We demonstrate the efficacy of our algorithm on data commonly used in computational conformal geometry.

@InProceedings{Le_2016_CVPR,author = {Le, Huu and Chin, Tat-Jun and Suter, David},title = {Conformal Surface Alignment With Optimal Mobius Search},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Seong Jae Hwang, Nagesh Adluru, Maxwell D. Collins, Sathya N. Ravi, Barbara B. Bendlin, Sterling C. Johnson, Vikas Singh

Abstract: There is a great deal of interest in using large scale brain imaging studies to understand how brain connectivity evolves over time for an individual and how it varies over different levels/quantiles of cognitive function. To do so, one typically performs so-called tractography procedures on diffusion MR brain images and derives measures of brain connectivity expressed as graphs. The nodes correspond to distinct brain regions and the edges encode the strength of the connection. The scientific interest is in characterizing the evolution of these graphs over time or from healthy individuals to diseased. We pose this important question in terms of the Laplacian of the connectivity graphs derived from various longitudinal or disease time points - quantifying its progression is then expressed in terms of coupling the harmonic bases of a full set of Laplacians. We derive a coupled system of generalized eigenvalue problems (and corresponding numerical optimization schemes) whose solution helps characterize the full life cycle of brain connectivity evolution in a given dataset. Finally, we show a set of results on a diffusion MR imaging dataset of middle aged people at risk for Alzheimer's disease (AD), who are cognitively healthy. In such asymptomatic adults, we find that a framework for characterizing brain connectivity evolution provides the ability to predict cognitive scores for individual subjects, and for estimating the progression of participant's brain connectivity into the future.

@InProceedings{Hwang_2016_CVPR,author = {Hwang, Seong Jae and Adluru, Nagesh and Collins, Maxwell D. and Ravi, Sathya N. and Bendlin, Barbara B. and Johnson, Sterling C. and Singh, Vikas},title = {Coupled Harmonic Bases for Longitudinal Characterization of Brain Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Jae Shin, Nima Tajbakhsh, R. Todd Hurst, Christopher B. Kendall, Jianming Liang

Abstract: Cardiovascular disease (CVD) is the leading cause of mortality yet largely preventable, but the key to prevention is to identify at risk individuals before adverse events. For predicting individual CVD risk, carotid intima-media thickness (CIMT), a noninvasive ultrasound method, has proven to be valuable, offering several advantages over CT coronary artery calcium score. However, each CIMT examination includes several ultrasound videos, and interpreting each of these CIMT videos involves three operations: (1) select three end-diastolic ultrasound frames (EUF) in the video, (2) localize a region of interest (ROI) in each selected frame, and (3) trace the lumen-intima interface and the media-adventitia interface in each ROI to measure CIMT. These operations are tedious, laborious, and time consuming, a serious limitation that hinders the widespread utilization of CIMT in clinical practice. To overcome this limitation, this paper presents a new system to automate CIMT video interpretation. Our extensive experiments demonstrate that the suggested system significantly outperforms the state-of-the-art methods. The superior performance is attributable to our unified framework based on convolutional neural networks (CNNs) coupled with our informative image representation and effective post-processing of the CNN outputs, which are uniquely designed for each of the above three operations.

@InProceedings{Shin_2016_CVPR,author = {Shin, Jae and Tajbakhsh, Nima and Hurst, R. Todd and Kendall, Christopher B. and Liang, Jianming},title = {Automating Carotid Intima-Media Thickness Video Interpretation With Convolutional Neural Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, Alexei A. Efros

Abstract: We present an unsupervised visual feature learning algorithm driven by context-based pixel prediction. By analogy with auto-encoders, we propose Context Encoders -- a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings. In order to succeed at this task, context encoders need to both understand the content of the entire image, as well as produce a plausible hypothesis for the missing part(s). When training context encoders, we have experimented with both a standard pixel-wise reconstruction loss, as well as a reconstruction plus an adversarial loss.  The latter produces much sharper results because it can better handle multiple modes in the output. We found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures. We quantitatively demonstrate the effectiveness of our learned features for CNN pre-training on classification, detection, and segmentation tasks. Furthermore, context encoders can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.

@InProceedings{Pathak_2016_CVPR,author = {Pathak, Deepak and Krahenbuhl, Philipp and Donahue, Jeff and Darrell, Trevor and Efros, Alexei A.},title = {Context Encoders: Feature Learning by Inpainting},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Chenyi Lei, Dong Liu, Weiping Li, Zheng-Jun Zha, Houqiang Li

Abstract: In many image-related tasks, learning expressive and discriminative representations of images is essential, and deep learning has been studied for automating the learning of such representations. Some user-centric tasks, such as image recommendations, call for effective representations of not only images but also preferences and intents of users over images. Such representations are termed hybrid and addressed via a deep learning approach in this paper. We design a dual-net deep network, in which the two sub-networks map input images and preferences of users into a same latent semantic space, and then the distances between images and users in the latent space are calculated to make decisions. We further propose a comparative deep learning (CDL) method to train the deep network, using a pair of images compared against one user to learn the pattern of their relative distances. The CDL embraces much more training data than naive deep learning, and thus achieves superior performance than the latter, with no cost of increasing network complexity. Experimental results with real-world data sets for image recommendations have shown the proposed dual-net network and CDL greatly outperform other state-of-the-art image recommendation solutions.

@InProceedings{Lei_2016_CVPR,author = {Lei, Chenyi and Liu, Dong and Li, Weiping and Zha, Zheng-Jun and Li, Houqiang},title = {Comparative Deep Learning of Hybrid Representations for Image Recommendations},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: We revisit the idea of brain damage, i.e. the pruning of the coefficients of a neural network, and suggest how brain damage can be modified and used to speedup convolutional layers in ConvNets. The approach uses the fact that many efficient implementations reduce generalized convolutions to matrix multiplications. The suggested brain damage process prunes the convolutional kernel tensor in a group-wise fashion. After such pruning, convolutions can be reduced to multiplications of thinned dense matrices, which leads to speedup. We investigate different ways to add group-wise prunning to the learning process, and show that several-fold speedups of convolutional layers can be attained using group-sparsity regularizers. Our approach can adjust the shapes of the receptive fields in the convolutional layers, and even prune excessive feature maps from ConvNets, all in a data-driven way.

@InProceedings{Lebedev_2016_CVPR,author = {Lebedev, Vadim and Lempitsky, Victor},title = {Fast ConvNets Using Group-Wise Brain Damage},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Zeeshan Hayder, Xuming He, Mathieu Salzmann

Abstract: Generating object proposals has become a key component of modern object detection pipelines. However, most existing methods generate the object candidates independently of each other. In this paper, we present an approach to co-generating object proposals in multiple images, thus leveraging the collective power of multiple object candidates. In particular, we introduce a deep structured network that jointly predicts the objectness scores and the bounding box locations of multiple object candidates. Our deep structured network consists of a fully-connected Conditional Random Field built on top of a set of deep Convolutional Neural Networks, which learn features to model both the individual object candidate and the similarity between multiple candidates. To train our deep structured network, we develop an end-to-end learning algorithm that, by unrolling the CRF inference procedure, lets us backpropagate the loss gradient throughout the entire structured network. We demonstrate the effectiveness of our approach on two benchmark datasets, showing significant improvement over state-of-the-art object proposal algorithms.

@InProceedings{Hayder_2016_CVPR,author = {Hayder, Zeeshan and He, Xuming and Salzmann, Mathieu},title = {Learning to Co-Generate Object Proposals With a Deep Structured Network},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Pascal Frossard

Abstract: State-of-the-art deep neural networks have achieved impressive results on many image classification tasks. However, these same architectures have been shown to be unstable to small, well sought, perturbations of the images. Despite the importance of this phenomenon, no effective methods have been proposed to accurately compute the robustness of state-of-the-art deep classifiers to such perturbations on large-scale datasets. In this paper, we fill this gap and propose the DeepFool algorithm to efficiently compute perturbations that fool deep networks, and thus reliably quantify the robustness of these classifiers. Extensive experimental results show that our approach outperforms recent methods in the task of computing adversarial perturbations and making classifiers more robust.

@InProceedings{Moosavi-Dezfooli_2016_CVPR,author = {Moosavi-Dezfooli, Seyed-Mohsen and Fawzi, Alhussein and Frossard, Pascal},title = {DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Calvin Murdock, Zhen Li, Howard Zhou, Tom Duerig

Abstract: Most deep architectures for image classification--even those that are trained to classify a large number of diverse categories--learn shared image representations with a single model. Intuitively, however, categories that are more similar should share more information than those that are very different. While hierarchical deep networks address this problem by learning separate features for subsets of related categories, current implementations require simplified models using fixed architectures specified via heuristic clustering methods. Instead, we propose Blockout, a method for regularization and model selection that simultaneously learns both the model architecture and parameters. A generalization of Dropout, our approach gives a novel parametrization of hierarchical architectures that allows for structure learning via back-propagation. To demonstrate its utility, we evaluate Blockout on the CIFAR and ImageNet datasets, demonstrating improved classification accuracy, better regularization performance, faster training, and the clear emergence of hierarchical network structures.

@InProceedings{Murdock_2016_CVPR,author = {Murdock, Calvin and Li, Zhen and Zhou, Howard and Duerig, Tom},title = {Blockout: Dynamic Model Selection for Hierarchical Deep Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Kurt Keutzer

Abstract: Long training times for high-accuracy deep neural networks (DNNs) impede research into new DNN architectures and slow the development of high-accuracy DNNs. In this paper we present FireCaffe, which successfully scales deep neural network training across a cluster of GPUs. We also present a number of best practices to aid in comparing advancements in methods for scaling and accelerating the training of deep neural networks. The speed and scalability of distributed algorithms is almost always limited by the overhead of communicating between servers; DNN training is not an exception to this rule. Therefore, the key consideration here is to reduce communication overhead wherever possible, while not degrading the accuracy of the DNN models that we train. Our approach has three key pillars. First, we select network hardware that achieves high bandwidth between GPU servers -- Infiniband or Cray interconnects are ideal for this. Second, we consider a number of communication algorithms, and we find that reduction trees are more efficient and scalable than the traditional parameter server approach. Third, we optionally increase the batch size to reduce the total quantity of communication during DNN training, and we identify hyperparameters that allow us to reproduce the small-batch accuracy while training with large batch sizes. When training GoogLeNet and Network-in-Network on ImageNet, we achieve a 47x and 39x speedup, respectively, when training on a cluster of 128 GPUs.

@InProceedings{Iandola_2016_CVPR,author = {Iandola, Forrest N. and Moskewicz, Matthew W. and Ashraf, Khalid and Keutzer, Kurt},title = {FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Sarah Rastegar, Mahdieh Soleymani, Hamid R. Rabiee, Seyed Mohsen Shojaee

Abstract: Deep learning has received much attention as of the most powerful approaches for multimodal representation learning in recent years. An ideal model for multimodal data can reason about missing modalities using the available ones, and usually provides more information when multiple modalities are being considered. All the previous deep models contain separate modality-specific networks and find a shared representation on top of those networks. Therefore, they only consider high level interactions between modalities to find a joint representation for them. In this paper, we propose a multimodal deep learning framework (MDL-CW) that exploits the cross weights between representation of modalities, and try to gradually learn interactions of the modalities in a deep network manner (from low to high level interactions). Moreover, we theoretically show that considering these interactions provide more intra-modality information, and introduce a multi-stage pre-training method that is based on the properties of multi-modal data. In the proposed framework, as opposed to the existing deep methods for multi-modal data, we try to reconstruct the representation of each modality at a given level, with representation of other modalities in the previous layer. Extensive experimental results show that the proposed model outperforms state-of-the-art information retrieval methods for both image and text queries on the PASCAL-sentence and SUN-Attribute databases.

@InProceedings{Rastegar_2016_CVPR,author = {Rastegar, Sarah and Soleymani, Mahdieh and Rabiee, Hamid R. and Shojaee, Seyed Mohsen},title = {MDL-CW: A Multimodal Deep Learning Framework With Cross Weights},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Structured Receptive Fields in CNNs

Jorn-Henrik Jacobsen, Jan van Gemert, Zhongyu Lou, Arnold W. M. Smeulders

Abstract: Learning powerful feature representations with CNNs is hard when training data are limited. Pre-training is one way to overcome this, but it requires large datasets sufficiently similar to the target domain. Another option is to design priors into the model, which can range from tuned hyperparameters to fully engineered representations like Scattering Networks. We combine these ideas into structured receptive field networks, a model which has a fixed filter basis and yet retains the flexibility of CNNs. This flexibility is achieved by expressing receptive fields in CNNs as a weighted sum over a fixed basis which is similar in spirit to Scattering Networks. The key difference is that we learn arbitrary effective filter sets from the basis rather than modeling the filters. This approach explicitly connects classical multiscale image analysis with general CNNs. With structured receptive field networks, we improve considerably over unstructured CNNs for small and medium dataset scenarios as well as over Scattering for large datasets. We validate our findings on ILSVRC2012, Cifar-10, Cifar-100 and MNIST. As a realistic small dataset example, we show state-of-the-art classification results on popular 3D MRI brain-disease datasets where pre-training is difficult due to a lack of large public datasets in a similar domain.

@InProceedings{Jacobsen_2016_CVPR,author = {Jacobsen, Jorn-Henrik and van Gemert, Jan and Lou, Zhongyu and Smeulders, Arnold W. M.},title = {Structured Receptive Fields in CNNs},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Suriya Singh, Chetan Arora, C. V. Jawahar

Abstract: We focus on the problem of wearer's action recognition in first person a.k.a. egocentric videos. This problem is more challenging than third person activity recognition due to unavailability of wearer's pose and sharp movements in the videos caused by the natural head motion of the wearer. Carefully crafted features based on hands and objects cues for the problem have been shown to be successful for limited targeted datasets. We propose convolutional neural networks (CNNs) for end to end learning and classification of wearer's actions. The proposed network makes use of egocentric cues by capturing hand pose, head motion and saliency map. It is compact. It can also be trained from relatively small number of labeled egocentric videos that are available. We show that the proposed network can generalize and give state of the art performance on various disparate egocentric action datasets.

@InProceedings{Singh_2016_CVPR,author = {Singh, Suriya and Arora, Chetan and Jawahar, C. V.},title = {First Person Action Recognition Using Deep Learned Descriptors},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Ryo Yonetani, Kris M. Kitani, Yoichi Sato

Abstract: We aim to understand the dynamics of social interactions between two people by recognizing their actions and reactions using a head-mounted camera. Our work will impact several first-person vision tasks that need the detailed understanding of social interactions, such as automatic video summarization of group events and assistive systems. To recognize micro-level actions and reactions, such as slight shifts in attention, subtle nodding, or small hand actions, where only subtle body motion is apparent, we propose to use paired egocentric videos recorded by two interacting people. We show that the first-person and second-person points-of-view features of two people, enabled by paired egocentric videos, are complementary and essential for reliably recognizing micro-actions and reactions. We also build a new dataset of dyadic (two-persons) interactions that comprises more than 1000 pairs of egocentric videos to enable systematic evaluations on the task of micro-action and reaction recognition.

@InProceedings{Yonetani_2016_CVPR,author = {Yonetani, Ryo and Kitani, Kris M. and Sato, Yoichi},title = {Recognizing Micro-Actions and Reactions From Paired Egocentric Videos},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Chunyu Wang, Yizhou Wang, Alan L. Yuille

Abstract: Recognizing an action from a sequence of 3D skeletal poses is a challenging task. First, different actors may perform the same action in various performing styles. Second, the estimated poses are sometimes inaccurate due to sensory noises. These challenges can cause large variations between instances of the same class. Third, the datasets are usually small, with only a few actors performing few repetitions of each action. Hence training complex classifiers risks over-fitting the data. We address this task by mining a set of key-pose-motifs for each action class. A key-pose-motif contains a set of ordered  poses or  action units(a short sequence of poses), which are required to be close but not necessarily adjacent in the action sequences. The representation is robust to style variations and outlier poses. The key-pose-motifs are represented in terms of a dictionary using soft-quantization (probabilities) to deal with inaccuracies caused by quantization. We propose an efficient algorithm to mine key-pose-motifs taking into account these probabilities. We classify a sequence by matching it to the motifs of each class and select the class that maximizes the matching score. This simple classifier obtains state-of-the-art performance on two benchmark datasets and outperforms a deep network approach.

@InProceedings{Wang_2016_CVPR,author = {Wang, Chunyu and Wang, Yizhou and Yuille, Alan L.},title = {Mining 3D Key-Pose-Motifs for Action Recognition},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Actions ~ Transformations

Xiaolong Wang, Ali Farhadi, Abhinav Gupta

Abstract: What defines an action like "kicking ball"? We argue that the true meaning of an action lies in the change or transformation an action brings to the environment. In this paper, we propose a novel representation for actions by modeling an action as a transformation which changes the state of the environment before the action happens (precondition) to the state after the action (effect). Motivated by recent advancements of video representation using deep learning, we design a Siamese network which models the action as a transformation on a high-level feature space. We show that our model gives improvements on standard action recognition datasets including UCF101 and HMDB51. More importantly, our approach is able to  generalize beyond learned action categories and shows significant performance improvement on cross-category generalization on our new ACT dataset.

@InProceedings{Wang_2016_CVPR,author = {Wang, Xiaolong and Farhadi, Ali and Gupta, Abhinav},title = {Actions ~ Transformations},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Khurram Soomro, Haroon Idrees, Mubarak Shah

Abstract: This paper proposes a novel approach to tackle the challenging problem of 'online action localization' which entails predicting actions and their locations as they happen in a video. Typically, action localization or recognition is performed in an offline manner where all the frames in the video are processed together and action labels are not predicted for the future. This dis-allows timely localization of actions - an important consideration for surveillance tasks. In our approach, given a batch of frames from the immediate past in a video, we estimate pose and over- segment the current frame into superpixels. Next, we discriminatively train an actor foreground model on the superpixels using the pose bounding boxes. A Conditional Random Field with superpixels as nodes, and edges connecting spatio-temporal neighbors is used to obtain action segments. The action confidence is predicted using dynamic programming on SVM scores obtained on short segments of the video, thereby capturing sequential information of the actions. The issue of visual drift is handled by updating the appearance model and pose refinement in an online manner. Lastly, we introduce a new measure to quantify the performance of action prediction (i.e. online action localization), which analyzes how the prediction accuracy varies as a function of observed portion of the video. Our experiments suggest that despite using only a few frames to localize actions at each time instant, we are able to predict the action and obtain competitive results to state-of-the-art offline methods.

@InProceedings{Soomro_2016_CVPR,author = {Soomro, Khurram and Idrees, Haroon and Shah, Mubarak},title = {Predicting the Where and What of Actors and Actions Through Online Action Localization},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


YoungJoon Yoo, Kimin Yun, Sangdoo Yun, JongHee Hong, Hawook Jeong, Jin Young Choi

Abstract: This paper proposes a novel path prediction algorithm for progressing one step further than the existing works focusing on single target path prediction. In this paper, we consider moving dynamics of co-occurring objects for path prediction in a scene that includes crowded moving objects. To solve this problem, we first suggest a two-layered probabilistic model to find major movement patterns and their co-occurrence tendency. By utilizing the unsupervised learning results from the model, we present an algorithm to find the future location of any target object. Through extensive qualitative/quantitative experiments, we show that our algorithm can find a plausible future path in complex scenes with a large number of moving objects.

@InProceedings{Yoo_2016_CVPR,author = {Yoo, YoungJoon and Yun, Kimin and Yun, Sangdoo and Hong, JongHee and Jeong, Hawook and Choi, Jin Young},title = {Visual Path Prediction in Complex Scenes With Crowded Moving Objects},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Serena Yeung, Olga Russakovsky, Greg Mori, Li Fei-Fei

Abstract: In this work we introduce a fully end-to-end approach for action detection in videos that learns to directly predict the temporal bounds of actions. Our intuition is that the process of detecting actions is naturally one of observation and refinement: observing moments in video, and refining hypotheses about when an action is occurring. Based on this insight, we formulate our model as a recurrent neural network-based agent that interacts with a video over time. The agent observes video frames and decides both where to look next and whether to emit a prediction. Since backpropagation is not adequate in this non-differentiable setting, we use REINFORCE to learn the agent's task-specific decision policy. Our model achieves state-of-the-art results on the THUMOS'14 and ActivityNet datasets while observing only a fraction (2% or less) of the video frames.

@InProceedings{Yeung_2016_CVPR,author = {Yeung, Serena and Russakovsky, Olga and Mori, Greg and Fei-Fei, Li},title = {End-To-End Learning of Action Detection From Frame Glimpses in Videos},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: This work presents an approach to category-based action recognition in video using sparse coding techniques. The proposed approach includes two main contributions: i) A new method to handle intra-class variations by decomposing each video into a reduced set of representative atomic action acts or key-sequences, and ii) A new video descriptor, ITRA: Inter-Temporal Relational Act Descriptor, that exploits the power of comparative reasoning to capture relative similarity relations among key-sequences. In terms of the method to obtain key-sequences, we introduce a loss function that, for each video, leads to the identification of a sparse set of representative key-frames capturing both, relevant particularities arising in the input video, as well as relevant generalities arising in the complete class collection. In terms of the method to obtain the ITRA descriptor, we introduce a novel scheme to quantify relative intra and inter-class similarities among local temporal patterns arising in the videos. The resulting ITRA descriptor demonstrates to be highly effective to discriminate among action categories. As a result, the proposed approach reaches remarkable action recognition performance on several popular benchmark datasets, outperforming alternative state-of-the-art techniques by a  large margin.

@InProceedings{Alfaro_2016_CVPR,author = {Alfaro, Anali and Mery, Domingo and Soto, Alvaro},title = {Action Recognition in Video Using Sparse Coding and Relative Features},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Limin Wang, Yu Qiao, Xiaoou Tang, Luc Van Gool

Abstract: Actionness was introduced to quantify the likelihood of containing a generic action instance at a specific location. Accurate and efficient estimation of actionness is important in video analysis and may benefit other relevant tasks such as action recognition and action detection. This paper presents a new deep architecture for actionness estimation, called hybrid fully convolutional network (H-FCN), which is composed of appearance FCN (A-FCN) and motion FCN (M-FCN). These two FCNs leverage the strong capacity of deep models to estimate actionness maps from the perspectives of static appearance and dynamic motion, respectively. In addition, the fully convolutional nature of H-FCN allows it to efficiently process videos with arbitrary sizes. Experiments are conducted on the challenging datasets of Stanford40, UCF Sports, and JHMDB to verify the effectiveness of H-FCN on actionness estimation, which demonstrate that our method achieves superior performance to previous ones. Moreover, we apply the estimated actionness maps on action proposal generation and action detection. Our actionness maps advance the current state-of-the-art performance of these tasks substantially.

@InProceedings{Wang_2016_CVPR,author = {Wang, Limin and Qiao, Yu and Tang, Xiaoou and Van Gool, Luc},title = {Actionness Estimation Using Hybrid Fully Convolutional Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: In this paper we consider the task of recognizing human actions in realistic video where human actions are dominated by irrelevant factors. We first study the benefits of removing non-action video segments, which are the ones that do not portray any human action. We then learn a non-action classifier and use it to down-weight irrelevant video segments. The non-action classifier is trained using ActionThread, a dataset with shot-level annotation for the occurrence or absence of a human action. The non-action classifier can be used to identify non-action shots with high precision and subsequently used to improve the performance of action recognition systems.

@InProceedings{Wang_2016_CVPR,author = {Wang, Yang and Hoai, Minh},title = {Improving Human Action Recognition by Non-Action Classification},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Bowen Zhang, Limin Wang, Zhe Wang, Yu Qiao, Hanli Wang

Abstract: The deep two-stream architecture exhibited excellent performance on video based action recognition. The most computationally expensive step in this approach comes from the calculation of optical flow which prevents it to be real-time. This paper accelerates this architecture by replacing optical flow with motion vector which can be obtained directly from compressed videos without extra calculation. However, motion vector lacks fine structures, and contains noisy and inaccurate motion patterns, leading to the evident degradation of recognition performance. Our key insight for relieving this problem is that optical flow and motion vector are inherent correlated. Transferring the knowledge learned with optical flow CNN to motion vector CNN can significantly boost the performance of the latter. Specifically, we introduce three strategies for this, initialization transfer, supervision transfer and their combination. Experimental results show that our method achieves comparable recognition performance to the state-of-the-art, while our method can process 390.7 frames per second, which is 27 times faster than the original two-stream method.

@InProceedings{Zhang_2016_CVPR,author = {Zhang, Bowen and Wang, Limin and Wang, Zhe and Qiao, Yu and Wang, Hanli},title = {Real-Time Action Recognition With Enhanced Motion Vector CNNs},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Laplacian Patch-Based Image Synthesis

Abstract: Patch-based image synthesis has been enriched with global optimization on the image pyramid. Successively, the gradient-based synthesis has improved structural coherence and details. However, the gradient operator is directional and inconsistent and requires computing multiple operators. It also introduces a significantly heavy computational burden to solve the Poisson equation that often accompanies artifacts in non-integrable gradient fields. In this paper, we propose a patch-based synthesis using a Laplacian pyramid to improve searching correspondence with enhanced awareness of edge structures. Contrary to the gradient operators, the Laplacian pyramid has the advantage of being isotropic in detecting changes to provide more consistent performance in decomposing the base structure and the detailed localization. Furthermore, it does not require heavy computation as it employs approximation by the differences of Gaussians. We examine the potentials of the Laplacian pyramid for enhanced edge-aware correspondence search. We demonstrate the effectiveness of the Laplacian-based approach over the state-of-the-art patch-based image synthesis methods.

@InProceedings{Lee_2016_CVPR,author = {Lee, Joo Ho and Choi, Inchang and Kim, Min H.},title = {Laplacian Patch-Based Image Synthesis},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Rain Streak Removal Using Layer Priors

Yu Li, Robby T. Tan, Xiaojie Guo, Jiangbo Lu, Michael S. Brown

Abstract: This paper addresses the problem of rain streak removal from a single image. Rain streaks impair visibility of an image and introduce undesirable interference that can severely affect the performance of computer vision algorithms.   Rain streak removal can be formulated as a layer decomposition problem, with a rain streak layer superimposed on a background layer containing the true scene content. Existing decomposition methods that address this problem employ either dictionary learning methods or impose a low rank structure on the appearance of the rain streaks.   While these methods can improve the overall visibility, they tend to leave too many rain streaks in the background image or over-smooth the background image.   In this paper, we propose an effective method that uses simple patch-based priors for both the background and rain layers.  These priors are based on Gaussian mixture models and can accommodate multiple orientations and scales of the rain streaks. This simple approach removes rain streaks better than the existing methods qualitatively and quantitatively.  We overview our method and demonstrate its effectiveness over prior work on a number of examples.

@InProceedings{Li_2016_CVPR,author = {Li, Yu and Tan, Robby T. and Guo, Xiaojie and Lu, Jiangbo and Brown, Michael S.},title = {Rain Streak Removal Using Layer Priors},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Takashi Shibata, Masayuki Tanaka, Masatoshi Okutomi

Abstract: This paper presents a novel unified gradient-domain image reconstruction framework with intensity-range constraint and base-structure constraint. The existing method for manipulating base structures and detailed textures are classifiable into two major approaches: i) gradient-domain and ii) layer-decomposition. To generate detail-preserving and artifact-free output images, we combine the benefits of the two approaches into the proposed framework by introducing the intensity-range constraint and the base-structure constraint. To preserve details of the input image, the proposed method takes advantage of reconstructing the output image in the gradient domain, while the output intensity is guaranteed to lie within the specified intensity range, e.g. 0-to-255, by the intensity-range constraint. In addition, the reconstructed image lies close to the base structure by the base-structure constraint, which is effective for restraining artifacts. Experimental results show that the proposed framework is effective for various applications such as tone mapping, seamless image cloning, detail enhancement, and image restoration.

@InProceedings{Shibata_2016_CVPR,author = {Shibata, Takashi and Tanaka, Masayuki and Okutomi, Masatoshi},title = {Gradient-Domain Image Reconstruction Framework With Intensity-Range and Base-Structure Constraints},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Jialei Wang, Peder A. Olsen, Andrew R. Conn, Aurelie C. Lozano

Abstract: We consider the problem of removing and replacing clouds in satellite image sequences, which has a wide range of applications in remote sensing. Our approach first detects and removes the cloud-contaminated part of the image sequences, then recovers the missing scenes from the clean parts by the proposed "TECROMAC" (TEmporally Contiguous RObust MAtrix Completion) objective. The objective function balances temporal smoothness with a low rank solution while staying close to the original observations. The matrix where the rows are pixels and columns are the days of the image has low-rank because the pixels reflect land-types such as vegetation, roads and lakes and there are relatively few of these. We provide efficient optimization algorithms for TECROMAC, so we can run on images containing millions of pixels. Empirical results on real satellite image sequences as well as simulated data demonstrate that our approach is able to recover underlying images from heavily cloud-contaminated observations.

@InProceedings{Wang_2016_CVPR,author = {Wang, Jialei and Olsen, Peder A. and Conn, Andrew R. and Lozano, Aurelie C.},title = {Removing Clouds and Recovering Ground Observations in Satellite Image Sequences via Temporally Contiguous Robust Matrix Completion},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Zhangyang Wang, Ding Liu, Shiyu Chang, Qing Ling, Yingzhen Yang, Thomas S. Huang

Abstract: In this paper, we design a Deep Dual-Domain (D3) based fast restoration model to remove artifacts of JPEG compressed images. It leverages the large learning capacity of deep networks, as well as the problem-specific expertise that was hardly incorporated in the past design of deep architectures. For the latter, we take into consideration both the prior knowledge of the JPEG compression scheme, and the successful practice of the sparsity-based dual-domain approach. We further design the One-Step Sparse Inference (1-SI) module, as an efficient and light-weighted feed-forward approximation of sparse coding. Extensive experiments verify the superiority of the proposed D3 model over several state-of-the-art methods. Specifically, our best model is capable of outperforming the latest deep model for around 1 dB in PSNR, and is 30 times faster.

@InProceedings{Wang_2016_CVPR,author = {Wang, Zhangyang and Liu, Ding and Chang, Shiyu and Ling, Qing and Yang, Yingzhen and Huang, Thomas S.},title = {D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Vijay Rengarajan, Ambasamudram N. Rajagopalan, Rangarajan Aravind

Abstract: The rule of perspectivity that 'straight-lines-must-remain-straight' is easily inflected in CMOS cameras by distortions introduced by motion. Lines can be rendered as curves due to the row-wise exposure mechanism known as rolling shutter (RS). We solve the problem of correcting distortions arising from handheld cameras due to RS effect from a single image free from motion blur with special relevance to urban scenes. We develop a procedure to extract prominent curves from the RS image since this is essential for deciphering the varying row-wise motion. We pose an optimization problem with line desirability costs based on straightness, angle, and length, to resolve the geometric ambiguities while estimating the camera motion based on a rotation-only model assuming known camera intrinsic matrix. Finally, we rectify the RS image based on the estimated camera trajectory using inverse mapping. We show rectification results for RS images captured using mobile phone cameras. We also compare our single image method against existing video and nonblind RS rectification methods that typically require multiple images.

@InProceedings{Rengarajan_2016_CVPR,author = {Rengarajan, Vijay and Rajagopalan, Ambasamudram N. and Aravind, Rangarajan},title = {From Bows to Arrows: Rolling Shutter Rectification of Urban Scenes},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Xueyang Fu, Delu Zeng, Yue Huang, Xiao-Ping Zhang, Xinghao Ding

Abstract: We propose a weighted variational model to estimate both the reflectance and the illumination from an observed image. We show that, though it is widely adopted for ease of modeling, the log-transformed image for this task is not ideal. Based on the previous investigation of the logarithmic transformation, a new weighted variational model is proposed for better prior representation, which is imposed in the regularization terms. Different from conventional variational models, the proposed model can preserve the estimated reflectance with more details. Moreover, the proposed model can suppress noise to some extent. An alternating minimization scheme is adopted to solve the proposed model. Experimental results demonstrate the effectiveness of the proposed model with its algorithm. Compared with other variational methods, the proposed method yields comparable or better results on both subjective and objective assessments.

@InProceedings{Fu_2016_CVPR,author = {Fu, Xueyang and Zeng, Delu and Huang, Yue and Zhang, Xiao-Ping and Ding, Xinghao},title = {A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: A number of recent approaches have used deep convolutional neural networks (CNNs) to build texture representations. Nevertheless, it is still unclear how these mod- els represent texture and invariances to categorical variations. This work conducts a systematic evaluation of recent CNN-based texture descriptors for recognition and attempts to understand the nature of invariances captured by these representations. First we show that the recently proposed bilinear CNN model [25] is an excellent generalpurpose texture descriptor and compares favorably to other CNN-based descriptors on various texture and scene recognition benchmarks. The model is translationally invariant and obtains better accuracy on the ImageNet dataset without requiring spatial jittering of data compared to corresponding models trained with spatial jittering. Based on recent work [13, 28] we propose a technique to visualize pre-images, providing a means for understanding categorical properties that are captured by these representations. Finally, we show preliminary results on how a unified parametric model of texture analysis and synthesis can be used for attribute-based image manipulation, e.g. to make an image more swirly, honeycombed, or knitted. The source code and additional visualizations are available at http://vis-www.cs.umass.edu/texture.

@InProceedings{Lin_2016_CVPR,author = {Lin, Tsung-Yu and Maji, Subhransu},title = {Visualizing and Understanding Deep Texture Representations},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hanwang Zhang, Xindi Shang, Wenzhuo Yang, Huan Xu, Huanbo Luan, Tat-Seng Chua

Abstract: We focus on learning open-vocabulary visual classifiers, which scale up to a large portion of natural language vocabulary (e.g., over tens of thousands of classes). In particular, the training data are large-scale weakly labeled Web images since it is difficult to acquire sufficient well-labeled data at this category scale.  In this paper, we propose a novel online learning paradigm towards this challenging task. Different from traditional N-way independent classifiers that generally fail to handle the extremely sparse and inter-related labels, our classifiers learn from continuous label embeddings discovered by collaboratively decomposing the sparse image-label matrix. Leveraging on the structure of the proposed collaborative learning formulation, we develop an efficient online algorithm that can jointly learn the label embeddings and visual classifiers. The algorithm can learn over 30,000 classes of 1,000 training images within 1 second on a standard GPU. Extensively experimental results on four benchmarks demonstrate the effectiveness of our method.

@InProceedings{Zhang_2016_CVPR,author = {Zhang, Hanwang and Shang, Xindi and Yang, Wenzhuo and Xu, Huan and Luan, Huanbo and Chua, Tat-Seng},title = {Online Collaborative Learning for Open-Vocabulary Visual Classifiers},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Jinshan Pan, Zhouchen Lin, Zhixun Su, Ming-Hsuan Yang

Abstract: Estimating blur kernels from real world images is a challenging problem as the linear image formation assumption does not hold when significant outliers, such as saturated pixels and non-Gaussian noise, are present. While some existing non-blind deblurring algorithms can deal with outliers to a certain extent, few blind deblurring methods are developed to well estimate the blur kernels from the blurred images with outliers. In this paper, we present an algorithm to address this problem by exploiting reliable edges and removing outliers in the intermediate latent images, thereby estimating blur kernels robustly. We analyze the effects of outliers on kernel estimation and show that most state-of-the-art blind deblurring methods may recover delta kernels when blurred images contain significant outliers. We propose a robust energy function which describes the properties of outliers for the final latent image restoration. Furthermore, we show that the proposed algorithm can be applied to improve existing methods to deblur images with outliers. Extensive experiments on different kinds of challenging blurry images with significant amount of outliers demonstrate the proposed algorithm performs favorably against the state-of-the-art methods.

@InProceedings{Pan_2016_CVPR,author = {Pan, Jinshan and Lin, Zhouchen and Su, Zhixun and Yang, Ming-Hsuan},title = {Robust Kernel Estimation With Outliers Handling for Image Deblurring},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, Zbigniew Wojna

Abstract: Convolutional networks are at the core of most state-of-the-art computer vision solutions for a wide variety of tasks. Since 2014 very deep convolutional networks started to become mainstream, yielding substantial gains in various benchmarks. Although increased model size and computational cost tend to translate to immediate quality gains for most tasks (as long as enough labeled data is provided for training), computational efficiency and low parameter count are still enabling factors for various use cases such as mobile vision and big-data scenarios. Here we are exploring ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible. We benchmark our methods on the ILSVRC 2012 classification challenge validation set and demonstrate substantial gains over the state of the art via to carefully factorized convolutions and aggressive regularization: 21.2% top-1 and 5.6% top-5 error for single frame evaluation using a network with a computational cost of 5 billion multiply-adds per inference and with using less than 25 million parameters.

@InProceedings{Szegedy_2016_CVPR,author = {Szegedy, Christian and Vanhoucke, Vincent and Ioffe, Sergey and Shlens, Jon and Wojna, Zbigniew},title = {Rethinking the Inception Architecture for Computer Vision},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Saurabh Gupta, Judy Hoffman, Jitendra Malik

Abstract: In this work we propose a technique that transfers supervision between images from different modalities. We use learned representations from a large labeled modality as supervisory signal for training representations for a new unlabeled paired modality. Our method enables learning of rich representations for unlabeled modalities and can be used as a pre-training procedure for new modalities with limited labeled data. We transfer supervision from labeled RGB images to unlabeled depth and optical flow images and demonstrate large improvements for both these cross modal supervision transfers.

@InProceedings{Gupta_2016_CVPR,author = {Gupta, Saurabh and Hoffman, Judy and Malik, Jitendra},title = {Cross Modal Distillation for Supervision Transfer},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Trung T. Pham, Seyed Hamid Rezatofighi, Ian Reid, Tat-Jun Chin

Abstract: We tackle the problem of large-scale object detection in images, where the number of objects can be arbitrarily large, and can exhibit significant overlap/occlusion.  A successful approach to modelling the large-scale nature of this problem has been via point process density functions which jointly encode object qualities and spatial interactions.  But the corresponding optimisation problem is typically difficult or intractable, and many of the best current methods rely on Monte Carlo Markov Chain (MCMC) simulation, which converges slowly in a large solution space.   We propose an efficient point process inference for large-scale object detection using discrete energy minimization. In particular, we approximate the solution space by a finite set of object proposals and cast the point process density function to a corresponding energy function of binary variables whose values indicate which object proposals are accepted. We resort to the local submodular approximation (LSA) based trust-region optimisation to find the optimal solution. Furthermore we analyse the error of LSA approximation, and show how to adjust the point process energy to dramatically speed up the convergence without harms in the optimality. We demonstrate the superior efficiency and accuracy of our method using a variety of large-scale object detection applications such as crowd human detection, birds, cells counting/localization.

@InProceedings{Pham_2016_CVPR,author = {Pham, Trung T. and Rezatofighi, Seyed Hamid and Reid, Ian and Chin, Tat-Jun},title = {Efficient Point Process Inference for Large-Scale Object Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: Weakly supervised learning of object detection is an important problem in image understanding that still does not have a satisfactory solution.  In this paper, we address this problem by exploiting the power of deep convolutional neural networks pre-trained on large-scale image-level classification tasks. We propose a weakly supervised deep detection architecture that modifies one such network to operate at the level of image regions, performing simultaneously region selection and classification. Trained as an image classifier, the architecture implicitly learns object detectors that are better than alternative weakly supervised detection systems on the PASCAL VOC data. The model, which is a simple and elegant end-to-end architecture, outperforms standard data augmentation and fine-tuning techniques for the task of image-level classification as well.

@InProceedings{Bilen_2016_CVPR,author = {Bilen, Hakan and Vedaldi, Andrea},title = {Weakly Supervised Deep Detection Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Jacob Chan, Jimmy Addison Lee, Qian Kemao

Abstract: This paper presents an algorithm coined BORDER (Bounding Oriented-Rectangle Descriptors for Enclosed Regions) for texture-less object recognition. By fusing a regional object encompassment concept with descriptor-based pipelines, we extend local-patches into scalable object-sized oriented rectangles for optimal object information encapsulation with minimal outliers. We correspondingly introduce a modified line-segment detection technique termed Linelets to stabilize keypoint repeatability in homogenous conditions. In addition, a unique sampling technique facilitates the incorporation of robust angle primitives to produce discriminative rotation-invariant descriptors. BORDER's high competence in object recognition particularly excels in homogenous conditions obtaining superior detection rates in the presence of high-clutter, occlusion and scale-rotation changes when compared with modern state-of-the-art texture-less object detectors such as BOLD and LINE2D on public texture-less object databases.

@InProceedings{Chan_2016_CVPR,author = {Chan, Jacob and Lee, Jimmy Addison and Kemao, Qian},title = {BORDER: An Oriented Rectangles Approach to Texture-Less Object Recognition},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Active Image Segmentation Propagation

Abstract: We propose a semi-automatic method to obtain foreground object masks for a large set of related images.  We develop a stagewise active approach to propagation: in each stage, we actively determine the images that appear most valuable for human annotation, then revise the foreground estimates in all unlabeled images accordingly.  In order to identify images that, once annotated, will propagate well to other examples, we introduce an active selection procedure that operates on the joint segmentation graph over all images.  It prioritizes human intervention for those images that are uncertain and influential in the graph, while also mutually diverse.  We apply our method to obtain foreground masks for over 1 million images.  Our method yields state-of-the-art accuracy on the ImageNet and MIT Object Discovery datasets, and it focuses human attention more effectively than existing propagation strategies.

@InProceedings{Jain_2016_CVPR,author = {Jain, Suyog Dutt and Grauman, Kristen},title = {Active Image Segmentation Propagation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Sean Bell, C. Lawrence Zitnick, Kavita Bala, Ross Girshick

Abstract: It is well known that contextual and multi-scale representations are important for accurate visual recognition. In this paper we present the Inside-Outside Net (ION), an object detector that exploits information both inside and outside the region of interest. Contextual information outside the region of interest is integrated using spatial recurrent neural networks. Inside, we use skip pooling to extract information at multiple scales and levels of abstraction. Through extensive experiments we evaluate the design space and provide readers with an overview of what tricks of the trade are important. ION improves state-of-the-art on PASCAL VOC 2012 object detection from 73.9% to 77.9% mAP. On the new and more challenging MS COCO dataset, we improve state-of-the-art from 19.7% to 33.1% mAP. In the 2015 MS COCO Detection Challenge, our ION model won "Best Student Entry" and finished 3rd place overall. As intuition suggests, our detection results provide strong evidence that context and multi-scale representations improve small object detection.

@InProceedings{Bell_2016_CVPR,author = {Bell, Sean and Zitnick, C. Lawrence and Bala, Kavita and Girshick, Ross},title = {Inside-Outside Net: Detecting Objects in Context With Skip Pooling and Recurrent Neural Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: Thanks to the powerful feature representations obtained through deep convolutional neural network (CNN), the performance of object detection has recently been substantially boosted. Despite the remarkable success, the problems of object rotation, within-class variability, and between-class similarity remain several major challenges. To address these problems, this paper proposes a novel and effective method to learn a rotation-invariant and Fisher discriminative CNN (RIFD-CNN) model. This is achieved by introducing and learning a rotation-invariant layer and a Fisher discriminative layer, respectively, on the basis of the existing high-capacity CNN architectures. Specifically, the rotation-invariant layer is trained by imposing an explicit regularization constraint on the objective function that enforces invariance on the CNN features before and after rotating. The Fisher discriminative layer is trained by imposing the Fisher discrimination criterion on the CNN features so that they have small within-class scatter but large between-class separation. In the experiments, we comprehensively evaluate the proposed method for object detection task on a public available aerial image dataset and the PASCAL VOC 2007 dataset. State-of-the-art results are achieved compared with the existing baseline methods.

@InProceedings{Cheng_2016_CVPR,author = {Cheng, Gong and Zhou, Peicheng and Han, Junwei},title = {RIFD-CNN: Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Stefan Mathe, Aleksis Pirinen, Cristian Sminchisescu

Abstract: One of the most widely used strategies for visual object detection is based on exhaustive spatial hypothesis search. While methods like sliding windows have been successful and effective for many years, they are still brute-force, independent of the image content and the visual category being searched. In this paper we present formally rigorous sequential models that accumulate evidence collected at a small set of image locations in order to detect visual objects effectively. By formulating sequential search as reinforcement learning of the search policy (including the stopping condition), our fully trainable model can explicitly balance for each class, specifically, the conflicting goals of exploration -- sampling more image regions for better accuracy --, and exploitation -- stopping the search efficiently when sufficiently confident in the target's location. The methodology is general and applicable to any detector response function. We report encouraging results in the PASCAL VOC 2012 object detection test set showing that the proposed methodology achieves almost  two orders of magnitude speed-up over sliding window methods.

@InProceedings{Mathe_2016_CVPR,author = {Mathe, Stefan and Pirinen, Aleksis and Sminchisescu, Cristian},title = {Reinforcement Learning for Visual Object Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abstract: In this paper we describe a new method for detecting and counting a repeating object in an image. While the method relies on a fairly sophisticated deformable part model, unlike existing techniques it estimates the model parameters in an unsupervised fashion thus alleviating the need for a user-annotated training data and avoiding the associated specificity. This automatic fitting process is carried out by exploiting the recurrence of small image patches associated with the repeating object and analyzing their spatial correlation. The analysis allows us to reject outlier patches, recover the visual and shape parameters of the part model, and detect the object instances efficiently.  In order to achieve a practical system which is able to cope with diverse images, we describe a simple and intuitive active-learning procedure that updates the object classification by querying the user on very few carefully chosen marginal classifications. Evaluation of the new method against the state-of-the-art techniques demonstrates its ability to achieve higher accuracy through a better user experience.

@InProceedings{Huberman_2016_CVPR,author = {Huberman, Inbar and Fattal, Raanan},title = {Detecting Repeating Objects Using Patch Correlation Analysis},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Sebastian Lapuschkin, Alexander Binder, Gregoire Montavon, Klaus-Robert Muller, Wojciech Samek

Abstract: Fisher vector (FV) classifiers and Deep Neural Networks (DNNs) are popular and successful algorithms for solving image classification problems. However, both are generally considered `black box' predictors as the non-linear transformations involved have so far prevented transparent and interpretable reasoning. Recently, a principled technique, Layer-wise Relevance Propagation (LRP), has been developed in order to better comprehend the inherent structured reasoning of complex nonlinear classification models such as Bag of Feature models or DNNs. In this paper we (1) extend the LRP framework also for Fisher vector classifiers and then use it as analysis tool to (2) quantify the importance of context for classification, (3) qualitatively compare DNNs against FV classifiers in terms of important image regions and (4) detect potential flaws and biases in data.  All experiments are performed on the PASCAL VOC 2007 and ILSVRC 2012 data sets.

@InProceedings{Lapuschkin_2016_CVPR,author = {Lapuschkin, Sebastian and Binder, Alexander and Montavon, Gregoire and Muller, Klaus-Robert and Samek, Wojciech},title = {Analyzing Classifiers: Fisher Vectors and Deep Neural Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba

Abstract: In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network (CNN) to have remarkable localization ability despite being trained on image-level labels. While this technique was previously proposed as a means for regularizing training, we find that it actually builds a generic localizable deep representation that exposes the implicit attention of CNNs on image. Despite the apparent simplicity of global average pooling, we are able to achieve 37.1% top-5 error for object localization on ILSVRC 2014 without training on any bounding box annotation. We demonstrate that our network is able to localize the discriminative image regions on a variety of tasks despite not being trained for them.

@InProceedings{Zhou_2016_CVPR,author = {Zhou, Bolei and Khosla, Aditya and Lapedriza, Agata and Oliva, Aude and Torralba, Antonio},title = {Learning Deep Features for Discriminative Localization},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Ishan Misra, C. Lawrence Zitnick, Margaret Mitchell, Ross Girshick

Abstract: When human annotators are given a choice about what to label in an image, they apply their own subjective judgments on what to ignore and what to mention.  We refer to these noisy "human-centric" annotations as exhibiting human reporting bias.  Examples of such annotations include image tags and keywords found on photo sharing sites, or in datasets containing image captions. In this paper, we use these noisy annotations for learning visually correct image classifiers. Such annotations do not use consistent vocabulary, and miss a significant amount of the information present in an image; however, we demonstrate that the noise in these annotations exhibits structure and can be modeled. We propose an algorithm to decouple the human reporting bias from the correct visually grounded labels. Our results are highly interpretable for reporting "what's in the image" versus "what's worth saying."  We demonstrate the algorithm's efficacy along a variety of metrics and datasets, including MS COCO and Yahoo Flickr 100M. We show significant improvements over traditional algorithms for both image classification and image captioning, doubling the performance of existing methods in some cases.

@InProceedings{Misra_2016_CVPR,author = {Misra, Ishan and Zitnick, C. Lawrence and Mitchell, Margaret and Girshick, Ross},title = {Seeing Through the Human Reporting Bias: Visual Classifiers From Noisy Human-Centric Labels},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Lluis Castrejon, Yusuf Aytar, Carl Vondrick, Hamed Pirsiavash, Antonio Torralba

@InProceedings{Castrejon_2016_CVPR,author = {Castrejon, Lluis and Aytar, Yusuf and Vondrick, Carl and Pirsiavash, Hamed and Torralba, Antonio},title = {Learning Aligned Cross-Modal Representations From Weakly Aligned Data},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Sijia Cai, Lei Zhang, Wangmeng Zuo, Xiangchu Feng

@InProceedings{Cai_2016_CVPR,author = {Cai, Sijia and Zhang, Lei and Zuo, Wangmeng and Feng, Xiangchu},title = {A Probabilistic Collaborative Representation Based Approach for Pattern Classification},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hexiang Hu, Guang-Tong Zhou, Zhiwei Deng, Zicheng Liao, Greg Mori

@InProceedings{Hu_2016_CVPR,author = {Hu, Hexiang and Zhou, Guang-Tong and Deng, Zhiwei and Liao, Zicheng and Mori, Greg},title = {Learning Structured Inference Neural Networks With Label Relations},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hongyuan Zhu, Jean-Baptiste Weibel, Shijian Lu

@InProceedings{Zhu_2016_CVPR,author = {Zhu, Hongyuan and Weibel, Jean-Baptiste and Lu, Shijian},title = {Discriminative Multi-Modal Feature Fusion for RGBD Indoor Scene Recognition},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Qiang Li, Maoying Qiao, Wei Bian, Dacheng Tao

@InProceedings{Li_2016_CVPR,author = {Li, Qiang and Qiao, Maoying and Bian, Wei and Tao, Dacheng},title = {Conditional Graphical Lasso for Multi-Label Image Classification},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Wei_2016_CVPR,author = {Wei, Zijun and Hoai, Minh},title = {Region Ranking SVM for Image Classification},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Carl Vondrick, Deniz Oktay, Hamed Pirsiavash, Antonio Torralba

@InProceedings{Vondrick_2016_CVPR,author = {Vondrick, Carl and Oktay, Deniz and Pirsiavash, Hamed and Torralba, Antonio},title = {Predicting Motivations of Actions by Leveraging Text},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Sochor_2016_CVPR,author = {Sochor, Jakub and Herout, Adam and Havel, Jiri},title = {BoxCars: 3D Boxes as CNN Input for Improved Fine-Grained Vehicle Recognition},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Xu Liu, Zilei Wang, Jiashi Feng, Hongsheng Xi

@InProceedings{Liu_2016_CVPR,author = {Liu, Xu and Wang, Zilei and Feng, Jiashi and Xi, Hongsheng},title = {Highway Vehicle Counting in Compressed Domain},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Shiyao Huang, Xianghua Ying, Jiangpeng Rong, Zeyu Shang, Hongbin Zha

@InProceedings{Huang_2016_CVPR,author = {Huang, Shiyao and Ying, Xianghua and Rong, Jiangpeng and Shang, Zeyu and Zha, Hongbin},title = {Camera Calibration From Periodic Motion of a Pedestrian},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hakan Bilen, Basura Fernando, Efstratios Gavves, Andrea Vedaldi, Stephen Gould

@InProceedings{Bilen_2016_CVPR,author = {Bilen, Hakan and Fernando, Basura and Gavves, Efstratios and Vedaldi, Andrea and Gould, Stephen},title = {Dynamic Image Networks for Action Recognition},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Vignesh Ramanathan, Jonathan Huang, Sami Abu-El-Haija, Alexander Gorban, Kevin Murphy, Li Fei-Fei

@InProceedings{Ramanathan_2016_CVPR,author = {Ramanathan, Vignesh and Huang, Jonathan and Abu-El-Haija, Sami and Gorban, Alexander and Murphy, Kevin and Fei-Fei, Li},title = {Detecting Events and Key Actors in Multi-Person Videos},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Mahasseni_2016_CVPR,author = {Mahasseni, Behrooz and Todorovic, Sinisa},title = {Regularizing Long Short Term Memory With 3D Human-Skeleton Sequences for Action Recognition},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


James Charles, Tomas Pfister, Derek Magee, David Hogg, Andrew Zisserman

@InProceedings{Charles_2016_CVPR,author = {Charles, James and Pfister, Tomas and Magee, Derek and Hogg, David and Zisserman, Andrew},title = {Personalizing Human Video Pose Estimation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Wei Yang, Wanli Ouyang, Hongsheng Li, Xiaogang Wang

@InProceedings{Yang_2016_CVPR,author = {Yang, Wei and Ouyang, Wanli and Li, Hongsheng and Wang, Xiaogang},title = {End-To-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Xu_2016_CVPR,author = {Xu, Chenliang and Corso, Jason J.},title = {Actor-Action Semantic Segmentation With Grouping Process Models},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Jun Yuan, Bingbing Ni, Xiaokang Yang, Ashraf A. Kassim

@InProceedings{Yuan_2016_CVPR,author = {Yuan, Jun and Ni, Bingbing and Yang, Xiaokang and Kassim, Ashraf A.},title = {Temporal Action Localization With Pyramid of Score Distribution Features},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Katsunori Ohnishi, Atsushi Kanehira, Asako Kanezaki, Tatsuya Harada

@InProceedings{Ohnishi_2016_CVPR,author = {Ohnishi, Katsunori and Kanehira, Atsushi and Kanezaki, Asako and Harada, Tatsuya},title = {Recognizing Activities of Daily Living With a Wrist-Mounted Camera},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Zuxuan Wu, Yanwei Fu, Yu-Gang Jiang, Leonid Sigal

@InProceedings{Wu_2016_CVPR,author = {Wu, Zuxuan and Fu, Yanwei and Jiang, Yu-Gang and Sigal, Leonid},title = {Harnessing Object and Scene Semantics for Large-Scale Video Understanding},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Choi_2016_CVPR,author = {Choi, Jinsoo and Oh, Tae-Hyun and Kweon, In So},title = {Video-Story Composition via Plot Analysis},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Richard_2016_CVPR,author = {Richard, Alexander and Gall, Juergen},title = {Temporal Action Detection Using a Statistical Language Model},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Shu Liu, Xiaojuan Qi, Jianping Shi, Hong Zhang, Jiaya Jia

@InProceedings{Liu_2016_CVPR,author = {Liu, Shu and Qi, Xiaojuan and Shi, Jianping and Zhang, Hong and Jia, Jiaya},title = {Multi-Scale Patch Aggregation (MPA) for Simultaneous Detection and Segmentation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Dai_2016_CVPR,author = {Dai, Jifeng and He, Kaiming and Sun, Jian},title = {Instance-Aware Semantic Segmentation via Multi-Task Network Cascades},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, Jian Sun

@InProceedings{Lin_2016_CVPR,author = {Lin, Di and Dai, Jifeng and Jia, Jiaya and He, Kaiming and Sun, Jian},title = {ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Abhijit Kundu, Vibhav Vineet, Vladlen Koltun

@InProceedings{Kundu_2016_CVPR,author = {Kundu, Abhijit and Vineet, Vibhav and Koltun, Vladlen},title = {Feature Space Optimization for Semantic Video Segmentation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Maros Blaha, Christoph Vogel, Audrey Richard, Jan D. Wegner, Thomas Pock, Konrad Schindler

@InProceedings{Blaha_2016_CVPR,author = {Blaha, Maros and Vogel, Christoph and Richard, Audrey and Wegner, Jan D. and Pock, Thomas and Schindler, Konrad},title = {Large-Scale Semantic 3D Reconstruction: An Adaptive Multi-Resolution Model for Multi-Class Volumetric Labeling},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Xiaodan Liang, Xiaohui Shen, Donglai Xiang, Jiashi Feng, Liang Lin, Shuicheng Yan

@InProceedings{Liang_2016_CVPR,author = {Liang, Xiaodan and Shen, Xiaohui and Xiang, Donglai and Feng, Jiashi and Lin, Liang and Yan, Shuicheng},title = {Semantic Object Parsing With Local-Global Long Short-Term Memory},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Guosheng Lin, Chunhua Shen, Anton van den Hengel, Ian Reid

@InProceedings{Lin_2016_CVPR,author = {Lin, Guosheng and Shen, Chunhua and van den Hengel, Anton and Reid, Ian},title = {Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Seunghoon Hong, Junhyuk Oh, Honglak Lee, Bohyung Han

@InProceedings{Hong_2016_CVPR,author = {Hong, Seunghoon and Oh, Junhyuk and Lee, Honglak and Han, Bohyung},title = {Learning Transferrable Knowledge for Semantic Segmentation With Deep Convolutional Neural Network},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele

@InProceedings{Cordts_2016_CVPR,author = {Cordts, Marius and Omran, Mohamed and Ramos, Sebastian and Rehfeld, Timo and Enzweiler, Markus and Benenson, Rodrigo and Franke, Uwe and Roth, Stefan and Schiele, Bernt},title = {The Cityscapes Dataset for Semantic Urban Scene Understanding},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Raviteja Vemulapalli, Oncel Tuzel, Ming-Yu Liu, Rama Chellapa

@InProceedings{Vemulapalli_2016_CVPR,author = {Vemulapalli, Raviteja and Tuzel, Oncel and Liu, Ming-Yu and Chellapa, Rama},title = {Gaussian Conditional Random Field Network for Semantic Segmentation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, Antonio M. Lopez

@InProceedings{Ros_2016_CVPR,author = {Ros, German and Sellart, Laura and Materzynska, Joanna and Vazquez, David and Lopez, Antonio M.},title = {The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Alex Locher, Michal Perdoch, Luc Van Gool

@InProceedings{Locher_2016_CVPR,author = {Locher, Alex and Perdoch, Michal and Van Gool, Luc},title = {Progressive Prioritized Multi-View Stereo},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Angjoo Kanazawa, David W. Jacobs, Manmohan Chandraker

@InProceedings{Kanazawa_2016_CVPR,author = {Kanazawa, Angjoo and Jacobs, David W. and Chandraker, Manmohan},title = {WarpNet: Weakly Supervised Matching for Single-View Reconstruction},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Ole Johannsen, Antonin Sulc, Bastian Goldluecke

@InProceedings{Johannsen_2016_CVPR,author = {Johannsen, Ole and Sulc, Antonin and Goldluecke, Bastian},title = {What Sparse Light Field Coding Reveals About Scene Structure},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Wang_2016_CVPR,author = {Wang, Hao and Wang, Jun and Liang, Wang},title = {Online Reconstruction of Indoor Scenes From RGB-D Streams},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Ali Osman Ulusoy, Michael J. Black, Andreas Geiger

@InProceedings{Ulusoy_2016_CVPR,author = {Ulusoy, Ali Osman and Black, Michael J. and Geiger, Andreas},title = {Patches, Planes and Probabilities: A Non-Local Prior for Volumetric 3D Reconstruction},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Schillebeeckx_2016_CVPR,author = {Schillebeeckx, Ian and Pless, Robert},title = {Single Image Camera Calibration With Lenticular Arrays for Augmented Reality},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Thomas_2016_CVPR,author = {Thomas, Diego and Taniguchi, Rin-ichiro},title = {Augmented Blendshapes for Real-Time Simultaneous 3D Head Modeling and Facial Motion Capture},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Xie_2016_CVPR,author = {Xie, Jin and Wang, Meng and Fang, Yi},title = {Learned Binary Spectral Shape Descriptor for 3D Shape Correspondence},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Magri_2016_CVPR,author = {Magri, Luca and Fusiello, Andrea},title = {Multiple Model Fitting as a Set Coverage Problem},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Cedric Verleysen, Christophe De Vleeschouwer

@InProceedings{Verleysen_2016_CVPR,author = {Verleysen, Cedric and De Vleeschouwer, Christophe},title = {Piecewise-Planar 3D Approximation From Wide-Baseline Stereo},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Olivier Saurer, Marc Pollefeys, Gim Hee Lee

@InProceedings{Saurer_2016_CVPR,author = {Saurer, Olivier and Pollefeys, Marc and Lee, Gim Hee},title = {Sparse to Dense 3D Reconstruction From Rolling Shutter Images},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Matthew Trager, Martial Hebert, Jean Ponce

@InProceedings{Trager_2016_CVPR,author = {Trager, Matthew and Hebert, Martial and Ponce, Jean},title = {Consistency of Silhouettes and Their Duals},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Cenek Albl, Zuzana Kukelova, Tomas Pajdla

@InProceedings{Albl_2016_CVPR,author = {Albl, Cenek and Kukelova, Zuzana and Pajdla, Tomas},title = {Rolling Shutter Absolute Pose Problem With Known Vertical Direction},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Eric Brachmann, Frank Michel, Alexander Krull, Michael Ying Yang, Stefan Gumhold, carsten Rother

@InProceedings{Brachmann_2016_CVPR,author = {Brachmann, Eric and Michel, Frank and Krull, Alexander and Yang, Michael Ying and Gumhold, Stefan and Rother, carsten},title = {Uncertainty-Driven 6D Pose Estimation of Objects and Scenes From a Single RGB Image},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Andrey Bushnevskiy, Lorenzo Sorgi, Bodo Rosenhahn

@InProceedings{Bushnevskiy_2016_CVPR,author = {Bushnevskiy, Andrey and Sorgi, Lorenzo and Rosenhahn, Bodo},title = {Multicamera Calibration From Visible and Mirrored Epipoles},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Lazaros Zafeiriou, Epameinondas Antonakos, Stefanos Zafeiriou, Maja Pantic

@InProceedings{Zafeiriou_2016_CVPR,author = {Zafeiriou, Lazaros and Antonakos, Epameinondas and Zafeiriou, Stefanos and Pantic, Maja},title = {Joint Unsupervised Deformable Spatio-Temporal Alignment of Sequences},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Kaili Zhao, Wen-Sheng Chu, Honggang Zhang

@InProceedings{Zhao_2016_CVPR,author = {Zhao, Kaili and Chu, Wen-Sheng and Zhang, Honggang},title = {Deep Region and Multi-Label Learning for Facial Action Unit Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Wu_2016_CVPR,author = {Wu, Yue and Ji, Qiang},title = {Constrained Joint Cascade Regression Framework for Simultaneous Facial Action Unit Recognition and Facial Landmark Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Shizhan Zhu, Cheng Li, Chen-Change Loy, Xiaoou Tang

@InProceedings{Zhu_2016_CVPR,author = {Zhu, Shizhan and Li, Cheng and Loy, Chen-Change and Tang, Xiaoou},title = {Unconstrained Face Alignment via Cascaded Compositional Learning},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Piotraschke_2016_CVPR,author = {Piotraschke, Marcel and Blanz, Volker},title = {Automated 3D Face Reconstruction From Multiple Images Using Quality Measures},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Jie Zhang, Meina Kan, Shiguang Shan, Xilin Chen

@InProceedings{Zhang_2016_CVPR,author = {Zhang, Jie and Kan, Meina and Shan, Shiguang and Chen, Xilin},title = {Occlusion-Free Face Alignment: Deep Regression Networks Coupled With De-Corrupt AutoEncoders},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Zheng Zhang, Jeff M. Girard, Yue Wu, Xing Zhang, Peng Liu, Umur Ciftci, Shaun Canavan, Michael Reale, Andy Horowitz, Huiyuan Yang, Jeffrey F. Cohn, Qiang Ji, Lijun Yin

@InProceedings{Zhang_2016_CVPR,author = {Zhang, Zheng and Girard, Jeff M. and Wu, Yue and Zhang, Xing and Liu, Peng and Ciftci, Umur and Canavan, Shaun and Reale, Michael and Horowitz, Andy and Yang, Huiyuan and Cohn, Jeffrey F. and Ji, Qiang and Yin, Lijun},title = {Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Yu_2016_CVPR,author = {Yu, Pei and Zhou, Jiahuan and Wu, Ying},title = {Learning Reconstruction-Based Remote Gaze Estimation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hongwei Qin, Junjie Yan, Xiu Li, Xiaolin Hu

@InProceedings{Qin_2016_CVPR,author = {Qin, Hongwei and Yan, Junjie and Li, Xiu and Hu, Xiaolin},title = {Joint Training of Cascaded CNN for Face Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Rui Zhao, Quan Gan, Shangfei Wang, Qiang Ji

@InProceedings{Zhao_2016_CVPR,author = {Zhao, Rui and Gan, Quan and Wang, Shangfei and Ji, Qiang},title = {Facial Expression Intensity Estimation Using Ordinal Information},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Proposal Flow

Bumsub Ham, Minsu Cho, Cordelia Schmid, Jean Ponce

Abstract: Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category.  We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that proposal flow can effectively be transformed into a conventional dense flow field. We introduce a new dataset that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use this benchmark to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

@InProceedings{Ham_2016_CVPR,author = {Ham, Bumsub and Cho, Minsu and Schmid, Cordelia and Ponce, Jean},title = {Proposal Flow},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Chen Sun, Manohar Paluri, Ronan Collobert, Ram Nevatia, Lubomir Bourdev

Abstract: This paper aims to classify and locate objects accurately and efficiently, without using bounding box annotations. It is challenging as objects in the wild could appear at arbitrary locations and in different scales. In this paper, we propose a novel classification architecture ProNet based on convolutional neural networks. It uses computationally efficient neural networks to propose image regions that are likely to contain objects, and applies more powerful but slower networks on the proposed regions. The basic building block is a multi-scale fully-convolutional network which assigns object confidence scores to boxes at different locations and scales. We show that such networks can be trained effectively using image-level annotations, and can be connected into cascades or trees for efficient object classification. ProNet outperforms previous state-of-the-art significantly on PASCAL VOC 2012 and MS COCO datasets for object classification and point-based localization.

@InProceedings{Sun_2016_CVPR,author = {Sun, Chen and Paluri, Manohar and Collobert, Ronan and Nevatia, Ram and Bourdev, Lubomir},title = {ProNet: Learning to Propose Object-Specific Boxes for Cascaded Neural Networks},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Thomas_2016_CVPR,author = {Thomas, Christopher and Kovashka, Adriana},title = {Seeing Behind the Camera: Identifying the Authorship of a Photograph},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Shuochen Su, Felix Heide, Robin Swanson, Jonathan Klein, Clara Callenberg, Matthias Hullin, Wolfgang Heidrich

@InProceedings{Su_2016_CVPR,author = {Su, Shuochen and Heide, Felix and Swanson, Robin and Klein, Jonathan and Callenberg, Clara and Hullin, Matthias and Heidrich, Wolfgang},title = {Material Classification Using Raw Time-Of-Flight Measurements},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Dong Li, Jia-Bin Huang, Yali Li, Shengjin Wang, Ming-Hsuan Yang

@InProceedings{Li_2016_CVPR,author = {Li, Dong and Huang, Jia-Bin and Li, Yali and Wang, Shengjin and Yang, Ming-Hsuan},title = {Weakly Supervised Object Localization With Progressive Domain Adaptation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Roozbeh Mottaghi, Hessam Bagherinezhad, Mohammad Rastegari, Ali Farhadi

@InProceedings{Mottaghi_2016_CVPR,author = {Mottaghi, Roozbeh and Bagherinezhad, Hessam and Rastegari, Mohammad and Farhadi, Ali},title = {Newtonian Scene Understanding: Unfolding the Dynamics of Objects in Static Images},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


@InProceedings{Harakeh_2016_CVPR,author = {Harakeh, Ali and Asmar, Daniel and Shammas, Elie},title = {Identifying Good Training Data for Self-Supervised Free Space Estimation},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Hani Altwaijry, Eduard Trulls, James Hays, Pascal Fua, Serge Belongie

@InProceedings{Altwaijry_2016_CVPR,author = {Altwaijry, Hani and Trulls, Eduard and Hays, James and Fua, Pascal and Belongie, Serge},title = {Learning to Match Aerial Images With Deep Attentive Architectures},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Krishna Kumar Singh, Fanyi Xiao, Yong Jae Lee

@InProceedings{Singh_2016_CVPR,author = {Singh, Krishna Kumar and Xiao, Fanyi and Lee, Yong Jae},title = {Track and Transfer: Watching Videos to Simulate Strong Human Supervision for Weakly-Supervised Object Detection},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}


Ali Diba, Ali Mohammad Pazandeh, Hamed Pirsiavash, Luc Van Gool

@InProceedings{Diba_2016_CVPR,author = {Diba, Ali and Pazandeh, Ali Mohammad and Pirsiavash, Hamed and Van Gool, Luc},title = {DeepCAMP: Deep Convolutional Action & Attribute Mid-Level Patterns},booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},month = {June},year = {2016}}