CE Seminar by Dr. Fatma Güney

Time: 10:00
Location: ENG 208







Speaker: Dr. Fatma Güney

Title:  Combining Features and Semantics for Low-level Vision

Date: 18 September, Monday  , 2016

Time: 10:00

Place: ENG 208

Host: Barış Akgün



I will talk about feature representations for matching based problems and their failure cases. I will specifically talk about algorithms which exploit higher-level cues such as semantics and object knowledge in order to resolve inherent ambiguities in matching, in particular, for stereo estimation, 3D reconstruction and optical flow estimation. Traditional methods for these problems usually ignore our prior knowledge on the semantics and typically model only the structuredness of scenes via local interactions or planarity assumptions. We explored efficient ways of extending reasoning beyond these by incorporating our prior knowledge into the process. This kind of reasoning is particularly useful for regions where low-level cues are weak due to reflective and textureless surfaces such as car surfaces. For stereo matching, we sample likely configurations of cars in a scene using 3D CAD models and perform joint inference over these samples and depth. In a following project, instead of using CAD models, we proposed a model to learn the typical shape of cars and buildings jointly with the 3D reconstruction on very challenging street scenes from fish-eye cameras. By exploiting the regularities and repetitiveness of objects in man-made environments, we showed a principled way of completing missing parts of the scene without compromising accuracy. Motivated by the success of deep learning for stereo matching, we developed an algorithm which learns context-aware features for optical flow using Siamese Networks. We further propose to improve the performance by jointly modeling the background motion in static parts of the scene. We have shown that joint reasoning over low-level and high-level cues leads to significant improvements in performance. In particular, our methods have scored the highest performance in standard benchmarks for stereo and optical flow estimation. 



 Dr. Fatma Güney is a post-doctoral fellow in Visual Geometry Group at the University of Oxford. She received her PhD from Max Planck Institute and the University of Tübingen in 2017. Her thesis was about efficient inference methods and feature learning for low-level vision problems. Her research was funded by the Center for Learning Systems (a collaboration between MPI and ETH).  She completed her Master's degree at Bogazici University in 2012 where she worked on facial image analysis. She graduated from the Computer Science Department of the Bilkent University in 2010.