- Tenured Professor & Head of VisLab at the Polytechnic Institute – IPRJ/UERJ, Brazil
- PhD in Computer Engineering from LEMS, Brown University, with prof. Ben Kimia
- Google Book Search Team, Intern 2008, Full-time 2010
- Visual Computing Tech Lead & co-founder of Lab Macambira hacker collective
- Visiting Professor, Brown University 2015
- Google Scholar
My research is moved by the desire for a scientific and deeply conceptual theory of the principles behind the visual world. While biology provides reference visual systems, my focus is to devise scientific and mathematical theories with large-scale, realistic and useful computer systems. Without a working computational model, can we claim to understand anything completely? Computer systems are theories made alive; they incorporate physical contexts against which the principles of a theory can be checked, generalized and improved. The mathematical theory in my research is based on differential geometry for modeling general structures from multiple images and for machine learning – a "geometrization" of computer vision (vaguely speaking). I have focused on two lines of research:
- Multiview stereo 3D reconstruction, camera estimation and recognition using curves and surfaces:
modeling the dynamic and geometric phenomena (possibly high
dimensional) that take place in 3-space at the human scale, mainly using multiple images of the same scene (as
acquired by a camera in different positions, systems of multiple cameras, or
is in devising robust, precise
and automatic methods for tackling problems in this area. In order to meet such
requirements, I explore the use of grouped primitives such as curves and their
differential geometry as the basis for the methods, complementing
interest points and other approaches. The computational end-goal is a system based
on video sequences, that would work for any type of scene
geometry, without need for calibration or textured regions, being able to
identify not only the 3D structure of objects, but also their dynamics, camera
position, the reflectance properties, and lighting conditions.
- Collaborators: Ben Kimia's group at Brown, Peter Giblin and Anil Usumezbas from SRI.
- Additional information Multiview Differential Geometry of Curves and Surfaces
- Geometric Machine Learning/Manifold Learning: the understanding of typically high-dimensional patterns or manifolds from images, for applications such as robust recognition, tracking, occlusion completion, through Diffusion Maps, digital exterior calculus, and other general or specialized manifestations of differential geometry somewhat related to Hodge Theory, using principles from Pattern Theory. For instance, I have been applying this to the 3D modeling of ocean waves from multiple videos.
Current focus: Mathematics. I will be prioritizing mathematics and geometry for the next couple of years, in the broad sense of extracting deep concepts to produce better algorithms and theories for the visual world. My primary interest is in differential geometry, both in the traditional and discrete/probabilistic/graphical manifestations in fields such as diffusion maps, spectral graph theory, and manifold learning.
- The Surfacing of Multiview 3D Drawings via Lofting and Occlusion Reasoning, CVPR 2017, with Anil Usumezbas & Ben Kimia, Expanded version available (pdf | poster | datasets | website | bib) new!
- Multiview Differential Geometry of Curves,
International Journal of Computer Vision, 2016, with Ben Kimia. (pdf | code | bib)
- The main result is the reconstruction of torsion and curvature derivative of a space curve from multiple perspectives, a fundamental theorem I proved in 2004. This paper is the outgrowth of that effort, including continuous motion, non-rigid scenes, intrinsic parameters and camera estimation from occluding contours. It is the seed for a monograph I've been writing, a kind of an extended Hartley & Zisserman's book for general curves and differential geometry. For instance, if you need to reconstruct orientations from two or more views (SIFT orientations or other vector fields, including curvature and torsion fields), this is the place to go.
- From Multiview Image Curves to 3D Drawings, ECCV 2016, with Anil Usumezbas & Ben Kimia, Expanded version available (pdf | supplement | code | datasets | website | bib) new!
- Camera Pose Estimation Using Curve Differential Geometry, ECCV 2012, Firenze, Italy, with Peter J. Giblin & Ben Kimia (pdf | supplementary | code | bib)
- Multiview Differential Geometry in Application to Computer Vision, my Ph.D. Thesis, School of Engineering, Brown University, 2010. Partly superseded by the above papers, still has valuable insight (pdf | bib)
- 3D Curve Sketch: Flexible Curve-Based Stereo Reconstruction and Calibration, CVPR 2010, with B. Kimia. The practical basis of the 3D Curve Drawing work above (pdf | poster | code | website | bib)
- 2D Euclidean Distance Transform Algorithms: A Comparative Survey, ACM Computing Surveys, 2008. My most cited work, with L. da F. Costa, J. Torelli & O. M. Bruno. Actually used in the 3D Curve Sketch and Drawing system for multiview curve stereo (pdf | bib | errata | supplement | code)
- High-Order Differential Geometry of Curves for Multiview Reconstruction and Matching, EMMCVPR 2005, Lecture Notes in Computer Science, 3757, pp. 645-660, 2005, with B. Kimia. A new theorem on recovering torsion of a space curve from multiple perspectives, please refer to my IJCV'16 paper (original | corrected)
- On Voronoi Diagrams and Medial Axes Journal of Mathematical Imaging and Vision, 17(1), 27-40, jul 2002, with L. F. Estrozi & L. da F. Costa, my first journal paper
- Remaining papers in Google Scholar
- Differentiable Manifolds (graduate)
- Riemannian Geometry (graduate)
PhD Students and Notable Msc StudentsCurrent
- Msc: Caio dos Santos Souza - 3D reconsruction of the ocean surface from multiple self-calibrating video streams using curve and surface differential geometry, pattern theory, and nonlinear dimentionality reduction. Caio is starting with machine-learning in edge detection for obtaining suitable representations of ocean waves to be used in photogrammetry.
- PhD: Lucia Maria dos Santos Pinto - Diffusion Maps, geometric diffusions, manifold learning, machine learning, applied differential geometry and stochastic modeling
- PhD: Mauro Amorim - 3D reconsruction of the ocean surface from multiple self-calibrating video streams using curve and surface differential geometry, pattern theory, and nonlinear dimentionality reduction
- Msc: David da Costa de Pinho - the theory of trifocal pose estimation using differential geometry
- Computer Vision and Image Processing
- Automated 3D reconstruction from multiple views, camera auto-calibration, shape from shading
- Algorithms related to propagation and curve evolution: diffusion maps, distance transforms, medial axes for shape representation, image segmentation, Voronoi diagrams, geodesics, shortest paths, inpainting
- Use of shape/context/geometric information in computer vision and machine learning
- Applying Mathematics (esp. geometry) to computer vision and machine learning
- Differential Geometry, Hogdge Theory, Singularity Theory, Catastrophe Theory
- Algebraic Geometry (solving systems of polynomial equations, projective geometry), Geometric Algebra
- Partial Differential Equations, specially geometric aspects
- Inverse problems connected to geometry
- Computational Geometry
- Lab Macambira: an open source development group I co-founded, focusing on advanced audio, vision and web technolgy
- VxL C++ Computer vision libraries. The foundation of my research code, as fostered at Brown with Joe Mundy
- SIP: Image processing toolbox I wrote for Scilab, similar to Matlab but with true freedom
- Gentoo Linux: an excellent OS for computer scientists and experienced users.
Research and study material
- Summary of multiple view geometry as in Hartley & Zisserman's book (pdf | ppt)
- Summary of a recent paper in Auto-Calibration (pdf | odp)
- 3D Reconstruction from a Handheld camera: a project page on point-based SfM.
- Social Twitter | Facebook | Blog
- My brother's sonic compositions Renato Fabbri
- Comics and Animations by my Cousin and Friends Pula Pirata
- Other PhD Alumni from LEMS / Brown University
Do the vertical borders of this page look strange to you? They are exactly the same gradients, reflected vertically. On their own, they should look convex or concave. But your brain is hypothesizing a global light source, and both bars can't be both convex and consistent with the same light source. There are also biases in direction - the brain tends to favor certain directions for the light source over others. As you might know, this is known as the convex concave crater illusion, and is related to the hollow-face illusion.