Ricardo Fabbri
- Tenured Professor & Head of VisLab at the Polytechnic Institute – IPRJ/UERJ, Brazil
- PhD in Computer Engineering from LEMS, Brown University, with prof. Ben Kimia
- Google Book Search Team, Intern 2008, Full-time 2010
- Visual Computing Tech Lead & co-founder of Lab Macambira hacker collective
- Visiting Professor, Brown University 2015
- Visiting Professor, ICERM - Brown University 2018, 2019
- Main organizer, Computer Vision Working Group of the Nonlinear Algebra Program 2018
- Main organizer, ICERM - Brown University, Algebraic Vision Research Cluster 2019
- Google Scholar
Research statement
My research is moved by the desire for a scientific and deeply conceptual theory of the principles behind the visual world. While biology provides reference visual systems, my focus is to devise scientific and athematical theories with large-scale, realistic and useful computer systems. Without a working computational model, can we claim to understand anything completely? Computer systems are theories made alive; they incorporate physical contexts against which the principles of a theory can be checked, generalized and improved. The mathematical theory in my research is based on differential geometry for modeling general structures from multiple images and for machine learning – a "geometrization" of computer vision (vaguely speaking). I have focused on two lines of research:
- Multiview stereo 3D reconstruction, camera estimation and recognition using curves and surfaces:
modeling the dynamic and geometric phenomena (possibly high
dimensional) that take place in 3-space at the human scale, mainly using multiple images of the same scene (as
acquired by a camera in different positions, systems of multiple cameras, or
video).
My interests
is in devising robust, precise
and automatic methods for tackling problems in this area. In order to meet such
requirements, I explore the use of grouped primitives such as curves and their
differential geometry as the basis for the methods, complementing
interest points and other approaches. The computational end-goal is a system based
on video sequences, that would work for any type of scene
geometry, without need for calibration or textured regions, being able to
identify not only the 3D structure of objects, but also their dynamics, camera
position, the reflectance properties, and lighting conditions.
- Collaborators: Ben Kimia's group at Brown, Peter Giblin, Anil Usumezbas from SRI, and Peter Olver.
- Additional information Multiview Differential Geometry of Curves and Surfaces
- Geometric Machine Learning/Manifold Learning: the understanding of typically
high-dimensional patterns or manifolds from images, for applications such as robust recognition, tracking, occlusion completion, through Diffusion Maps, digital exterior calculus, and other general or
specialized manifestations of differential geometry somewhat related to Hodge
Theory, using principles from Pattern Theory. For instance, I have been
applying this to the 3D modeling of ocean waves from multiple videos.
- Collaborators: Francisco Duarte Moura Neto (Ph.D. Berkeley), a mathematician working on inverse problems, my colleague at UERJ. He is my primary and close collaborator in this front. Check out his book on Inverse Problems. We are finishing up a book on Diffusion Maps.
Selected Publications
- Camera Pose Estimation Using Curve Differential Geometry, IEEE Transactions on Pattern Analysis and Machine Intelligence - PAMI, 2020, Ricardo Fabbri, Peter J. Giblin and Benjamin Kimia (pdf | code | bib) new!
- Trifocal Relative Pose from Lines at Points and its Efficient Solution, CVPR 2020 (online march 23 2019 4:29 UTC), Ricardo Fabbri, Timothy Duff, Hongyi Fan, Margaret Regan, David de Pinho (my former MSc. Student), Elias Tsigaridas, Charles Wampler, Jonathan Hauenstein, Peter Giblin, Benjamin Kimia, Anton Leykin and Tomas Pajdla (pdf |
code | datasets | bib)
accepted!
Awarded: NSF Research Institute Highlight- The research led and mentored by Fabbri (with close leadership by his former advisor Ben Kimia at Brown) have been spawning a number of notorious works, including an award-winning paper at CVPR 2019 (Duff, Kohn, Leykin, Pajdla) which Fabbri pioneered while at ICERM. Fabbri's interest is in pushing the results beyond points and straight lines, while using sophisticated machinery of Projective Differential Geometry and nonlinear PDEs for building fast and robust algorithms.
- The Surfacing of Multiview 3D Drawings via Lofting and Occlusion Reasoning, CVPR 2017, Ricardo Fabbri, Anil Usumezbas and Benjamin Kimia, Expanded version available (pdf | poster | datasets | website | bib)
- Multiview Differential Geometry of Curves,
International Journal of Computer Vision, 2016, Ricardo Fabbri and Benjamin Kimia. (pdf | code | bib)
- The main result is the reconstruction of torsion and curvature derivative of a space curve from multiple perspectives, a fundamental theorem I proved in 2004. This paper is the outgrowth of that effort, including continuous motion, non-rigid scenes, intrinsic parameters and camera estimation from occluding contours. It is the seed for a monograph I've been writing, a kind of an extended Hartley & Zisserman's book for general curves and differential geometry. For instance, if you need to reconstruct orientations from two or more views (SIFT orientations or other vector fields, including curvature and torsion fields), this is the place to go.
- From Multiview Image Curves to 3D Drawings, ECCV 2016, Ricardo Fabbri, Anil Usumezbas and Benjamin Kimia, Expanded version available (pdf | supplement | code | datasets | website | bib)
- Camera Pose Estimation Using Curve Differential Geometry, Updated and published into PAMI 2020!, ECCV 2012, Firenze, Italy, Ricardo Fabbri, Peter J. Giblin and Benjamin Kimia (pdf | code | bib)
- Multiview Differential Geometry in Application to Computer Vision, Ricardo Fabbri, Ph.D. Thesis, School of Engineering, Brown University, 2010. Partly superseded by the above papers, still has valuable insight (pdf | bib)
- 3D Curve Sketch: Flexible Curve-Based Stereo Reconstruction and Calibration, CVPR 2010, Ricardo Fabbri and Benjamin Kimia. The practical basis of the 3D Curve Drawing work above (pdf | poster | code | website | bib)
- 2D Euclidean Distance Transform Algorithms: A Comparative
Survey, ACM Computing Surveys, 2008. Ricardo Fabbri and Luciano da F. Costa, Julio C. Torelli and Odemir M. Bruno.
My most cited work, Actually used in the 3D Curve Sketch and Drawing system for multiview curve stereo (pdf | bib | errata | supplement | code)
- Evolved from my 2004 master thesis, which contains additional information on the Image Foresting Transform and color figures (pdf | presentation )
- High-Order Differential Geometry of Curves for Multiview Reconstruction and Matching, EMMCVPR 2005, Lecture Notes in Computer Science, 3757, pp. 645-660, 2005, Ricardo Fabbri and Benjamin Kimia. A new theorem on reconstructing the torsion of a space curve from multiple perspectives, please refer to my IJCV'16 paper (original | corrected)
- On Voronoi Diagrams and Medial Axes Journal of Mathematical Imaging and Vision, 17(1), 27-40, jul 2002, Ricardo Fabbri, Leandro F. Estrozi and Luciano da F. Costa, my first journal paper
- Remaining papers in Google Scholar
Teaching
- 2020/1 (2020/2 during pandemics)
- Riemannian Geometry (graduate) (sample lecture)
- Stochastic Processes (undergraduate) (sample lecture) (YouTube playlist)
- 2019/2
- Computational Geometry (graduate)
- Numerical Linear Algebra (undergraduate)
- Real Analysis (undergraduate, split)
- Discrete Mathematics II (undergraduate, split)
- 2019/1
- Computational Techniques (graduate)
- Stochastic Processes (undergraduate)
- 2018/2
- Differentiable Manifolds (graduate)
- Numerical Linear Algebra (undergraduate)
- 2018/1
- Stochastic Processes (undergraduate)
- 3D Computer Vision (undergraduate)
- 2017/2
- Differentiable Manifolds (graduate)
- Numerical Linear Algebra (undergraduate)
- 2017/1
- Geometric Measure Theory (graduate)
- Parallel Computing (graduate)
- 2016/2
- Differentiable Manifolds (graduate)
- 2016/1
- Riemannian Geometry (graduate)
- 2013/2
- Computer Vision (graduate and undergraduate)
- Computer Graphics (undergraduate)
- 2013/1
- Pattern Theory and Applications (graduate)
- Numerical Linear Algebra (undergraduate)
- 2012/2
- Parallel Computing (graduate)
- Discrete Mathematics I (undergraduate)
- 2012/1
- Algorithm Design (undergraduate)
- Matrix Analysis (graduate)
PhD Students and Notable Msc Students
Current- PhD: Juliana Chagas Ventura - Machine learning by Graph Diffusion and Pattern Theory for the 3D reconstruction of the trinocular geometry of Curves- unites my past PhD student Lucia Pinto's work with my past MSc student David Pinho's work
- PhD: Lucia Maria dos Santos Pinto - Diffusion Maps, geometric diffusions, manifold learning, machine learning, applied differential geometry and stochastic modeling
- PhD: Mauro Amorim - 3D reconsruction of the ocean surface from multiple self-calibrating video streams using curve and surface differential geometry, pattern theory, and nonlinear dimentionality reduction
- Msc: David da Costa de Pinho - the theory of trifocal pose estimation using differential geometry. Recently published a paper on a new trifocal camera pose solver.
- Msc: Caio dos Santos Souza - 3D reconsruction of the ocean surface from multiple self-calibrating video streams using curve and surface differential geometry, pattern theory, and nonlinear dimentionality reduction. Caio started with machine-learning in edge detection for obtaining suitable representations of ocean waves to be used in photogrammetry.
Interests
- Computer Vision and Image Processing
- Automated 3D reconstruction from multiple views, camera auto-calibration, shape from shading
- Algorithms related to propagation and curve evolution: diffusion maps, distance transforms, medial axes for shape representation, image segmentation, Voronoi diagrams, geodesics, shortest paths, inpainting
- Use of shape/context/geometric information in computer vision and machine learning
- Applying Mathematics (esp. geometry) to computer vision and machine learning
- Differential Geometry, Hogdge Theory, Singularity Theory, Catastrophe Theory
- Algebraic Geometry (solving systems of polynomial equations, projective geometry), Geometric Algebra
- Partial Differential Equations, specially geometric aspects
- Inverse problems connected to geometry
- Computational Geometry
Software
- Lab Macambira: an open source development group I co-founded, focusing on advanced audio, vision and web technolgy
- VxL C++ Computer vision libraries. The foundation of my research code, as fostered at Brown with Joe Mundy
- SIP: Image processing toolbox I wrote for Scilab, similar to Matlab but with true freedom
- Gentoo Linux: an excellent OS for computer scientists and experienced users.
Research and study material
- Fabbri's lecture notes on Numerical Linear Algebra for undergrads (pdf) new!
- Older: Summary of multiple view geometry as in Hartley & Zisserman's book (pdf | ppt)
- Older: Summary of an important paper in Auto-Calibration (pdf | odp)
- Older: 3D Reconstruction from a Handheld camera: a project page on point-based SfM.
Links
- Social Twitter | Facebook | Blog
- My brother's sonic compositions Renato Fabbri
- Comics and Animations by my Cousin and Friends Pula Pirata
- Other PhD Alumni from LEMS / Brown University
Curiosities
Do the vertical borders of this page look strange to you? They are exactly the same gradients, reflected vertically. On their own, they should look convex or concave. Is your brain is hypothesizing a global light source, and both bars can't be convex and consistent with the same light source? There are also biases in direction - the brain tends to favor certain directions for the light source over others. Some curious illusions: convex concave crater illusion, and the hollow-face illusion.