The goal in this tutorial is to non-rigidly fit a shape model to a target surface using Iterative Closest Points (ICP) in order to establish correspondences among two surfaces.
The following resources from our online course may provide some helpful context for this tutorial:
As in the previous tutorials, we start by importing some commonly used objects and initializing the system.
Let's load and visualize a target mesh; I.e. a mesh, which we want to fit with our model, as well as a statistical shape model.
As you can see in the 3D scene, the instance of the model taht we are currently displaying (the mean), does not resemble the target face. The goal in shape model fitting is to find an instance of our shape model, which resembles at best the given target face. As we will see, a good fit directly leads to a way of establishing correspondences between the points of our model and the points of the target shape.
Iterative Closest Points (ICP) and GP regression
In a previous tutorial, we introduced rigid ICP to find the best rigid transformation between two meshes. We recall that the main steps of the algorithms are as follows:
- Find candidate correspondences between the mesh to be aligned and the target one, by attributing the closest point on the target mesh as a candidate.
- Solve for the best rigid transform between the moving mesh and the target mesh using Procrustes analysis.
- Transform the moving mesh using the retrieved transform
- Loop to step 1 if the result is not aligned with the target (or if we didn't reach the limit number of iterations)
The non-rigid ICP algorithm, which we can use for model fitting, will perform exactly the same steps. However, instead of finding a rigid transformation in step 2, it finds a non-rigid one, using Gaussian process regression.
We start by first selecting the points for which we want to find the correspondences. We choose uniformly distributed points on the surface, which we can obtain as follows:
Instead of working directly with the points, it is easier to work with the point ids of the sampled points:
As in the previous tutorial, we write the method
attributeCorrespondences, which finds for each
point of interest the closest point on the target.
We can now use the correspondences we found to compute a Gaussian process regression.
While this one fitting iteration does not bring the points where we would like them to have, we are already a step closer. As in the Rigid ICP case, we now iterate the procedure.
Repeating the fitting steps iteratively for 20 times results in a good fit of our model