SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
<SDFDiff>
Sdfdiff: Differentiable rendering of signed distance fields for 3d shape optimization目录
Review
- 需要分割好的多视角图片
Motivation
- image-based shape optimization using differentiable rendering of 3D shapes represented by SDF
- SDF作为形状表征的优势:可以表征具有任意拓扑的形状,并且可以保证watertight
Overview
learn SDF on a
3D grid
perform ray-casting via
sphere tracing
differentiable renderer
- 学到的是voxelized SDF,然后通过linear interpolation获取任意连续位置处的SDF
- 给定像素值的导数只与interpolation时的8个邻居体素有关
- 或者说,sphere tracing本身不需要是可微分的
- 只需要 local 8个邻居的 local 计算需要可微分
energy function & losses
- 从geometry相机位置等\(\Theta\),可以render出image\(I\):\(I=R(\Theta)\)
inverse rendering就是\(\Theta=R^{-1}(I)\)
但是inverse rendering并不直接可逆,因此把问题建模为energy minimization problem
能量最小问题
\(\Theta^*=\underset{\Theta}{\arg\min} \mathcal{L}_{img}(R(\Theta),I)\) - 重点在于一个differentiable renderer:本篇强调shape。输入camera pose和shape,输出渲染图像
- \(\mathcal{L}_{img}\)衡量render图像和\(I\)的差别
- \(\mathcal{L}_{reg}\) 正则化项,保证\(\Theta\)是一个valid signed distance field(i.e. 梯度是单位向量)
实践中,是用\(\Delta\)近似的梯度
- 从geometry相机位置等\(\Theta\),可以render出image\(I\):\(I=R(\Theta)\)
single view:从图像encode到一个voxelized 稀疏SDF,经过一些3D卷积refinement,经过differentiable renderer到image
multi view:就用auto-decoder直接训练
results
- single view