目录

目录

SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization


<SDFDiff> Sdfdiff: Differentiable rendering of signed distance fields for 3d shape optimization

Review

  • 需要分割好的多视角图片

Motivation

  • image-based shape optimization using differentiable rendering of 3D shapes represented by SDF
    • SDF作为形状表征的优势:可以表征具有任意拓扑的形状,并且可以保证watertight

Overview

  • learn SDF on a 3D grid

  • perform ray-casting via sphere tracing

  • differentiable renderer

    • 学到的是voxelized SDF,然后通过linear interpolation获取任意连续位置处的SDF
    • 给定像素值的导数只与interpolation时的8个邻居体素有关
      • 或者说,sphere tracing本身不需要是可微分的
      • 只需要 local 8个邻居的 local 计算需要可微分
  • energy function & losses

    • 从geometry相机位置等\(\Theta\),可以render出image\(I\)\(I=R(\Theta)\)
      inverse rendering就是\(\Theta=R^{-1}(I)\)
      但是inverse rendering并不直接可逆,因此把问题建模为energy minimization problem能量最小问题
      \(\Theta^*=\underset{\Theta}{\arg\min} \mathcal{L}_{img}(R(\Theta),I)\)
    • 重点在于一个differentiable renderer:本篇强调shape。输入camera pose和shape,输出渲染图像
    • \(\mathcal{L}_{img}\)衡量render图像和\(I\)的差别
    • \(\mathcal{L}_{reg}\) 正则化项,保证\(\Theta\)是一个valid signed distance field(i.e. 梯度是单位向量)
      实践中,是用\(\Delta\)近似的梯度
  • single view:从图像encode到一个voxelized 稀疏SDF,经过一些3D卷积refinement,经过differentiable renderer到imagehttps://longtimenohack.com/posts/paper_reading/2020cvpr_jiang_sdfdiff/image-20201222103114471.png

  • multi view:就用auto-decoder直接训练

results

  • single view
    https://longtimenohack.com/posts/paper_reading/2020cvpr_jiang_sdfdiff/image-20201222110938605.png