目录

目录

DL methods for shape as explicit shape templates + deformation

[形状] 视作 空间曲面, 显式三角面模板+变形 的DL方法


目录
<CMR> Learning category-specific mesh reconstruction from image collections
 

Motivation

https://longtimenohack.com/posts/paper_reading/2018eccv_kanazawa_lerning/image-20201207225935008.png

Overview

https://longtimenohack.com/posts/paper_reading/2018eccv_kanazawa_lerning/image-20201207230015167.png

  • 一张图片encode到一个latent space, 被三个模块共享

  • shape predictor,学到的是从mean shape出发的顶点的位移改变量

  • texture predictor,学到的是从输入图像的texture flow

  • camera predictor,学到的是canonical space下的camera pose

  • deformation predictor事实上学到的是从一个learned mean shape的变形 texture使用标准UV映射定义

  • mesh定义在canonical frame下 mean shape和sphere有相同的geometry

    • 相同的顶点连接性,相当于fixed topology,拓扑是固定的
      • 思考甜甜圈和咖啡杯的拓扑是一样的:通过顶点移位变形可以变形过去
      • a fixed and pre-determined mesh connectivity 连接性是固定的
    • 所谓shape predictor,其实是预测固定个数的vertices的位置改变
      https://longtimenohack.com/posts/paper_reading/2018eccv_kanazawa_lerning/image-20201207231029039.png
    • 我们可以从uv图的坐标映射到球面坐标,再映射到mean shape上的坐标,再通过shape 变形(顶点移位)映射到当前shape上的顶点坐标
  • texture predictor 事实上学到的是从单张图片出发的texture flow https://longtimenohack.com/posts/paper_reading/2018eccv_kanazawa_lerning/image-20201207232213580.png

Learning shape templates with structured implicit functions
 

Review

  • learning generalized templates comprised of elements

Motivation

  • 给这类从canonical space下的shape template学出物体shape的方法,提供一种更通用于各种类别的shape template 学习方法
  • 由于现实世界的形状和拓扑变化丰富,过去的_这类_方法一般用a library of handmade templates
  • 本篇使用了一种基于若干个local shape elements的组合来构成shape template;
    每个element是一个隐式的surface representation
    • 每个element可以当做一个高斯椭球形状
    • 这样,不同的elements位置、扁圆、大小组合,就可以组合出==不同形状、不同拓扑==的shape template
  • 使用10,25,100个不同的elements训练的效果
    https://longtimenohack.com/posts/paper_reading/2019iccv_genova_learning/image-20201207235340273.png

隐式的shape表征

  • 假定每一个input shape都可以建模为一个watertight surface,由一个函数的 \(\mathcal{l}\) level set描述(l-等值面集);
  • 这个函数可以由N个local elements构成
  • 每个elements是一个 scaled axis-aligned anisotropic 3D Gaussians
    由参数 \(\theta_i\) 描述,\(\theta_i\) 包含 \(c_i, p_i \in \mathbb{R}^3, r_i \in \mathbb{R}^3\)
    https://longtimenohack.com/posts/paper_reading/2019iccv_genova_learning/image-20201208000148898.png
Deep mesh reconstruction from single rgb images via topology modification networks
 

Motivation


  • https://longtimenohack.com/posts/paper_reading/2019iccv_pan_deep/image-20201208110645619.png
  • 优化的时候,可以alternates between shape deformation和topology modification

overview

  • topology modification
    • 通过动态地修改 faces-to-vertices关系来实现
    • 学一个per face error estimation network
    • 通过去掉那些deviate significantly的face来更新topology structure

效果

  • https://longtimenohack.com/posts/paper_reading/2019iccv_pan_deep/image-20201208111115100.png
  • https://longtimenohack.com/posts/paper_reading/2019iccv_pan_deep/image-20201208111142570.png