目录

目录

DSM-Net: Disentangled Structured Mesh Net for Controllable Generation of Fine Geometry


Dsm-net: Disentangled structured mesh net for controllable generation of fine geometry

Motivation

  • 把structure(topology)和geometry进一步解耦,in a synergistic manner
  • https://longtimenohack.com/posts/paper_reading/2020arxiv_yang_dsm/image-20201217154222663.png

Overview

  • 用Recursive Neural Networks(RvNNs, 注意RNN是recurrent NN) hierarchically encode和decode structure和geometry,在hierarchy的每一层都有bijective mapping
    https://longtimenohack.com/posts/paper_reading/2020arxiv_yang_dsm/image-20201217155349248.png
  • 同时用两个分开的但是高度耦合的VAE学习structure 和geometry,把他们encode into two latent spaces

disentangled shape representation

structure hierarchy抽象出符号部件(symbolic parts)与关系

  • inspired by PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions. 2020
  • 每个部件用semantic label (e.g. chair back, chair leg)表示,引入PartNet dataset中丰富的部件关系
    • \(\boldsymbol{\rm H}\) 纵向的parent-child inclusion 关系 (e.g. chair back and chair back bars)
    • \(\boldsymbol{\rm R}\) 横向的among-sibling 部件对称性与邻接性(e.g. chair back bars have translational symmetry)

geometry hierarchy是部件的geometry

  • 表征就是正常的多顶点mesh
  • 假设一个5402顶点构成的封闭mesh,计算oriented bounding box
  • 然后通过non-rigid registration 变形这个mesh到target part geometry
  • 然后用ACAP作为部件表征
    • Sparse data driven mesh deformation. 2019
    • SDM-NET: Deep Generative Network for Structured Deformable Mesh. 2019

structure hierarchy和geometry hierarchy之间有bijective mapping

  • 符号部件 \(l_i\) 对应部件geometry \(G_i\) ,层级 \(\boldsymbol{\rm H}\) 和关系 \(\boldsymbol{\rm R}\) 则隐式地互相一致
    • 在学习的时候两个hierarchies有communication channels
  • 虽然结构和几何要解耦,但是他们还是需要彼此兼容来产生好的、现实的形状
    • 一方面,shape structure 为 part geometry提供high-level guidance
      • e.g. 如果four legs of a chair对称,那么他们应该具有identical part geometry
    • 另一方面,给定part geometry以后,只有若干种适用的shape structures(而不是全部)
      • e.g. 如果没有lift handle或者gas cylinder parts,不可能组装一个swivel chair

conditional part geometry VAE

  • encode和decode时候都condition on part structure information
  • https://longtimenohack.com/posts/paper_reading/2020arxiv_yang_dsm/image-20201217162903345.png

Disentangled Geometry and Structure VAEs

  • 下图蓝色代表geometry,红色代表structure
    encoding的时候,从geometry和structure feature encode出geometry
    decoding的时候,从geometry和structure feature decode出geometry
  • https://longtimenohack.com/posts/paper_reading/2020arxiv_yang_dsm/image-20201217162351563.png
  • Q: what?

results

  • https://longtimenohack.com/posts/paper_reading/2020arxiv_yang_dsm/image-20201217163302020.png