主页

Survey: nerf + surface enhancements

<UNISURF> Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction  ICCV2021   arXiv preprint arXiv:2104.10078 Michael Oechsle, Songyou Peng, Andreas Geiger University of Tübingen, ETH Zurich    MPI volume rendering occupancy network Cite Preprint   目录 Motivation task:做了什么 核心 insight: neural radiance model 和 neural implicit shape model 可以用一种统一的方式建模 diss 目前: Overview 对 nerf 的魔改: 思路: 训练 结果 Implementation 网络结构 最优化过程 数据集 DTU Indoor Scene from SceneNet BlendedMVS Motivation task:做了什么 从多视角无 mask 图像中重建表面,并且合成新视角观测 核心 insight: neural radiance model 和 neural implicit shape model 可以用一种统一的方式建模 更高效的 sampling 过程 没有 input mask (不像 DVR,IDR 那样)的情况下也可以学到精确的表面 diss 目前: nerf: cons:没有 accurate surface pros:对非 solid scene 也能用,比如烟雾;本文 focus on solid objects DVR / IDR: pros:可以从图像重建精确表面; cons: 需要 per-pixel mask;🤔 注意 per-pixel mask 和 sihoulette 的区别 per-pixel mask 意味着物体上那些有空洞的区域也要扣掉;不然会被认为是背景色实体 网络需要适当的初始化,因为 表面渲染技术 只能在局部提供梯度信息(也就是光线和表面的交点区域) -> 不像 nerf 那样整个空间都密布着梯度 直觉上讲,这种利用局部梯度信息的最优化过程就是在迭代对初始形状(总是一个球)进行变形 Overview 对 nerf 的魔改: \(\alpha(x) = 1-\exp\left(-\sigma(\mathbf{x})\delta\right)\) 直接改为 \(o(x)\), 即把 nerf 渲染过程中的 \(\alpha(x)\) 替换为\(o(x)\) ,即 Occupancy field,取值 \([0,1]\),\(o=0.

DL methods for shape as explicit shape templates + deformation

<CMR> Learning category-specific mesh reconstruction from image collections  ECCV2018   Proceedings of the european conference on computer vision (ECCV) Angjoo Kanazawa, Shubham Tulsiani, Alexei A Efros, Jitendra Malik UCB    category-specific canonical shape template Cite PDF Code Project   目录 Motivation Overview Motivation Overview 一张图片encode到一个latent space, 被三个模块共享 shape p

part/structure-aware shape representation

category dense shape correspondences 自监督/无监督地学习形状相关性/结构共性 <DIF> Deformed implicit field: Modeling 3d shapes with learned dense correspondence  CVPR2021   Proceedings of the IEEE/CVF conference on computer vision and pattern recognition Yu Deng, Jiaolong Yang, Xin Tong Tsinghua    MSRA 3D deformation field, template field, category shape correspondence Cite Preprint Code   目录 Motivation overview losses SDF

Survey: categorical nerf

<GRAF> GRAF: Generative radiance fields for 3D-aware image synthesis  NeuralIPS2020   Advances in Neural Information Processing Systems Katja Schwarz, Yiyi Liao, Michael Niemeyer, Andreas Geiger MPI Cite PDF Code Project   目录 Motivation 注:笔记在纸质版。待迁移电子版 Motivation While 2D generative adversarial networks have enabled high-resolution image synthesis, they largely lack an understanding of the 3D world and the image formation process.

Survey: compositional nerf

<GIRAFFE> Giraffe: Representing scenes as compositional generative neural feature fields  CVPR2021 ✔️ BEST   Proceedings of the IEEE/CVF conference on computer vision and pattern recognition Michael Niemeyer, Andreas Geiger University of Tübingen    MPI Cite PDF Code   目录 review Motivation overview 注:目前笔记在纸质打印版,待后续迁移电子版 review 用neural rendering “避开"了多物体lighting的显式建模 Motivation overview <OSF> Object-centric neural scene rendering  arXiv2020   arXiv preprint arXiv:2012.