Mar. - Apr. 2023

65 ideas from “Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations” (2019) by Vincent Sitzmann, Michael Zollhöfer, Gordon Wetzstein

I used this 2019 paper and the 2020 neural radiance fields paper as orientation for about 6 weeks; I dove into the citations and got a feel for what people are working on and what people are not working on within the emerging neural fields niche. How do humans represent scenes and how do we build machines that represent scenes in a more human-like manner? The brain does not implement ray tracing. The brain processes and reconstructs scenes compositionally. From one glance at an image or a scene, we can readily imagine novel views and mentally navigate through the imaged scene.

One exciting application of neural fields is in structural biology. My rabbit hole into mnemonics was just beginning to loop back into my undergrad biology interests. More on that later. I left off this month with the feeling like the neural fields space is incredibly competitive and fast-paced, like I'd just memorized the technical details of a paper that is already antiquated, and that if I want to eventually contribute to this area it should be by collaboration and via representation theory.