Today I read.

Normalization methods in DNNs; today’s reading. 20 Sep 2023
  1. Normalization Techniques in Training DNNs: Methodology, Analysis and Application.
    Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, and Ling Shao.

    notes: A review and commentary of normalization methods in DNNs. I skimmed this one. Good for definitions of all of the normalization terms and especially Figure 1.

Whitening transformations and orthogonality of random variables; today’s reading. 08 Sep 2023
  1. Optimal Whitening and Decorrelation.
    Agnan Kessy, Alex Lewin, and Korbinian Strimmer.

    notes: Covers ‘whitening’, linear transforms that convert random vectors to another random vector, where the new random vector has covariance equal to the identity matrix. Five types discussed: zero-phase components analysis (ZCA) or Mahalanobis whitening, PCA whitening, Cholesky whitening, ZCA-cor, and PCA-cor. ZCA whitening is used in paper ‘CamP: Camera Preconditioning for Neural Radiance Fields’, Park et al. 2023. ‘Whitening’ is equivalent to the term ‘sphering’.

Preconditioning in NeRF; today’s reading. 06 Sep 2023
  1. CamP: Camera Preconditioning for Neural Radiance Fields.
    Keunhong Park, Philipp Henzler, Ben Mildenhall, Jonathan T. Barron, and Ricardo Martin-Brualla.

    notes: NeRF joint optimization of camera parameters and scene reconstruction. Uses a left preconditioner for each camera’s parameters (Zero Component Analysis (ZCA) whitening transform (Kessy et al. 2018)), derived from a projection function; apply this at the initial iteration of the optimization. The new method is implemented on top of Zip-NeRF (Barron et al. 2023).

Deep learning terminology and references in François Fleuret’s book; today’s reading. 01 Sep 2023
  1. The Little Book of Deep Learning.
    François Fleuret.

    notes: I really like this introduction to deep learning and reference guide. Want to remember a term without getting in too deep? This little book has it, and the top-level references if I want to read more. See the website to order a physical version, printing two book pages per printed page worked well for me too.

Code optimization in Python; Today’s reading. 31 Aug 2023
  1. Making Your Python Code Run Faster.
    Brandon Rohrer.

    notes: Profiling, vectorization, pre-compilation with Numba, 10 optimization suggestions, "try it and test it", examples presented in context of a physics simulation. Good discussions about troubleshooting and debugging, when to visualize, determining project goals.

Aerial sampling of insects with a UAS; Today’s reading. 30 Aug 2023
  1. Potential of Unmanned Aerial Sampling for Monitoring Insect Populations in Rice Fields.
    Hong Geun Kim, Jong-Seok Park, and Doo-Hyung Lee.

    notes: Need to monitor for seasonal insect migrations in rice fields. Uses a UAS with small nets to collect samples of insects at different altitudes. To my knowledge, the only work to collect insects with a UAS versus using already-tagged insects.

VAEs; Today’s reading. 29 Aug 2023
  1. Tutorial on Variational Autoencoders.
    Carl Doersch.

    notes: Published 2016 with edits in 2021. Tutorial on Variational Autoencoders, reparameterization trick, and Conditional Variational Autoencoders. Examples using MNIST. Does not assume prior knowledge of variational Bayesian methods.