NanoFlow: Scalable Normalizing Flows with Sublinear Parameter Complexity.

Normalizing flows (NFs) have become a prominent method for deep generativemodels that allow for an analytic probability density estimation and efficientsynthesis. However, a flow-based network is considered to be inefficient inparameter complexity because of reduced expressiveness of bijective mapping,which renders the models unfeasibly expensive in terms of parameters. Wepresent an alternative parameterization scheme called NanoFlow, which uses asingle neural density estimator to model multiple transformation stages. Hence,we propose an efficient parameter decomposition method and the concept of flowindication embedding, which are key missing components that enable densityestimation from a single neural network. Experiments performed on audio andimage models confirm that our method provides a new parameter-efficientsolution for scalable NFs with significant sublinear parameter complexity.