Information-computation trade-offs in nonlinear transforms

In this work, we explore the interplay between information and computation in nonlinear transform-based compression for broad classes of modern information-processing tasks. We first investigate two emerging nonlinear data transformation frameworks for image compression: implicit neural representations (INRs) and two-dimensional (2D) Gaussian splatting (GS). We analyse their representational properties, behaviour under lossy compression and convergence dynamics. Our results highlight key trade-offs between INR's compact, resolution-flexible neural field representations and GS's highly parallelizable, spatially interpretable fitting, providing insights for future hybrid and compression-aware frameworks. Next, we introduce the textual transform that enables efficient compression at ultra-low bit rates-regimes, and simultaneously enhances human perceptual satisfaction. When combined with the concept of denoising via lossy compression, the textual transform becomes a powerful tool for denoising tasks. Finally, we describe a Lempel-Ziv (LZ, specifically LZ78) transform, a universal method that, when applied to any member of a broad compressor family, produces new compressors that retain the asymptotic universality guarantees of the LZ78 algorithm. Collectively, these three transforms illuminate the fundamental trade-offs between coding efficiency and computational cost. We discuss how these insights extend beyond compression to tasks such as classification, denoising and generative AI, suggesting new pathways for using nonlinear transformations to balance resource constraints and performance. This article is part of the discussion meeting issue 'Bits, neurons and qubits for sustainable AI'.