Liquid AI: The Underdog in the Shadow of GenAI

The current AI landscape is dominated by GenAI, or in other words, generative models such as Large Language Models (LLMs). LLMs are attention-based models called transformers, and are at the heart of the AI hype. However, it’s easy to overlook groundbreaking innovations that don’t produce flashy text, images, videos, and even 4D-editing. One such overlooked marvel of innovation is Liquid Neural Networks (LNNs). LNNs are based on a fundamentally different approach to neural computation that could revolutionize how we build AI systems for the real world. While everyone’s eyes are on the next GPT model, LNNs are quietly solving problems that transformers can’t even touch. This article serves as a gentle explanation of LNNs, and makes the case for more attention on LNNs, as beyond the hype, true AI researchers, engineers, practitioners, and enthusiasts need to be aware or reminded of this powerful algorithmic technology.

What Are Liquid Neural Networks?

Hasani et al. revived the concept of LNNs in their foundational paper Liquid Time-constant Networks (2021)¹, represent a radical departure from traditional neural architectures. Unlike conventional neural networks with fixed weights and static connections, LNNs are dynamic systems inspired by the nervous system of the C. elegans worm—a creature with just 302 neurons that exhibits remarkably complex behaviors.

The key innovation lies in their continuous-time dynamics. While traditional neural networks process information in discrete steps, LNNs model neurons as differential equations that evolve continuously over time:

\tau \frac{dx_i}{dt} = -x_i + f\left(\sum_{j} w_{ij}(t) \cdot x_j + I_i\right)

Where: - x_i represents the state of neuron i - \tau is the time constant (which can vary) - w_{ij}(t) are time-varying synaptic weights - f is the activation function - I_i is the external input

This seemingly simple equation hides profound implications. The time-varying weights w_{ij}(t) allow the network to adapt its connectivity patterns based on the input, creating a truly dynamic computational graph.

The Architecture That Adapts

To understand the fundamental difference between traditional neural networks and LNNs, consider their architectural principles:

Aspect	Traditional Neural Network	Liquid Neural Network
Architecture	Fixed layers with predetermined connections	Dynamic connections that adapt during inference
Weights	Static weights learned during training	Time-varying weights that respond to input patterns
Information Flow	Information flows in one direction	Bidirectional information flow
Time Processing	Discrete time steps	Continuous-time evolution
Data Requirements	Requires extensive training data	Learns from limited data
Adaptability	Cannot adapt after deployment	Adapts to new situations in real-time

The mathematical representation highlights this difference. In a traditional network, the output is simply:

y = f(W \cdot x + b)

In contrast, a liquid network’s state evolves according to coupled differential equations:

\begin{aligned} \frac{dx_i}{dt} &= \frac{1}{\tau_i(t)} \left[-x_i + f\left(\sum_j w_{ij}(t) \cdot x_j + I_i\right)\right] \\ \tau_i(t) &= \tau_{min} + (\tau_{max} - \tau_{min}) \cdot \sigma(v_i^T \cdot x + b_i) \end{aligned}

This continuous-time formulation allows LNNs to naturally process temporal information without the need for recurrent connections or attention mechanisms.

The architecture of LNNs is fundamentally different from traditional networks. Instead of fixed layers with predetermined connections, LNNs feature:

1. Dynamic Synapses: Connections that strengthen or weaken based on the temporal patterns in the data
2. Adaptive Time Constants: Neurons that can speed up or slow down their responses
3. Sparse Connectivity: Highly efficient architectures with orders of magnitude fewer parameters

Why LNNs Matter (And Why We’re Ignoring Them)

While the tech world obsesses over the latest 175-billion parameter language model, LNNs are achieving remarkable results with just a few thousand parameters. Consider these achievements:

• Autonomous Driving: LNNs with 19 neurons matching the performance of deep networks with 100,000 parameters²
• Time-Series Prediction: Superior performance on chaotic systems like weather prediction
• Robustness: Exceptional resistance to adversarial attacks and distribution shifts
• Interpretability: With fewer neurons, we can actually understand what the network is doing

The irony is palpable. While we pour billions into training ever-larger transformers that require data centers to run, LNNs can run on a microcontroller and adapt in real-time to changing conditions.

The Mathematics of Adaptation

The secret sauce of LNNs lies in their liquid time-constant mechanism. Unlike fixed neural networks, the time constant \tau itself becomes a learnable function:

\tau_i = \sigma(W_\tau \cdot [x_i, I_i] + b_\tau)

This allows each neuron to adjust its temporal dynamics based on the current state and input. The result is a network that can naturally handle:

• Variable-length sequences without padding or truncation
• Irregular sampling rates in sensor data
• Long-term dependencies without gradient vanishing

The synaptic weights also follow their own dynamics:

\frac{dw_{ij}}{dt} = \alpha \cdot (A_{ij} - w_{ij}) \cdot g(x_i, x_j)

Where A_{ij} represents the target connectivity pattern and g is a gating function. This creates a network that literally rewires itself during inference.

Real-World Applications: Where LNNs Shine

The efficiency advantage of LNNs becomes stark when we compare their computational requirements:

This 100-1000x reduction in computational requirements isn’t just about efficiency—it fundamentally changes where and how AI can be deployed.

LNNs excel in domains where traditional deep learning struggles:

1. Edge Computing: Running sophisticated AI on devices with limited computational resources
2. Adaptive Control: Robots and drones that need to adapt to changing environments
3. Medical Devices: Implantable systems that must operate for years on minimal power
4. Financial Systems: Trading algorithms that adapt to market dynamics in real-time

The Overshadowing Effect

So why aren’t LNNs getting the attention they deserve? The answer lies in the economics and psychology of AI development:

1. The Wow Factor: GenAI produces immediately impressive results that capture public imagination
2. Investment Momentum: Billions flow into transformer-based models, creating a self-reinforcing cycle
3. Publication Bias: Papers on large language models get more citations and media coverage
4. Complexity Bias: The assumption that bigger and more complex must be better

This creates a dangerous monoculture in AI research. While we optimize for benchmark performance and parameter counts, we’re missing opportunities to build AI systems that are: - More energy-efficient - More interpretable - More robust to real-world conditions - More suitable for safety-critical applications

Looking Forward: The Liquid Future

The potential of LNNs extends far beyond their current applications. Imagine:

• Hybrid Systems: LNNs handling real-time perception and control while transformers handle high-level reasoning
• Neuromorphic Computing: Hardware specifically designed for liquid dynamics, achieving unprecedented efficiency
• Biological Integration: Brain-computer interfaces that speak the language of biological neural networks
• Swarm Intelligence: Networks of simple LNN agents exhibiting complex collective behaviors

Conclusion: Don’t Forget the Underdogs

As we stand in awe of the latest generative AI achievements, let’s not forget that intelligence comes in many forms. Liquid Neural Networks represent a fundamentally different approach to artificial intelligence—one that prioritizes efficiency, adaptability, and real-world performance over raw computational power.

The next breakthrough in AI might not come from adding another trillion parameters to a transformer. It might come from understanding how 302 neurons in a microscopic worm can navigate, hunt, and survive in a complex world. In our rush toward artificial general intelligence, let’s not overlook the elegant solutions that nature has already provided.

To Cite This Article

@misc{SadouneBlog2025h,
  title = {Liquid AI: The Underdog in the Shadow of GenAI},
  author = {Igor Sadoune},
  year = {2025},
  url = {https://sadoune.me/posts/liquid_ai/}
}

Footnotes

1. Hasani, R., Lechner, M., Amini, A., Rus, D., & Grosu, R. (2021). Liquid Time-constant Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 7657-7666.

2. Lechner, M., Hasani, R., Amini, A., Henzinger, T. A., Rus, D., & Grosu, R. (2020). Neural circuit policies enabling auditable autonomy. Nature Machine Intelligence, 2(10), 642-652.