Researchers at MIT Solve a Differential Equation Behind the Interaction of Two Neurons Through Synapses to Unlock a New Type of Speedy and Efficient Artificial Intelligence AI Algorithm

Continuous-time neural networks are a subset of machine learning systems capable of performing representational learning for spatiotemporal decision-making tasks. Continuous differential equations are often used to represent these models (DEs). However, numerical DE solvers limit their expressive potential when used on computers. The scaling and understanding of many natural physical processes, such as the dynamics of neural systems, have been severely hampered by this limitation.

Inspired by the brains of microscopic creatures, MIT researchers have engineered “Fluid” neural networks, a fluid, robust ML model that can learn and adapt to changing situations. These methods can be used in safety-critical tasks such as driving and flying.

However, as the number of neurons and synapses in the model increases, the underlying math becomes more difficult to solve and the processing cost of the model increases.

Neural network systems based on differential equations are difficult to solve and scale to a large number of parameters. Complex neural networks can be built with a physical description of the cell interactions along with the threshold. Any future embedded intelligence system should use this framework as a foundation as it can help solve more complex machine learning tasks by enabling enhanced representation learning.

The same research group has now found a way to overcome this hurdle by solving the differential equation underlying the interaction of two neurons through synapses. They then present the new machine learning models called closed-form continuous-time (CfCs), which retain the attractive properties of fluid networks but eliminate the need for numerical integration.

This method is as fast and scalable as fluid neural networks, and they share their causal, robust, and explainable properties. Because these neural networks are small and evolve after training, they can be used for any task that requires an understanding of data over time, whereas many traditional models are rigid.

This paves the way for reliable machine learning in mission-critical environments. Not only does this differential equation no longer have to be solved step by step, the computing time has also been greatly reduced.

The models significantly outperformed their state-of-the-art counterparts in various tasks, including recognizing human behavior using motion sensors, modeling the physical dynamics of a simulated walking robot, and processing events in sequential images. For example, the models were 220 times faster than human experts on a task to predict mortality for a group of 8,000 patients.

.

The closed form solution is a good approximation of the actual system dynamics. So if you replaced them within that network you would get the exact behavior. Therefore, they can solve it with even fewer neurons, making it computationally cheaper and faster.

Time series (events that have occurred over time) can be used as inputs to these models, which can then be used for categorization, vehicle control, humanoid robot motion, or even financial and medical forecasting. Many different settings can be used to improve the system’s accuracy, resilience, performance and – most importantly – calculation speed, although these improvements are not always free of charge.

Finding a solution to this equation will have far-reaching consequences for the study of both natural and artificial intelligence. The results of this research show how increasing the computational efficiency for this category of neural networks can open up new possibilities for use in areas such as safety-critical commercial and defense systems.

Try this paper, code, record, and reference article. All credit for this research goes to the researchers on this project. Also don’t forget to participate our Reddit page and Discord Channelwhere we share the latest AI research news, cool AI projects and more.