{"id":219742,"date":"2025-01-20T12:23:08","date_gmt":"2025-01-20T12:23:08","guid":{"rendered":"https:\/\/news.talkwithrattan.com\/index.php\/2025\/01\/20\/googles-new-titans-ai-architecture-could-give-llms-long-term-memory\/"},"modified":"2025-01-20T12:23:08","modified_gmt":"2025-01-20T12:23:08","slug":"googles-new-titans-ai-architecture-could-give-llms-long-term-memory","status":"publish","type":"post","link":"https:\/\/news.talkwithrattan.com\/index.php\/2025\/01\/20\/googles-new-titans-ai-architecture-could-give-llms-long-term-memory\/","title":{"rendered":"Google\u2019s New Titans AI Architecture Could Give LLMs ‘Long-Term Memory’"},"content":{"rendered":"
\"Google\u2019s<\/div>


\n<\/p>\n

\n

Google<\/a> researchers unveiled a new artificial intelligence (AI) architecture last week that can enable large language models (LLMs) to remember the long-term context of events and topics. A paper was published by the Mountain View-based tech giant on the topic, and the researchers claim that AI models trained using this architecture displayed a more \u201chuman-like\u201d memory retention capability. Notably, Google ditched the traditional Transformer and Recurrent Neural Network (RNN) architectures to develop a new method to teach AI models how to remember contextual information.<\/p>\n

Titans Can Scale AI Models’ Context Window More Than 2 Million Tokens<\/h2>\n

The lead researcher of the project, Ali Behrouz, posted<\/a> about the new architecture on X (formerly known as Twitter). He claimed that the new architecture provides a meta in-context memory with attention that teaches AI models how to remember the information at test-time compute.<\/p>\n

According to Google’s paper, which has been published<\/a> in the pre-print online journal arXiv, the Titans architecture can scale the context window of AI models to larger than two million tokens. Memory has been a tricky problem to solve for AI developers.<\/p>\n

Humans remember information and events with context. If someone asked a person about what he wore last weekend, they would be able to remember additional contextual information, such as attending a birthday party of a person who they have known for the last 12 years.This way, when asked a follow-up question about why they wore a brown jacket and denim jeans last weekend, the person would be able to contextualise it with all these short-term and long-term information.<\/p>\n

AI models, on the other hand, typically use retrieval-augmented generation (RAG) systems, modified for Transformer and RNN architectures. It uses information as neural nodes. So, when an AI model has been asked a question, it accesses the particular node that contains the main information, as well as the nearby nodes that might contain additional or related information. However, once a query is solved, the information is removed from the system to save processing power.<\/p>\n

However, there are two downsides to this. First, an AI model cannot remember information in the long run. If one wanted to ask a follow-up question after a session was over, one would have to provide the full context again (unlike how humans function). Second, AI models do a poor job of retrieving information involving long-term context.<\/p>\n

With Titans AI, Behrouz and other Google researchers sought to build an architecture which enables AI models to develop a long-term memory that can be continually run, while forgetting information so that it be computationally optimised.<\/p>\n

To this end, the researchers designed an architecture that encodes history into the parameters of a neural network. Three variants were used \u2014 Memory as Context (MAC), Memory as Gating (MAG), and Memory as a Layer (MAL). Each of these variants is suited for particular tasks.<\/p>\n

Additionally, Titans uses a new surprise-based learning systen, which tells AI models to remember unexpected or key information about a topic. These two changes allow Titans architecture to showcase improved memory function in LLMs.<\/p>\n

\n
\n

In the BABILong benchmark, Titans (MAC) shows outstanding performance, where it effectively scales to larger than 2M context window, outperforming large models like GPT-4, Llama3 + RAG, and Llama3-70B. pic.twitter.com\/ZdngmtGIoW<\/a><\/p>\n

\u2014 Ali Behrouz (@behrouz_ali) January 13, 2025<\/a><\/p><\/blockquote>\n<\/div>\n

In a separate post, Behrouz claimed that based on internal testing on the BABILong benchmark (needle-in-a-haystack approach), Titans (MAC) models were able to outperform large AI models such as GPT-4, LLama 3 + RAG, and LLama 3 70B.<\/p>\n<\/div>\n