How do you implement long-term memory in an AI agent, c...

The pattern that works reliably: after a conversation ends, extract the facts worth keeping rather than storing the raw transcript, embed those extracted summaries, and store the embeddings in a vector database. When a new conversation starts, embed the incoming query, run a similarity search against stored memories, and inject whatever comes back into the system prompt before the agent responds.

A user who discussed building a FastAPI project with an Azure SQL backend gets that extracted and stored as a short summary. Days later, “help me fix my API” triggers a similarity search that surfaces exactly that memory, and the agent opens already knowing the context instead of asking the user to re-explain their setup.

The trade-off worth being explicit about: vector search is the right tool for fuzzy, semantic recall — “something about their project” — but the wrong tool for facts that need an exact match, like a user’s plan tier or account status. Those belong in an ordinary structured database alongside the vector store, queried directly rather than via similarity.