AI, like humans, needs long-term memory—not just intelligence—so I began exploring how to build the world’s first large-scale visual memory model.

Alumnus Shawn Shen

In 2024, Shen co-founded Memories.ai with Ben Zhou, another former researcher from Meta Reality Labs. Shen leads the company's technical vision and business strategy. The company was established to address what Shen identified as a critical gap in AI capabilities: long-term visual memory. Under his leadership, the company secured significant seed funding and began a talent acquisition campaign to compete with established technology giants for top AI researchers.

Tell me about your career path to date 

I’m the first in my family to attend university. That made education feel both personal and mission-critical. At Cambridge, my research centered on multimodal systems—teaching models to connect vision, language, and context. After my PhD, I went to Meta’s Reality Labs as a research scientist. Now, I am an assistant professor at the University of Bristol and also co-founder of Memories.ai.

What inspired you into your field?

When I was 14 years old, I moved from China to the UK on a scholarship to attend high school, leaving behind everything familiar: my home, my community, my friends. In those early months, I experienced firsthand how memory shapes who we are. I watched myself adapt to a completely new educational system and culture. I saw how my memories of home became both more precious and more distant. But what struck me most was how visual these memories were. I could close my eyes and see my childhood bedroom, remember the exact layout of my neighbourhood, and recall the faces of friends I'd left behind. These weren't just facts stored in my mind; they were rich, visual experiences that I could revisit and connect to new experiences in my UK school.

Most importantly, I realised that adaptation isn't just about learning new things. It's about how you connect new experiences to existing memories, how you build context over time through these visual connections. That journey taught me something profound about intelligence itself: memory isn't just storage. It's the foundation of understanding, learning, and growth.

Fast forward to my time at Meta Reality Labs, where my co-founder Ben (Enmin) Zhou and I worked side by side, often until midnight, pushing the boundaries of what AI could do. We published four top-tier research papers in a single year (work that typically represents an entire PhD career). But the more we advanced AI's capabilities, the more we realised we were missing something fundamental.

Current AI systems are incredibly intelligent, but they lack persistent memory.

How did you first get involved in your specialist area?

I first became involved in my field through computational neuroscience in my third year at Cambridge, working with Professor Máte Lengyel. I was fascinated by how the human brain encodes, stores, and retrieves information, and this inspired me to think about how memory could be built into machines. For my PhD, I worked with Professor Per Ola Kristensson on multimodal AI, studying how systems could combine language, vision, and interaction. That path led me to the realisation that AI, like humans, needs long-term memory—not just intelligence—so I began exploring how to build the world’s first large-scale visual memory model.

Please tell the story of Memories.ai and explain how it works

Most “AI memory” today is just text context. We’re different. Memories.ai builds the visual memory layer: we convert massive volumes of video into structured, searchable data—a kind of Databricks-for-video—then power applications on top of that data lake.

Under the hood:

• Ultra-efficient vision encoders (specialised embedding/caption/indexing models) that can even run on-device.
   
• End-to-end system engineering—compression, indexing, distributed storage, retrieval, re-ranking, and reasoning—to operate at tens of millions of videos.
   
• Data network effects—the more video we index, the better the models and downstream insights.
   

We also support enterprises with internal video (media archives, security footage, product videos), and we’re collaborating with mobile and wearable partners to bring memory into everyday devices. Our models are tiny, accurate, and cost-efficient—which is why consumer electronics leaders are working with us.

Contribution you’re most proud of—and why

My proudest contribution is launching what we believe is the world’s first Large Visual Memory Model (LVMM): a system that converts video into structured, searchable memory with temporal grounding and entity linking. By combining on-device encoders, distributed indexing, and our memory communication protocol, L-VMM enables AI to remember and reason over what it sees—transforming weeks of manual video work into minutes of answers.

What aspect of your work gives you the most satisfaction?

What gives me the most satisfaction is the chance to work with some of the best people in the world—researchers, engineers, and entrepreneurs who share the ambition of shaping the future of AI. Every day we push the boundaries of what’s technically possible, from efficient visual encoders to large-scale memory systems. Being surrounded by people who challenge me, inspire me, and make our collective vision stronger is what makes this journey exciting. It’s the combination of brilliant minds and bold technology that motivates me most.

The most unique thing about your work

The most unique thing about my work is that it’s never static. I iterate every single day—whether on models, infrastructure, or product use cases—and with each cycle comes something new to learn. Working at the frontier of AI memory means the challenges aren’t solved yet, so progress depends on constant experimentation, discovery, and refinement. That pace of daily learning is both demanding and incredibly rewarding.

What impact do you hope your work will have on the world?

To make AI contextual, proactive, and trustworthy. If AI can see, remember, and reason about your world, it becomes a true partner—helping people and teams work faster and make better decisions.

A pivotal moment in your career

A pivotal moment in my career came when I joined Meta’s Reality Labs after completing my PhD. There, I saw firsthand where the largest technology companies were steering the future of AI—toward multimodal systems that could see, hear, and understand the world. That experience convinced me that while the big labs would focus on intelligence, there was an equally vital space—memory—that no one was addressing at scale. Realising this gap inspired me to leave Meta and build the groundbreaking technology for visual memory.

The next big thing in your field

Multimodal AI with built-in memory. Assistants will move from reactive chatbots to agents that understand long-term context, anticipate needs, and operate across software and the physical world.

Advice for someone considering a career in engineering

Stay curious, ship often, and chase hard problems. Learn the theory—but also build. The fastest feedback loops (and best careers) come from iterating with real users.