Co-founders behind Reface and Prisma join hands to improve on-device model inference with Mirai | TechCrunch

Much of the conversation around AI today is focused on building cloud capacity and massive data centers to run models. Companies like Apple and Qualcomm are in the early stages of making on-device AI more useful. Amid all that, the 14-person technical team of London-based Mirai is working to improve how models run on phones and laptops.

Mirai, which is backed by a $10 million seed round led by Uncork Capital, was founded by Dima Shvets and Alexey Moiseenkov last year. Both founders have experience in building scalable consumer apps. Shvets co-founded face-swapping app Reface, which was backed by a16z. Shvets later also became a scout for the venture firm. Moiseenkov was CEO and co-founder of the last decade’s viral AI filters app, Prisma.

As consumer developers, both had been thinking about AI and machine learning on devices even before generative AI became popular, Shvets said.

“When we met together in London, we started to chat about technology, and we realized that within the hype of GenAI and more AI adoption, everybody speaks about cloud, about servers, about AGI coming. But the missing piece is on-device [AI] for consumer hardware,” he told TechCrunch.

Shvets and Moiseenkov wanted to use AI to create a pipeline that would allow them to enable complex tasks on the phone, which led them to start Mirai. When they asked others who developed consumer apps, they heard that many wanted better cost optimization and margin per token usage, too.

Today, Mirai is developing a framework for models so they can perform better on devices. The company has built an inference engine for Apple Silicon that optimizes on-device throughput. With its upcoming SDK, developers can integrate the runtime in their apps with only a few lines, the company says.

“One of the visions why we started the company was that we wanted to give developers, like this Stripe-like, eight lines of code [integration] experience…you basically go to our platform, integrate the key, and start working with summarization, classification, or whatever your use case is,” Shvets said.

The startup built this engine in Rust, which can bump up a model’s generation speed by up to 37%, they claim. The company said that, while tuning the model for a platform, it doesn’t tinker with model weights to ensure there is no loss in quality of the output.

Mirai’s stack currently focuses on improving text and voice modalities on the platform, with plans to support vision in the future. The team has started to work with frontier model providers to tune their models for edge use and is in talks with different chipmakers. Later, it plans to bring its engine to Android, too.

In addition, Mirai aims to release on-device benchmarks so model makers can test on-device performance. Shvets recognizes that not all AI work can be done on-device, though. To enable a mixed mode of operation, the team is building an orchestration layer to send requests that can’t be fulfilled on the device up to the cloud.

While the startup is not directly working with apps just yet, its engine could power on-device assistants, transcribers, translators, and chat apps, we’re told.

Andy McLoughlin, managing partner at Uncork Capital, noted that he invested in an edge machine learning company in the last decade. He said that the company was early and eventually sold its business to Spotify. In today’s world, the situation is different, he thinks.

“Given the cost of cloud inference, something has to change… For now, VCs are happy to continue funding the rocket ship companies, spending inordinate sums on cloud inference. But that won’t last  —  at some point, people will focus on the underlying economics of these businesses and realize that something has to change,” he said. “It feels like every model maker will wantto run partof their inference workloads at the edge, and Miraifeels very well-positioned to capture this demand.”

Mirai’s seed round also saw participation from individuals, including Dreamer CEO David Singleton, YC Partner Francois Chaubard, Snowflake co-founder Marcin Żukowski, ElevenLabs co-founder Mati Staniszewski, former Google AdSense product manager and Coinbase board member Gokul Rajaram, Groq investor Scooter Braun, Turing.com CTO Vijay Krishnan, Theory Forge Ventures’ Ben Parr and Matt Schlicht, and ex-Netflix technical leader Aditya Jami.