Latent Space: Inside the Model Factory with Eiso Kant, Poolside AI

⬅️ Back to Podcasts

Latent Space: Inside the Model Factory with Eiso Kant, Poolside AI

swyx and Vibhu with Eiso Kant. Duration: 1h 54m

Listen on Apple · Episode page

Timestamps 0:00 Intro 0:54 Karpathy, RNNs, and building code models before Transformers 2:26 The $12M failure and ChatGPT vindication 3:39 Open source and the case for 100 foundation model companies 16:04 The Model Factory (90% engineering) 20:19 Agents inside the model factory 36:07 Laguna S: persistence vs. raw intelligence 58:37 Model harnesses, coding agents, and the path to AGI 1:09:26 Why MCP and traditional tool calls are “stupid” 1:27:37 Open models, AI safety, and the risk of an oligopoly

Eiso Kant spent four years and $12 million building language models for code before the world cared. His first company failed. Then ChatGPT happened, and suddenly everyone wanted what he’d been working on since 2015. This is the story of what he built next: Poolside’s Model Factory.

  1. Eiso’s origin story starts with Andrej Karpathy’s 2015 blog post “The Unreasonable Effectiveness of Recurrent Neural Networks.” He read it and pivoted his startup overnight to work on RNNs for code. “I had built a completely unreasonable belief, that neural nets should be able to generalize to anything and everything.” He spent four years and $12 million on this before the company failed in 2019. “People would just laugh at us.”

  2. ChatGPT felt like vindication. People started texting him old decks and talks from his failed startup. When he co-founded Poolside, “no one argued if these were stochastic parrots anymore.” The market had caught up to his thesis, which meant he could finally build what he’d always believed in without having to convince anyone it mattered.

  3. “Model building is 90% engineering.” Poolside treats training as an industrialized process, not a science experiment. They built what they call the Model Factory with thousands of components that turn raw data into trained models, designed from day one by distributed systems engineers embedded in the research team. The metric they optimize for: speed from a researcher’s idea to a trusted experimental result.

  4. Fewer than 70 researchers run 10,000–20,000 experiments per month. Their Laguna XS 2 shipped in five weeks from start of training. Laguna S shipped in eight weeks. The next model started training the day after Laguna S launched. “The model should be an artifact of someone’s process it shouldn’t be a thing in itself.” They treat model building like a SpaceX factory where rockets roll off a production line.

  5. Agents are starting to run the Model Factory. Eiso walks behind researchers’ screens and sees agents writing code, launching jobs, evaluating results, and modifying pipelines. “You’re starting to see these twinklings of what RSI is gonna look like.” During the entire Laguna S training run, there were zero on-call events. The only hiccup was the first six hours of a new run when a config inevitably breaks.

  6. Eiso thinks MCP and traditional tool calls are “stupid.” His argument: future agents will write scripts instead of choosing from dozens of predefined tools. Rather than giving an agent a catalog of API functions, give it a container and let it write whatever code it needs. Minimal harness, maximum freedom.

  7. He would rather live in a world with 100 foundation model companies than 5, even if Poolside were one of the five. This is why they open-sourced Laguna S. He actively wants capable researchers to leave their labs and become his competitors. “If we don’t encourage more labs now, there’s a small window before models are really impacting recursive self-improvement to a level where catching up becomes unfeasible.”

  8. Laguna S 2.1 is a 118B total parameter MoE with 8B active per token, beating models nearly 10x its size. It has a 1M token context window, thinking and no-thinking modes, and was trained from scratch in eight weeks. Poolside trains from scratch instead of distilling larger models because they believe the capability ceiling is higher when you own the whole training process.

  9. Poolside raised $500 million while investors still questioned whether AGI was real. The name comes from the idea of “refusing to lower your ambitions”, the story goes that when faced with a impossible-looking challenge, you don’t scale down your goal, you scale up your approach. Eiso is a “utopian sci-fi guy” who believes intelligence will become the world’s most demanded and most commoditized resource.

  10. Regulation could accidentally lock in an oligopoly of two or three AI companies, and unilateral AI safety doesn’t work in a globally competitive environment. Eiso thinks open models will eventually become too capable to release without restrictions, but we’re not there yet and the window to build a diverse ecosystem is closing fast.

Related TMFNK Content

Crepi il lupo!