Role overview

Cohere is building the next generation of infrastructure to power autonomous AI agents at scale. As a Staff or Senior Staff Software Engineer on the Agent Infrastructure team, you’ll work at the intersection of machine learning systems, distributed infrastructure, and developer experience to create the platform that enables enterprises to deploy AI agents reliably and securely.

Key responsibilities

Design and build secure execution environments where agents can safely run code generated during their operations.
Develop identity, authentication, and trust boundary systems that verify and authorize agent actions across the platform.
Create model routing and orchestration capabilities that intelligently direct requests across different model types and deployment environments.
Implement rate limiting, quota systems, and resource management to ensure agent workflows run reliably at scale.
Build state management, memory, and filesystem abstraction layers that allow agents to maintain context and access data across long-running operations.
Transform emerging research ideas from the ML team into production-ready infrastructure components.
Partner with research teams to understand future agent system requirements and shape platform design accordingly.

Requirements

Proven experience building production machine learning infrastructure with strong systems-level fundamentals.
Hands-on background working with agentic systems, multi-agent workflows, or agent development frameworks.
Familiarity with model routing approaches and LLM provider frameworks across different model types and environments.
Experience designing and operating scalable, fault-tolerant distributed systems, including work with Kubernetes.
Demonstrated ability to rapidly prototype solutions and make sound decisions about which technologies and approaches should move into production.

What makes this role worth considering

You’ll be working on emerging infrastructure challenges with few established patterns in the field, combining systems depth with understanding of how new agentic capabilities reshape platform design. Cohere is a global technology company headquartered in Toronto with offices across London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul, offering a remote-friendly work environment with flexibility on where and how you work.