Role overview

Cohere is a security-first enterprise AI company that develops foundation models and products for real-world business applications. As a Software Engineer in ML Ops, you will join the Internal Infrastructure team, which builds the systems that enable researchers and ML engineers to train, evaluate, and serve Cohere’s foundation models at scale.

Key responsibilities

Operate and maintain secure developer environments that support research teams while maintaining control and accelerating developer velocity as the organization grows.
Manage container image build pipelines and CI/CD tooling infrastructure.
Instrument training code and develop self-service tools that enable researchers to diagnose run failures and debug issues independently.
Establish infrastructure best practices through AI enablement, comprehensive documentation, and regular office hours with stakeholders.
Respond to production incidents, diagnose performance problems, and implement improvements that make the system easier to operate over time.
Create automation, infrastructure-as-code solutions, and observability systems that keep the platform maintainable and scalable.
Participate in a compensated 24×7 on-call rotation to support critical infrastructure.

Requirements

Production experience running a developer platform, including CI systems, image builds, dev environments, or internal tooling relied upon by multiple engineers.
Demonstrated ability to improve the productivity of ML engineers or researchers without introducing fragility into the platform.
Sufficient knowledge of Linux and Kubernetes to troubleshoot production systems, with experience operating GPU clusters considered advantageous.
Software development proficiency in Python or Go, with a preference for extending solid open-source projects over rebuilding solutions from scratch.
Experience collaborating with researchers, ML engineers, or cross-functional teams whose objectives differed from your own.
Track record of identifying infrastructure bottlenecks, proposing solutions, and driving improvements in dynamic environments.

What makes this role worth considering

You will work directly with AI researchers and ML engineers on systems that directly impact model capabilities and customer value. Cohere offers substantial benefits including six weeks of paid vacation, full health and dental coverage with a dedicated mental health budget, and a $75 weekly lunch stipend. The company supports remote work across global offices in Toronto, London, New York, San Francisco, Montreal, Paris, Berlin, and Seoul, with home office setup support and co-working benefits for those not based near an office.