Senior ML Systems Engineer, Frameworks & Tooling
Role overview
Cohere is a security-first enterprise AI company building frontier foundation models and end-to-end products for real-world business applications. We are seeking a senior engineer to design, build, and maintain the training framework that powers our large-scale language models. This position sits at the intersection of distributed systems, high-performance computing, and ML infrastructure, offering the opportunity to directly impact the capabilities of models deployed across enterprises globally.
Key responsibilities
- Build and own the training framework responsible for large-scale language model training across multi-node GPU clusters.
- Design distributed training abstractions including data parallelism, tensor parallelism, pipeline parallelism, and memory management strategies for frontier hardware.
- Improve training throughput and stability across multi-node clusters running various GPU architectures.
- Develop and maintain monitoring, logging, debugging, and tooling infrastructure to increase developer productivity for ML teams.
- Collaborate with infrastructure teams to ensure cluster configurations, container environments, and hardware setups support high-performance training.
- Investigate and resolve performance bottlenecks across the complete ML systems stack.
- Build robust systems that enable reproducible, debuggable, and fault-tolerant large-scale training runs with reliable checkpointing.
Requirements
- Strong engineering experience with large-scale distributed training systems or high-performance computing infrastructure.
- Deep familiarity with distributed training libraries, JAX internals, custom kernels, or fused operations.
- Demonstrated experience with multi-node cluster orchestration tools such as Slurm, Ray, or Kubernetes.
- Proficiency debugging performance issues across CUDA, NCCL, networking, input/output, and data pipelines.
- Hands-on experience with containerized environments including Docker and similar technologies.
- Track record of building tools that measurably improve developer velocity for machine learning teams.
- Strong judgment in evaluating trade-offs between performance and complexity, and between research velocity and maintainability, combined with excellent cross-team collaboration skills.
What makes this role worth considering
You will work on infrastructure powering frontier-scale models at a company with global presence across Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul. The role offers genuine autonomy and full-stack ownership of critical systems that directly connect research innovation to thousands of GPUs, with the opportunity to tackle challenging problems in distributed systems and high-performance computing at scale.
How to apply
Use the Apply button to view the official job posting and submit your application on Cohere's careers site.
ImmigrationCafe is a job listing directory, not a recruiter. Applications are submitted on the employer's official careers website.