AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips

AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips - Amazon Science

Close

Close

Social

bluesky

threads

twitter

instagram

youtube

facebook

linkedin

github

rss

Menu

Research

Research areas

Automated reasoning

Cloud and systems

Computer vision

Conversational AI

Economics

Information and knowledge management

Machine learning

Operations research and optimization

Quantum technologies

Robotics

Search and information retrieval

Security, privacy, and abuse prevention

Sustainability

Our scientific contributions

Publications

Research from our scientists and collaborators.

Conferences

Our experts present and discuss cutting-edge research at scientific meetings globally.

Research areas

Automated reasoning

Cloud and systems

Computer vision

Conversational AI

Economics

Information and knowledge management

Machine learning

Operations research and optimization

Quantum technologies

Robotics

Search and information retrieval

Security, privacy, and abuse prevention

Sustainability

Our scientific contributions

Publications

Research from our scientists and collaborators.

Conferences

Our experts present and discuss cutting-edge research at scientific meetings globally.

News & blog

The latest from Amazon researchers

Amazon Science Blog

Technical deep-dives and perspectives from our scientists.

News

Research milestones and recent achievements.

The latest from Amazon researchers

Amazon Science Blog

Technical deep-dives and perspectives from our scientists.

News

Research milestones and recent achievements.

Collaborations

Amazon Research Awards

Overview

Call for proposals

Latest news

Research stories

Recipients

Amazon Nova AI Challenge

Overview

Rules

FAQs

Teams

Research collaborations

Overview

Carnegie Mellon University

Columbia University

Hampton University

Howard University

IIT Bombay

Johns Hopkins University

Max Planck Society

MIT

Tennessee State University

University of California, Los Angeles

University of Illinois Urbana-Champaign

University of Southern California

University of Texas at Austin

Virginia Tech

University of Washington

Amazon Research Awards

Overview

Call for proposals

Latest news

Research stories

Recipients

Amazon Nova AI Challenge

Overview

Rules

FAQs

Teams

Research collaborations

Overview

Carnegie Mellon University

Columbia University

Hampton University

Howard University

IIT Bombay

Johns Hopkins University

Max Planck Society

MIT

Tennessee State University

University of California, Los Angeles

University of Illinois Urbana-Champaign

University of Southern California

University of Texas at Austin

Virginia Tech

University of Washington

Resources

Code and datasets

Amazon Nova

Try Amazon’s frontier foundation models.

Code and datasets

Amazon Nova

Try Amazon’s frontier foundation models.

Careers

Careers

Explore our open roles.

Amazon Scholars

Faculty research opportunities on industry-scale technical challenges.

Postdoctoral Science Program

Early-career research opportunities alongside experienced industry scientists.

Careers

Explore our open roles.

Amazon Scholars

Faculty research opportunities on industry-scale technical challenges.

Postdoctoral Science Program

Early-career research opportunities alongside experienced industry scientists.

Search

Submit Search

Machine learning

AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips

A competition with a finalist ceremony during NeurIPS 2026, challenging researchers to train language models from scratch on Trainium, exploring what optimal architectures look like when the hardware changes.

By Louise Ping , John Gray , Emily Webber , Josh Longenecker

August 10, 2026

7 min read

Share

Share

Copy link

Email

X

LinkedIn

Facebook

Line

Reddit

QZone

Sina Weibo

WeChat

WhatsApp

分享到微信

x

Key takeaways

AWS Trainium Frontier is a competition challenging researchers to train language models from scratch on purpose-built AI chips, exploring how optimal model architectures differ when hardware constraints change fundamentally from conventional accelerators. The competition features two phases: Phase 1 (30 minutes on a single Trainium2 chip optimizing validation bits-per-byte) and Phase 2 (4 hours on a full server for top 10 teams, adding inference performance scoring across in-context learning tasks). Winners gain recognition at NeurIPS 2026 in Sydney, co-publication opportunities with Annapurna Labs researchers, and prize money ($25,000 first place, $10,000 second, $5,000 third), with top 10 teams receiving exclusive swag and jackets.

Was this answer helpful?

Modern LLM architectures have co-evolved within a single hardware family. The shapes of our attention mechanisms, the structure of our multilayer perceptrons (MLPs), the choice of numerical formats, and even the granularity of parallelism strategies have all been shaped by hardware constraints: warp sizes, tensor core geometries, memory hierarchies, and the kernel abstractions those chips expose. When the hardware changes, the efficient frontier of model architectures changes with it. Here we present an opportunity for academic and industry labs to explore this frontier in detail on AWS Trainium. Purpose-built accelerators like AWS Trainium present a genuinely different design surface. More on-chip SRAM (SBUF) , explicit software control over data movement and acceleration at the lowest levels, energy-efficient systolic matrix multiplication (matmuls), and a memory hierarchy designed for training and inference-scale data flows. The resulting TFLOPs-to-memory-bandwidth ratio shifts the performance bottleneck profile: key operations that are memory-bound on conventional accelerators may become compute-bound on Trainium, opening design space for architectures that trade additional computation for reduced memory traffic. These hardware differences mean the optimal attention patterns, MLP structures, and parallelism strategies may be fundamentally different. The research question is open: What does an optimal model look like when the hardware constraints are fundamentally different? The AWS Trainium Frontier is a competition designed to answer this question empirically: participants train language models from scratch on Trainium, exploring the full design space from model architecture to custom kernels. The core task is training a language model from scratch, starting from a...