Agora Enables Collective, Open-Source AI Training

Researchers developed Agora, a system that trains large language models using a network of individual, globally distributed GPUs. The system overcomes limitations of current AI training which requires massive, centralized datacenters with specialized hardware. Agora shards models across internet connections, enabling “Protocol Learning” where no single entity holds the full model weights.

The team demonstrated Agora’s capabilities by training Pluralis-8B, an 8.6 billion parameter model on 500 billion tokens using 330 contributor nodes. These nodes primarily consisted of consumer GPUs operating on standard internet connections, and participants joined and left the network throughout the 40-day training period.

Agora achieved a throughput of approximately 170,000 tokens per second with an efficiency of 63% compared to a centralized H100 system. This suggests a viable path toward decentralized, economically sustainable development of open-source frontier AI models.