SAKURAONE: Empowering Transparent and Open AI Platforms through Private-Sector HPC Investment in Japan

SAKURAONE is a managed high performance computing (HPC) cluster developed and operated by the SAKURA Internet Research Center. It reinforces the ``KOKARYOKU PHY’’ configuration of bare-metal GPU servers and is designed as a cluster computing resource optimized for advanced workloads, including large language model (LLM) training. In the ISC 2025 edition of the TOP500 list, SAKURAONE was ranked $\mathbf{\text{49th}}$ in the world based on its High Performance Linpack (HPL) score, demonstrating its global competitiveness. In particular, it is the $\mathbf{\text{only}}\mathbf{\text{system}}\mathbf{\text{within}}\mathbf{\text{the}}\mathbf{\text{top}}\mathbf{\text{100}}$ that employs a fully open networking stack based on $\mathbf{\text{800}}\mathbf{\text{GbE}}\mathbf{\text{(Gigabit}}\mathbf{\text{Ethernet)}}$ and the $\mathbf{\text{SONiC}}\mathbf{\text{(Software}}\mathbf{\text{for}}\mathbf{\text{Open}}\mathbf{\text{Networking}}\mathbf{\text{in}}\mathbf{\text{the}}\mathbf{\text{Cloud)}}$ operating system, highlighting the viability of open and vendor-neutral technologies in large-scale HPC infrastructure. SAKURAONE achieved a sustained performance of 33.95~PFLOP/s on the HPL benchmark (Rmax), and 396.295~TFLOP/s on the High Performance Conjugate Gradient (HPCG) benchmark. For the HPL-MxP benchmark, which targets low-precision workloads representative of AI applications, SAKURAONE delivered an impressive 339.86~PFLOP/s using FP8 precision. The system comprises 100 compute nodes, each equipped with eight NVIDIA H100 GPUs. It is supported by an all-flash Lustre storage subsystem with a total physical capacity of 2~petabytes, providing high-throughput and low-latency data access. Internode communication is enabled by a full-bisection bandwidth interconnect based on a Rail-Optimized topology, where the Leaf and Spine layers are interconnected via 800~GbE links. This topology, in combination with RoCEv2 (RDMA over Converged Ethernet version 2), enables high-speed, lossless data transfers and mitigates communication bottlenecks in large-scale parallel workloads.