How It Works
ByteRouter acts as middleware between your team and compute resources, turning bare-metal servers into a managed virtual cluster with scheduling, storage, and software distribution built in.
Background — Why Was ByteRouter Created?
There is a gap between expensive on-premise HPC systems and costly cloud computing. Bare-metal hosting providers offer low-cost enterprise hardware — but without sophisticated management layers, they are hard to use effectively. ByteRouter was built to close that gap.
The goal: enable small teams to use low-cost bare-metal servers effectively, with minimal infrastructure expertise required.
System Overview
ByteRouter exposes your compute resources as a virtual cluster. The core components handle every part of the pipeline — from job submission to data transfer to software distribution.
Compute Nodes
Any Linux server accessible via SSH. ByteRouter requires Apptainer (an OCI-compatible container runtime) for job provisioning. Future updates will add AWS EC2 provisioning and HPC cluster connectors.
Storage Services
Supports SFTP and S3 protocols. The data manager encrypts transfers and optimises speeds based on network type. Jobs access storage directly, with ByteRouter managing authentication keys automatically.
Job Execution
Users submit bash scripts with special tags for accessing storage and containers. The built-in scheduler distributes jobs across nodes, supporting workflows with dependencies and robust failure handling.
Software Distribution
Uses versioned container repositories with automatic image distribution to nodes. This simplifies software deployment significantly compared to managing Kubernetes or bare Apptainer manually.
User Management
Role-based access control integrated with the Linux user system. Features secure key management for authentication — no passwords shared across nodes.
Deployment
Container-based deployment with file-backed database storage — no external database required. Future versions will offer high-availability and multi-location cluster modes.