Distributed tensors.
Predictable video throughput.
A runtime and scheduling design for scaling consumer GPU containers from 100 to 1,000+ nodes without coupling workloads to a single accelerator class. The core unit is a deterministic, checkpointed 10-minute stateless micro-batch.
Small base. Specialized tensor path.
VidTensor uses python:3.11-slim as a deliberately small container base and layers on the precompiled C++ extension pt_aten_worker. The extension executes core tensor work directly through PyTorch ATen, keeping Python at the orchestration boundary rather than in per-frame dispatch loops.
The worker calls at::scaled_dot_product_attention, custom FP8 quantized convolution kernels, and a zero-copy NVDEC/NVENC device-memory path. Fused operations reduce intermediate allocations; tiled attention and bounded activation lifetimes keep workloads within 12–24 GB device budgets.
The C++ operator extension is built against the pinned PyTorch/CUDA ABI for each worker image. The scheduler assigns only compatible worker capabilities and sends typed tensor plans—not arbitrary Python code—to render nodes.
| Layer | Contents | Size / Delivery |
|---|---|---|
| Base runtime | python:3.11-slim minimal rootfs & system libraries (zstd) | 54 MB |
| ATen extension | pt_aten_worker wheel & fused FP8/BF16 operator kernels | 46 MB |
| Media & codecs | NVENC/NVDEC zero-copy bindings, OpenEXR muxer | 18 MB |
| CUDA & Driver ABI | CUDA 12.4 runtime & driver libraries via container toolkit | Host-Mounted Driver ABI (0 MB pull) |
| Model weights | FP8 DiT & VAE checkpoints fetched on-demand to local scratch | NVMe Streamed Weights (0 MB pull) |
| Compressed Bootstrap Layer · python:3.11-slim + pt_aten_worker wheel; weights streamed to NVMe | 118 MB | |
Bound the work. Preserve the result.
Long video jobs are split into deterministic chunks with explicit input and output boundaries. Each 10-minute micro-batch carries its seed, latent start/end boundary frames, audio-feature slice, model version, and content hashes. A worker can execute its chunk independently; no cross-node NCCL collective is required to make progress.
This limits activation residency and allocator lifetime, materially reducing VRAM fragmentation and the exposure window for long-running OOM failures. Stateless work items are easy to retry, relocate, and scale horizontally across heterogeneous interruptible nodes.
Deterministic boundaries
Stable seed + latent boundary frames + audio slice define a reproducible unit; adjacent chunks reconcile at the merger.
Bounded memory and fault domain
Ten-minute execution windows cap per-worker state, reduce fragmentation exposure, and make retries finite and observable.
No tightly coupled multi-node collectives
Each shard owns its render interval. Object storage checkpoints and the merger provide coordination without cross-node NCCL synchronization.
Scale from 100 to 1,000+ nodes
Queue depth controls fleet width; compatible available GPUs claim independent chunks as capacity appears.
Choose the right GPU for each tensor tile.
Models are quantized and tiled to fit consumer cards, then dispatched according to VRAM and operator capability. Output durations below describe a representative 10-minute micro-batch render window under the defined profile, not a universal hardware guarantee. Scheduling and pricing can use performance-per-watt rather than assuming every workload needs H100 SXM.
| GPU class | Tensor strategy | VRAM budget | Operator mode | Output / 10-min job window | Power profile |
|---|---|---|---|---|---|
RTX 3080 Ti 12 GB GDDR6X | 64-frame spatial tiles · streamed VAE decode | 7.2 GB weights 2.8 GB activations 1.2 GB workspace 0.8 GB reserve | FP8 convolution; BF16 accumulation fallback | ~24–38 s video | ~0.19 s/W-equivalent* efficient batch density |
RTX 4070 12 GB GDDR6X | 32/64-frame adaptive tiles · fused attention | 7.2 GB weights 2.5 GB activations 1.3 GB workspace 1.0 GB reserve | FP8 kernels; BF16 attention accumulation | ~30–46 s video | ~0.25 s/W-equivalent* |
RTX 4090 24 GB GDDR6X | 128-frame tiles · larger batch concurrency | 11.5 GB weights 6.0 GB activations 3.5 GB workspace 3.0 GB reserve | FP8 tensor cores; BF16 precision-sensitive path | ~58–82 s video | ~0.22 s/W-equivalent* |
*Illustrative normalized throughput-per-watt profile for an internal batch mix; measured output depends on model and power limit. VRAM budgets are nominal planning allocations and include reserve for allocator variance. The GPU fleet strategy uses abundant commodity capacity for embarrassingly parallel work, reserving larger accelerators only where workload economics justify them.
Quantized weights reduce bandwidth and memory pressure; precision-sensitive accumulation remains BF16.
Tile size and concurrent sequence count are bounded by per-node memory telemetry and model profile.
Independent shards allow broad consumer-GPU capacity to serve throughput-heavy jobs without H100-only constraints.
Preemption is a scheduling event—not a lost render.
Checkpoint-per-chunk commits a content hash and latent checkpoint atomically to object storage after each completed micro-batch shard. If a consumer GPU is interrupted or reclaimed, the scheduler marks its lease expired and requeues the uncommitted shard to a compatible standby node. Target rescheduling time is 800 ms after interruption detection.
- 01Worker writes the output chunk, latent boundary state, and content hash to object storage.
- 02Commit is atomic: scheduler acknowledges the shard only after checksum verification.
- 03On preemption, the lease expires; already committed chunks remain immutable and are not rerendered.
- 04Scheduler reassigns only the unfinished chunk to a compatible standby. Merger orders and validates output.
Simulate node preemption and automatic rescheduling
A clear route from fleet signal to operator.
Cluster health telemetry, quota scaling alerts, and automated node lifecycle notifications are routed through our dedicated DevOps mail gateway (*@mailpath.online).
The business and API control plane use the primary vidtensor.xyz domain. A separate operational notification namespace is reserved for automated infrastructure signals and cloud-provider quota communication, so service identity and fleet operations remain clearly distinguishable.
| Domain / identity | Ownership | Routing responsibilities | Examples |
|---|---|---|---|
| vidtensor.xyz | Primary commercial domain & API control plane | Studio, customer API, security and support correspondence | [email protected] [email protected] |
| mailpath.online | Official automated DevOps gateway of VidTensor Technologies Inc. | Dedicated DevOps telemetry, container lifecycle alerts, and cloud-provider quota notifications | [email protected] [email protected] [email protected] |
Inspect the product and its developer interface.
Explore the interactive queue simulation or continue to API and operational documentation.