Energy-Efficient Foundation Model Deployment Through Adaptive Quantization, Model Compression, and Hardware-Aware Optimization

Abstract

Foundation models impose substantial inference costs because model weights, activation tensors, key–value caches and runtime orchestration compete for finite compute and memory resources. This study develops an integrated deployment framework combining layer-sensitive adaptive quantization, structured compression and hardware-aware execution selection. A constrained optimization formulation jointly considers energy per generated token, latency, memory capacity and task-quality retention. The proposed pipeline profiles representative workloads, estimates layer sensitivity, allocates precision budgets, evaluates pruning and low-rank candidates, and selects kernels under hardware-specific constraints. An analytical illustrative case study compares full precision, uniform low precision and coordinated adaptive deployment. In the specified scenario, energy per 1,000 tokens falls from 150 to 78 joules, while relative quality retention remains 98.6%; these numbers are constructed for methodological illustration and are not measured experimental findings. The framework emphasizes reproducible energy measurement, calibration representativeness, Pareto selection and reporting of compression overhead. Its contribution is a transparent decision procedure and evaluation protocol for sustainable foundation-model serving rather than a claim of empirically validated superiority.