Enforcing Batch Invariance in Agentic Kernel Generation
ORIGINAL / God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation
This study identifies that dynamic batching in LLM serving can alter numerical outputs for identical requests when kernels lack batch invariance (BI), a property overlooked by existing kernel-generation agents. The proposed tool combines contract-guided static dependency analysis with dynamic bitwise-consistency validation as a plug-in for general kernel agents, improving BI pass rates while retaining most of the baseline speedup.
01 ABSTRACT
The paper reports 19 confirmed BI violations among 83 batch-applicable kernel artifacts, meaning identical inputs yield different numerical outputs across batch contexts. The authors propose a plug-in tool integrating static dependency analysis and dynamic bitwise-consistency validation as acceptance gates in the kernel optimization loop. Experiments across three agent frameworks on real-world LLM serving tasks raise the BI pass rate from 67.6% to 91.9% while retaining 91.8% of the baseline geometric mean speedup.
02 KEY FINDINGS
- 19 BI violations were confirmed among 83 batch-applicable released artifacts, indicating the issue is not rare in existing generated kernels.
- The tool uses contract-guided static dependency analysis and dynamic bitwise-consistency validation, designed as a plug-in for general kernel agents to reuse their original optimization capabilities.
- Across three agent frameworks on real-world LLM serving tasks, the BI pass rate improves from 67.6% to 91.9% while retaining 91.8% of the baseline geometric mean speedup.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG