Beyond Functional Correctness: Training-Time and Inference-Time Approaches to Improving Non-Functional Quality of LLM-Generated Code

Abstract

Large Language Models (LLMs) generate functionally correct code that often lacks software standard qualities such as security, readability, and maintainability. This thesis investigates complementary approaches to bridge this gap. The first study examines training-time alignment using DPO and BoNBoN across five LLMs and their instruction-tuned variants. Results show non-functional alignment achieves consistent improvements (10.6% average) while functional alignment proves unreliable (4.9% average), with effectiveness varying by model family and pathway. The second study introduces POSec, an inference-time framework combining automatic prompt optimization with Selective Prompt Anchoring for secure code generation. Evaluated across 8 LLMs, 5 languages, and 3 optimizers, POSec achieves +20–33% security improvements at Pass@10 without model retraining. Together, these studies demonstrate that improving LLM-generated code quality requires multi-level interventions, with training-time alignment offering breadth and inference-time optimization offering targeted depth.