LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints

The authors propose DeCRIM (Decompose, Critique, and Refine), a novel self-correction pipeline designed to enhance Large Language Models (LLMs) in following instructions with multiple constraints. Recognizing the limitations of LLMs in handling such instructions, especially when real-world user constraints are involved, the authors introduce RealInstruct, a benchmark based on real user queries. Through their analysis, they reveal that even state-of-the-art models like GPT-4 struggle with constraint satisfaction, failing to meet at least one constraint in over 21% of cases. DeCRIM improves LLM performance by decomposing instructions into individual constraints and using a Critic model to refine the output where necessary.

To assess the efficacy of DeCRIM, the authors conduct experiments comparing both proprietary and open-source models. Their findings show that DeCRIM significantly enhances the performance of the open-source Mistral model, leading to a 7.3% improvement on the RealInstruct benchmark and 8.0% on the IFEval benchmark. These results hold even with weak feedback, and stronger feedback allows DeCRIM-enhanced open-source models to outperform GPT-4 on both benchmarks. The authors also explore model-based evaluation as a cost-effective alternative to human evaluation, finding that GPT-4-Turbo with Chain-of-Thought prompting provides reliable results.

In conclusion, the authors contribute to the field by introducing RealInstruct as a new real-world benchmark and DeCRIM as an effective self-correction pipeline for multi-constrained instruction following. They also present the first systematic analysis of model-based evaluation for constraint satisfaction. The proposed pipeline closes the performance gap between open-source and proprietary models, pushing the capabilities of LLMs in handling real-world, complex instructions.