DeltaLogic: Minimal Premise Edits Reveal Belief-Revision Failures in Logical Reasoning Models

1. 1Convert natural-language reasoning examples into short revision episodes using DeltaLogic protocol
2. 2Evaluate model's initial conclusion under premises
3. 3Apply minimal premise edits and assess model's revised conclusion
4. 4Analyze model's performance on belief revision under minimal evidence change

AI engineers and ML researchers can benefit from DeltaLogic to improve their models' logical reasoning capabilities, while data scientists can use it to evaluate model performance

💡 DeltaLogic reveals belief-revision failures in logical reasoning models, highlighting the need for improved dynamic reasoning capabilities

🤖 DeltaLogic: a new benchmark to evaluate logical reasoning models' ability to revise beliefs under minimal premise edits