Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings

1. 1Extract a binary hierarchy by agglomerative clustering of class centroids
2. 2Verify the hierarchy using semantic similarity metrics
3. 3Align the hierarchy with a reference hierarchy to improve semantic consistency

This research benefits AI engineers and ML researchers working on vision-language models, as it provides a framework to understand and improve the semantic organization of the embedding space

💡 Understanding the semantic organization of the embedding space is crucial for improving the performance of vision-language models

🔍 New framework to explain & align semantic hierarchies in vision-language model embeddings!