GreedyViG: Dynamic Axial Graph Construction for Efficient Vision GNNs

Vision graph neural networks (ViG) offer a new avenue for exploration incomputer vision. A major bottleneck in ViGs is the inefficient k-nearestneighbor (KNN) operation used for graph construction. To solve this issue, wepropose a new method for designing ViGs, Dynamic Axial Graph Construction(DAGC), which is more efficient than KNN as it limits the number of consideredgraph connections made within an image. Additionally, we propose a novelCNN-GNN architecture, GreedyViG, which uses DAGC. Extensive experiments showthat GreedyViG beats existing ViG, CNN, and ViT architectures in terms ofaccuracy, GMACs, and parameters on image classification, object detection,instance segmentation, and semantic segmentation tasks. Our smallest model,GreedyViG-S, achieves 81.1Vision GNN and 2.2less GMACs and a similar number of parameters. Our largest model, GreedyViG-Bobtains 83.9decrease in parameters and a 69the same accuracy as ViHGNN with a 67.3decrease in GMACs. Our work shows that hybrid CNN-GNN architectures not onlyprovide a new avenue for designing efficient models, but that they can alsoexceed the performance of current state-of-the-art models.