Late Breaking Results on End-To-End Generation of Factorized Scene Graphs

Scene Graphs (SG) model the geometric-semantic information of the environment of the robot enabling it for any downstream task. However, SG generation has been limited to classifying the edge type between observable objects or ad-hoc algorithms to generate one specific type of semantic entity [1]. We overcome this with a GNN-based diffusion model which generates the SC independently of node type (G-GNN). At each denoising step, one node, its edges and tis node/edge features are generated based on GraphARM[2]. Furthermore, our work S-Graphs+ [1] includes a factor on every edge, tightly coupling the optimization of the SG with the SLAM graph. These factors are manually defined by different functions depending on the related node types. We present a novel factor definition based on GNN common for every edge (F-GNN). A unique architecture encodes the geometrical relationship between the entities of every generated edge with a different model for each combination of connected node types. G-GNN and F-FNN are trained on our synthetic dataset containing planes, walls, rooms and floors and tested in simulated and real scenarios.