AI Thrust Seminar | Applications of Graph Convolutional Networks (GCN)

Graph convolution networks have been a vital tool for a variety of tasks in computer vision. Beyond predefined array data, graph representation with vertexes and edges capture inherent data structure better. 3D shapes in mesh are probably the best example in graphs, and a set of multi-dimensional data vectors and their relations are often cast as nodes and edges in a graph. I will present two example studies of our own on GCN. In the first work, we focus on deep 3D morphable models that directly apply deep learning on 3D mesh data with a hierarchical structure to capture information at multiple scales. While great efforts have been made to design the convolution operator, how to best aggregate vertex features across hierarchical levels deserves further attention. In contrast to resorting to mesh decimation, we propose an attention based module to learn mapping matrices for better feature aggregation across hierarchical levels. Our proposed module for both mesh downsampling and upsampling achieves state-of-the art results on a variety of 3D shape datasets. In the second work, we propose a novel pool-based Active Learning framework constructed on a sequential Graph Convolution Network (GCN). Each images feature from a pool of data represents a node in the graph and the edges encode their similarities. With a small number of randomly sampled images as seed labelled examples, we learn the parameters of the graph to distinguish labelled vs unlabelled nodes. To this end, we utilise the graph node embeddings and their confidence scores and adapt sampling techniques such as CoreSet and uncertainty-based methods to query the nodes.