Learning representations on graphs with deep convolutional neural networks

Abstract

Graph representation learning algorithms embed vector representations of nodes into a low-dimensional space, where the proximity of objects in the original network is preserved. The node embeddings can then facilitate downstream tasks, such as classifi- cation, clustering and link prediction. In this thesis, we discuss three challenging problems in graph representation learning, i.e., (i) learning embeddings of both node attribute and network on graphs, (ii) embedding HIN objects into a low-dimensional Euclidean space, and (iii) object classification on HIN given a scarce set of labelled objects. First, learning embeddings of both node attribute and network on graphs is studied. We propose a deep generated model HOANE, for attributed networks. Embeddings learnt by HOANE caputures affinity between nodes and attributes, and are prepared to be directly applied in downstream tasks. Based on variational graph auto-encoder, HOANE employs a hierarchical variational framework to capture more complex posteriors than Gaussian. Moreover, HOANE jointly reconstructs links and attributes in the generative model, which drives learned embeddings to well preserve the information of graph structure and node attributes. Next, we study embedding HIN objects into a low-dimensional Euclidean space. A Heterogeneous information Network (HIN) is one whose objects are of different types and whose edges represent different types of relations between objects. Different from a homogeneous network, where objects (and edges) are all of one single type, HINs are more expressive in capturing the rich semantics of objects and their relationships in real-world applications. We propose CLING, an algorithm which extends graph convolutional networks to HINs. We encode the rich information of HINs in both nodes and edges by using meta-path, a model that captures the relationship between objects by a path of object types which connect them. We leverage both node-based and edge-based information to update node embeddings in a convolutional layer. We also employ the attention mechanism. Specifically, node-level attention is used to learn the importance of an object’s neighbors, while meta-path-level attention is used to learn the importance of meta-paths. We show that CLING is a generalization of many existing methods. Finally, we study how to learn representations when labelled objects in HINs are scarce. We propose a graph neural network model named ConCH. ConCH formulates the classification problem as a multi-task learning problem that combines semi-supervised learning with self-supervised learning to learn from both labeled and unlabeled data. ConCH employs meta-paths, which are sequences of object types that capture semantic relationships between objects. Based on meta-paths, it considers two sources of information for an object x: (1) Meta-path-based neighbors of x are retrieved and ranked, and the top-k neighbors are retained. (2) The meta-path instances of x to its selected neighbors are used to derive meta-path-based contexts. ConCH co-derive object embeddings and context embeddings via graph convolution. It also uses the attention mechanism to fuse such embeddings.