AmI · Lecture 15 — Graph Neural Networks

Graphs & Machine Learning on Graphs

A graph G = (V, E) is a set of nodes (vertices) V connected by edges E. Graphs are a natural way to represent entities and the relationships between them.

Kinds of graphs

Undirected — edges have no direction; directed — edges go from one node to another; weighted — each edge carries a weight.
Adjacency matrix A ∈ ℝ^|V|×|V| encodes the edges: A[u][v] tells whether (and how strongly) node u connects to node v. If the graph is undirected, A is symmetric.
Multi-relational graphs: edges can be of different relation types τ, so the adjacency becomes A ∈ ℝ^{|V|×|R|×|V|} (one slice per relation).
Heterogeneous graphs: nodes themselves have types, and V is partitioned into disjoint sets of typed nodes.

Node attributes

Nodes can carry features: a feature matrix X ∈ ℝ^|V|×m associates an m-dimensional attribute vector with each of the |V| nodes.

Machine-learning tasks on graphs

🔵

Node classification

Predict a label for each node.

🔗

Link prediction

Predict whether an edge exists between two nodes.

🟣

Graph classification / regression

Predict a label or value for a whole graph.

🧩

Graph clustering

Group nodes (or graphs) by similarity.

The Permutation Challenge & Graph Isomorphism

Why not just flatten the adjacency matrix into an MLP?

A tempting idea is to flatten the adjacency matrix into a vector and feed it to a standard MLP. This does not work: the ordering of the nodes is arbitrary, so the same graph has many different flattened matrices. An MLP is not permutation-invariant — it would treat each ordering as a different input.

Graph isomorphism — the core intuition

Two isomorphic graphs (the same graph drawn with the nodes relabeled) can have different adjacency matrices. We want a model whose representation does not depend on node ordering — so isomorphic graphs are treated the same way.

The goal of a GNN is therefore a permutation-independent representation: the nodes are treated as a set, not an ordered list. Whatever order we feed them in, the learned embeddings should be the same.

Message Passing, Self-Loops & GCNs

GNNs learn a per-node embedding through message passing: at each iteration, every node combines information coming from its neighbors and uses it to update its own state.

The two operations

AGGREGATE — combines the information coming from the neighbors N(u) of the target node.
UPDATE — updates the representation of the target node using the aggregated message and its own previous state.

h_u^(k+1) = UPDATE( h_u^(k) , AGGREGATE({ h_v^(k) , ∀ v ∈ N(u) }) )

Intuition: the initial embedding is the node feature, h_u⁽⁰⁾ = x_u. After K iterations, each node's embedding contains information from its K-hop neighborhood — analogous to how stacking convolutions in a CNN grows the receptive field.

Naive message passing

h_u^(k) = σ( W_self · h_u + W_neigh · Σ_v∈N(u) h_v + b )

Separate weight matrices transform the node's own state (W_self) and the summed neighbor states (W_neigh).

Message passing with self-loops

Instead of treating the node separately, we can aggregate over the neighborhood plus the node itself, N(u) ∪ {u}:

Pros

Simplifies the model and removes the UPDATE step (the node is already part of the aggregation)
Often alleviates overfitting

Cons

Limits expressivity: the model can no longer differentiate the neighbors' information from the node's own information

Exam trap — sim Q19 lives here

Self-loops remove the UPDATE step (not the AGGREGATE step) and mitigate overfitting. Swapping "UPDATE" for "AGGREGATE" is the classic one-word-swap distractor.

Normalization & GCNs

Normalization stabilizes aggregation: mean normalization divides by |N(u)|; symmetric normalization divides each neighbor by √(|N(u)|·|N(v)|).
A Graph Convolutional Network (GCN) combines self-loops with symmetric normalization — and that operation is exactly what the deck calls "Convolution!" on a graph.

Pooling & GNN Tasks Revisited

Aggregation needs a function that maps a set of neighbor embeddings to a single vector. Two families:

Set Pooling

A permutation-invariant universal set-function approximator: m_N(u) = MLP_θ( Σ MLP_φ(h_v) )
Sum makes it order-independent

Janossy Pooling

Permutation-sensitive (e.g., an LSTM over the neighbors)
Made order-independent by averaging over sampled permutations

Graph-level embedding (graph pooling)

To get a single embedding for the whole graph, pool the node embeddings: z_G = Σ_u z_u / f_n(|V|), optionally with an attention mechanism to weight nodes differently.

Tasks revisited with GNNs

Task	How a GNN does it
Node classification	Loss L = Σ_{u∈V_train} −log( softmax(z_u, y_u) ); inductive/test nodes are ignored during training.
Link prediction	Cast as metric learning on the node embeddings (do two nodes belong together?).
Graph classification	Use the graph-level embedding z_G as a single data point and classify it.

ThinkWhy is a plain sum a valid AGGREGATE function, while feeding the neighbors to an LSTM is problematic — and how does Janossy pooling rescue the LSTM?

Answer: A sum (as in Set Pooling) is permutation-invariant — reordering the neighbors gives the same result, which is exactly what a GNN needs since neighbors form a set. An LSTM processes its inputs in order, so it is permutation-sensitive: a different neighbor ordering yields a different embedding, breaking the set semantics. Janossy pooling rescues it by averaging the LSTM's output over many sampled permutations of the neighbors, recovering (approximate) order-independence while keeping the LSTM's expressive power.

GNNs in Ambient Intelligence

Why GNNs for sensor data?

CNNs and RNNs can be viewed as GNNs: CNNs work on fixed-size grid graphs, RNNs on line graphs.
Typical GNNs handle complex graphs with no fixed form, a variable number of unordered nodes, and a variable number of neighbors per node.
Advantage: they effectively encode complex relationships and interdependencies among devices in IoT sensing systems, and among data instances over time.

Constructing a graph from a multivariate time series

Each node represents a sensor (or a device with several sensors); its features are a time window of sensor data. How to set the edges?

Heuristic-based graphs

Edges from heuristics like spatial proximity between sensors or pairwise similarity computed on training data

Learning-based graphs

Edges learned end-to-end (start fully connected); attention derives the most important edges
More costly, but yields richer structures

Body-sensing & spatio-temporal graphs

Body-sensing graph: each node is a device at a body position (feature = a time window of that position's data); edges connect adjacent body parts to capture dependencies between close devices.
Spatio-temporal graphs encode both space and time as a multi-relational graph where layers are time instants. Three edge types: spatial dependency (nodes linked at the same instant), temporal correlation (the same node across instants), and spatial-temporal correlation (different nodes across different instants).

Smart-home graph construction

Heuristic-based (GNN-XAR): a window of sensor events becomes a graph; each node (a sensor event) is linked to another if they are temporally consecutive in the window.
Heuristic + learning (Know Thy Neighbors): start fully connected (each node a sensor), compute pairwise similarity from sensor behavior (activation frequency, periodicity, value range), keep only the highest-similarity edges, then weight them with attention (higher weight for sensors often triggered together).

Concrete GNN architectures & applications

⏱️

GraphConvLSTM

Graph convolution captures spatial properties; temporal convolution (1D-CNN per node) captures temporal properties; a final LSTM captures long-term dependencies before classification.

⚡

Load disaggregation

Cluster the aggregate smart-meter signal (each cluster = a node); edges weighted by transition probability (Markov chain); GCN message passing + graph pooling → decoder maps to per-appliance consumption.

📍

Indoor localization

Each node = an Access Point (feature = measured RSSI); weighted edge if two APs are spatially close; after message passing, graph pooling identifies the subject's position.

Advanced GNN Topics

Two questions connect GNNs back to earlier lectures: can GNNs be used for self-supervised learning? and can we employ XAI methods for GNNs?

Self-supervised learning with GNNs

Predictive approach

Graph reconstruction: an encoder maps the (partially hidden) graph to embeddings, a decoder reconstructs it, and the loss compares reconstruction to the original

Contrastive approach

Graph augmentation produces two views; a GNN-based feature extractor encodes both; positive/negative sampling + a contrastive loss pull matching views together

Explainability for GNNs — a perturbation-based approach

Given an input graph (features X and adjacency A), a mask-generation algorithm produces a feature mask, an edge mask and a node mask. These masks are multiplied into the GNN's input, and an objective function tied to the prediction optimizes the masks — revealing which features, edges and nodes most influenced the output.

✓

Final Quiz — Exam Style

16 MCQs + 2 open-ended. One is the actual simulation question (Q19) reproduced verbatim; the rest follow the professor's recipe — sibling distractors, one-word swaps, and a FALSE question hiding an absolutizer.

When is the adjacency matrix of a graph guaranteed to be symmetric?

AWhen the graph is directed

BWhen the graph is weighted

CWhen the graph is undirected

DWhen the graph is heterogeneous

C — in an undirected graph an edge (u,v) is also (v,u), so A is symmetric. Directed graphs (A) need not be; weighting (B) and node typing (D) don't force symmetry.

What distinguishes a heterogeneous graph from a multi-relational graph?

AIt allows edge weights, whereas multi-relational graphs do not

BIts nodes have types, and the node set is partitioned by type

CIts edges have several relation types stored in a 3-D adjacency tensor

DIt forbids self-loops between a node and itself

B — in a heterogeneous graph the nodes are typed and V is partitioned into typed sets. C describes a multi-relational graph (typed edges, A ∈ ℝ^{|V|×|R|×|V|}). A and D are not the defining property.

Which graph ML task predicts whether an edge exists between two nodes?

ALink prediction

BNode classification

CGraph classification

DGraph clustering

A — link prediction asks whether an edge connects two nodes. Node classification (B) labels nodes, graph classification (C) labels a whole graph, clustering (D) groups by similarity.

Why does flattening the adjacency matrix and feeding it to an MLP fail to learn on graphs?

AThe adjacency matrix is too sparse for an MLP to process

BMLPs cannot handle weighted edges

CThe matrix loses the node feature vectors X

DNode ordering is arbitrary and an MLP is not permutation-invariant

D — the node order is arbitrary, so one graph has many flattened matrices; an MLP treats each ordering as a different input because it is not permutation-invariant. A, B and C are not the core issue.

What does it mean that two graphs are isomorphic, and why does it matter for GNNs?

AThey have the same number of edges but different nodes; GNNs should rank them

BThey are the same graph relabeled; GNNs should treat them the same way

CThey share one identical node; GNNs should merge them

DThey have identical adjacency matrices; GNNs should ignore them

B — isomorphic graphs are the same graph with relabeled nodes; they may have different adjacency matrices, so a GNN should produce a permutation-independent representation and treat them identically. D is false (their matrices differ), A and C are wrong descriptions.

In message passing, what is the role of the AGGREGATE operation?

AIt updates the target node using its own previous state

BIt pools all node embeddings into one graph-level vector

CIt combines the information coming from the node's neighbors

DIt normalizes the adjacency matrix before training

C — AGGREGATE combines the neighbors' information; the subsequent UPDATE step (A) updates the target node. B is graph pooling, D is normalization — different operations.

After K iterations of message passing, what does each node's embedding capture?

AInformation from its K-hop neighborhood

BInformation from only its direct neighbors

CInformation from the entire graph regardless of K

DInformation from K randomly sampled nodes

A — each iteration reaches one hop further, so after K iterations the embedding encodes the K-hop neighborhood (like a CNN's growing receptive field). B is true only for K=1; C and D misstate the mechanism.

Simulation exam · Q19
What is the primary goal of adding self-loops for message passing within GNNs?

AEnsuring permutation-dependent graphs, so that isomorphic graphs are treated differently

BRemoving the need for the AGGREGATE step and mitigating overfitting

CEnsuring permutation-invariant graphs, so that isomorphic graphs are treated the same way

DRemoving the need for the UPDATE step and mitigating overfitting

D — the slide lists, as pros of self-loops, that they remove the UPDATE step (the node is already part of the aggregation) and often alleviate overfitting. B is the trap: a one-word swap of UPDATE → AGGREGATE (you still need to aggregate the neighborhood). A and C are about permutation invariance, which self-loops are not about.

What is the main downside of message passing with self-loops?

AIt makes the model permutation-sensitive

BIt cannot differentiate the neighbors' information from the node's own

CIt increases the risk of overfitting

DIt requires a separate model per relation type

B — by folding the node into its own neighborhood, self-loops limit expressivity: the model can't tell the node's own info from its neighbors'. C is backwards (self-loops alleviate overfitting), A and D are unrelated.

A Graph Convolutional Network (GCN) is obtained by combining which two ingredients?

AJanossy pooling and an LSTM read-out

BAttention weighting and graph augmentation

CSelf-loops and symmetric normalization

DMean normalization and a frozen backbone

C — the deck defines a GCN as self-loops + symmetric normalization, calling that operation "Convolution!". The other pairs mix in concepts from pooling, contrastive SSL or reprogramming.

How do Set Pooling and Janossy Pooling differ?

ASet pooling is permutation-invariant; Janossy is sensitive, fixed by averaging over permutations

BSet pooling uses an LSTM; Janossy uses a sum of MLPs

CSet pooling works only on directed graphs; Janossy only on undirected

DSet pooling needs node types; Janossy needs edge types

A — Set pooling (sum of MLPs) is naturally permutation-invariant; Janossy pooling is permutation-sensitive (e.g., an LSTM) and is made order-independent by averaging over sampled permutations. B swaps the two; C and D are invented.

How does a GNN obtain a single embedding for an entire graph (graph pooling)?

ABy flattening the adjacency matrix into a vector

BBy taking the embedding of a single chosen root node

CBy running message passing for exactly one iteration

DBy pooling the node embeddings (optionally with attention)

D — the graph-level embedding z_G = Σz_u/f_n(|V|) pools all node embeddings, optionally weighted by attention. A is the failed MLP idea, B and C discard most of the graph's information.

Which of the following statements about GNNs is FALSE?

AMessage passing aggregates information from a node's neighbors

BA GCN combines self-loops with symmetric normalization

CIsomorphic graphs always have identical adjacency matrices

DK message-passing steps capture K-hop neighborhood information

C is FALSE — isomorphic graphs can have different adjacency matrices (that's the whole reason GNNs must be permutation-independent); the absolutizer "always identical" gives it away. A, B and D are all true statements from the deck.

When building a graph from a multivariate time series, what is a learning-based edge strategy?

AConnecting sensors that are spatially close on the map

BStarting fully connected and learning edge importance with attention

CUsing pairwise similarity precomputed once on the training data

DLinking nodes that are temporally consecutive in the window

B — learning-based graphs are learned end-to-end (start fully connected, then attention derives the most important edges). A and C are heuristic-based; D is the consecutive-event heuristic used in smart homes.

In GraphConvLSTM, what captures the spatial properties of the graph?

AThe final LSTM layer

BThe temporal convolution (1D-CNN per node)

CThe graph convolution

DThe softmax classification head

C — graph convolution captures spatial properties; the temporal convolution (B) captures temporal properties per node, and the final LSTM (A) captures long-term temporal dependencies before classification (D).

Which describes the predictive (vs contrastive) approach to self-supervised learning with GNNs?

AGraph reconstruction: encode the graph, decode it, and compare to the original

BAugment the graph into two views and pull matching views together

CMask the edges and optimize an objective tied to the prediction

DSample positive and negative pairs for a contrastive loss

A — the predictive approach is graph reconstruction (encoder → decoder → loss vs the original). B and D describe the contrastive approach; C is the perturbation-based explainability method.

Open-ended · From graphs to message passing

a) Define a graph and its variants (undirected/directed/weighted, multi-relational, heterogeneous), and explain why flattening the adjacency matrix into an MLP fails — tie this to permutation invariance and graph isomorphism.

b) Explain message passing (AGGREGATE / UPDATE), the effect of K iterations, what self-loops change (pros & cons), and how a GCN is defined.

Model answer

a) A graph G = (V, E) has nodes and edges; it can be undirected (symmetric adjacency A), directed, or weighted. Multi-relational graphs have typed edges (A ∈ ℝ^{|V|×|R|×|V|}); heterogeneous graphs have typed nodes (V partitioned by type). Nodes may carry features X ∈ ℝ^|V|×m. Flattening A into an MLP fails because node ordering is arbitrary: the same graph yields many flattened matrices, and an MLP is not permutation-invariant, so it treats each ordering as a new input. Isomorphic graphs (the same graph relabeled) may have different adjacency matrices, so we need a permutation-independent representation — nodes as a set — so isomorphic graphs are treated identically.

b) A GNN learns node embeddings via message passing: AGGREGATE combines the neighbors' embeddings and UPDATE refreshes the target node — h_u^(k+1) = UPDATE(h_u^(k), AGGREGATE({h_v^(k): v∈N(u)})), starting from h_u⁽⁰⁾ = x_u. After K iterations each node sees its K-hop neighborhood. Self-loops aggregate over N(u)∪{u}: pros — they simplify the model, remove the UPDATE step, and alleviate overfitting; con — they limit expressivity (can't separate the node's own info from its neighbors'). A GCN = self-loops + symmetric normalization (dividing each neighbor by √(|N(u)|·|N(v)|)), which the deck calls graph "Convolution".

Open-ended · GNNs in Ambient Intelligence & advanced topics

a) Explain how graphs are constructed from sensor data (heuristic vs learning-based, body-sensing, spatio-temporal, smart-home) and describe at least two concrete GNN applications in AmI.

b) How can GNNs be used for self-supervised learning, and how can they be explained?

Model answer

a) CNNs/RNNs are special GNNs (grid graphs / line graphs); general GNNs handle variable-size, unordered nodes and encode device interdependencies. From a multivariate time series, each node is a sensor (feature = a time window); edges are set either heuristically (spatial proximity, pairwise similarity on training data; in smart homes, link temporally consecutive events — GNN-XAR) or by learning (start fully connected, use attention to derive important edges; Know Thy Neighbors computes similarity from sensor behavior, keeps top edges, weights them by attention). Body-sensing graphs place a node at each body position and connect adjacent parts; spatio-temporal graphs add spatial, temporal and spatial-temporal edges across time layers. Applications: GraphConvLSTM (graph conv = spatial, temporal conv = temporal, LSTM = long-term), load disaggregation (cluster the aggregate meter signal into nodes, Markov-chain edge weights, GCN + pooling → per-appliance power), and indoor localization (nodes = APs with RSSI features, edges between close APs, pooling → position).

b) Self-supervised learning: the predictive approach uses graph reconstruction (encoder → decoder → reconstruction loss); the contrastive approach uses graph augmentation into two views, GNN feature extractors, positive/negative sampling and a contrastive loss. Explainability: a perturbation-based method generates a feature mask, edge mask and node mask via a mask-generation algorithm, multiplies them into the GNN input, and optimizes an objective tied to the prediction to reveal which features, edges and nodes drove the output.

0/16