Static vs Adaptive Models
Every model so far was "static": trained once, with a predetermined set of classes, unable to evolve with new classes or new sensor patterns. But after deployment, temporal data streams continuously evolve.
The four changes in data streams
| Change | Slide definition |
|---|---|
| Concept drift | The target subject may perform activities with some differences (e.g., young vs elderly) |
| Concept evolution | New patterns emerge from data: new activities or new ways of performing an activity |
| Concept forgetting | To improve both accuracy and efficiency, it is crucial to forget patterns if not representative of the target user |
| Outliers / anomalies | New data may present sporadic samples particularly distant from learned patterns — distinguishing outliers from anomalies is crucial |
Why not re-training from scratch?
Periodically re-training as labeled data arrives is not scalable: it requires continuously storing labeled data (original training set + new data) and periodic re-training that is computationally costly and cannot adapt quickly to stream changes.
Why does this matter in Ambient Intelligence?
- Adapting to specific users (concept drift): changes in health conditions, routines/habits, living environments and sensing setups (feature evolution).
- Labeled data scarcity: the HAR model starts from a small unrepresentative dataset; new labeled and unlabeled data arrive incrementally; it's challenging to foresee all activities (→ activity discovery, §6).
- The "lifelong learning" loop: change detection → change annotation → change adaptation.
Catastrophic Forgetting
The stability-plasticity dilemma
Stability
- The capability of the model of retaining old knowledge
Plasticity
- The capability of the model of learning and adapting on new knowledge
The dilemma: how to achieve stability without rigidity, and plasticity without chaos? Goal: a good trade-off between continuously learning and maintaining old information — a challenging open problem in AI.
Continual Learning (CL) studies learning from an infinite stream of data — to adapt models to new domains, learn new classes, personalize models — gradually extending acquired knowledge and using it for future learning. CL approaches mitigate catastrophic forgetting by balancing plasticity and stability.
Desirable ingredients for continuous learning: constant memory, undefined set of classes, incremental learning, few-shot learning, retaining and improving acquired knowledge, learning also from unlabeled data, selective (graceful) forgetting.
Mitigations: Replay & Regularization
Replay methods
Alleviate forgetting using "old" labeled data: maintain a limited subset of stored samples (replay memory) representing old knowledge, provided to the classifier along with new data at each incremental update. Pros: low-cost, easy to implement. Cons: risk of overfitting over stored samples.
| Variant | How the buffer works |
|---|---|
| Random replay | The buffer is populated by randomly sampling previous labeled data points; when learning on new data, buffer data are also used |
| Exemplars replay | The buffer is populated with exemplars: representative labeled samples (e.g., samples close to the feature mean of each class) |
| Constrained-based replay | The last K data samples fill the buffer; learning on new data is constrained by a loss focused on old data: it can decrease but it can not increase |
| Generative replay | When storing samples is unfeasible: the output of previous versions of the model with synthetic input approximates "old" knowledge (e.g., a GAN generating high-quality data). Cons: complexity significantly increased |
Problems of replay-based methods: requires extra storage, the model may overfit on the buffer, buffer data may not be representative.
Regularization-based methods
Avoid storing raw inputs (prioritizing privacy) and alleviate memory requirements: extra regularization terms in the loss function consolidate previous knowledge. Two families:
Data-focused — LwF
- Learning without Forgetting leverages knowledge distillation (Lecture 7b!)
- The previous model (teacher) is maintained: it generates pseudo-labels on incoming data related to old tasks
- A specific loss considers pseudo-labels + classification on new tasks: the student outputs a similar probability distribution on old knowledge and, at the same time, learns the new tasks
Prior-focused — EWC, VCL
- Estimate an importance value for each weight (i.i.d. assumption)
- While learning on new data, a regularization term penalizes significant changes to important weights
- Important weights stay similar after training → knowledge maintained. Famous: Elastic Weight Consolidation, Variational Continual Learning
Semi-Supervised Learning
- Personalization: a model trained collaboratively by volunteers → the target user downloads it locally → semi-supervised learning adapts it to the user.
- Semi-supervised can be applied to static models too (when re-training from scratch is possible); here we focus on the incremental setting — more realistic in AmI.
- The three main strategies: self-training, co-training, active learning (§5).
Self-training
- Label propagation variant: data points close in space likely share the class. Build a graph G=(V,E) — nodes are (un)labeled data points, edges weighted by similarity; labels spread from labeled to unlabeled points based on a threshold on similarity, repeated until convergence.
- Label propagation cons: it must store unlabeled data points — impossible to store them all incrementally, so it runs periodically over a limited buffer of historical data.
- Self-training limits: errors in label spreading may be amplified with time; hard to spread labels to data "different" from the training data.
Co-training
- An ensemble of classifiers pre-trained on the available labeled data, each trained on a different view of the dataset (different feature vectors, different portions…).
- On each unlabeled point every model predicts; each model uses the most confident predictions from the other models as additional pseudo-labeled samples to improve on its own view.
- Cons: less effective when the views are not complementary (designing views is not trivial!); like self-training, errors in one model may be propagated to the others.
Active Learning
Uncertainty measures
| Measure | Formula | Rationale |
|---|---|---|
| Least confidence | φLC(x) = 1 − Pθ(y*|x) | The simplest: query when the confidence on the most likely class is below a threshold |
| Smallest margin | φM(x) = Pθ(y₁*|x) − Pθ(y₂*|x) | Difference between the two most confident predictions — how much more confident was the model on the top class vs the runner-up? |
| Entropy | −Σᵢ P(yᵢ|x;θ) log P(yᵢ|x;θ) | Considers the whole distribution: low entropy = one dominant class; high entropy = several activities with similar probabilities → query when above a threshold |
Worked example (3 activities): [0.8, 0.15, 0.05] → entropy ≈ 0.61, A1 clearly dominates, no query. [0.5, 0.4, 0.1] → entropy ≈ 0.94, the classifier isn't sure about A1 → a query may refine the model.
Adaptive threshold (VAR-UNCERTAINTY)
- Choosing a fixed threshold is hard in real scenarios → adjust it dynamically: when the system is uncertain, the threshold is adjusted to trigger more queries (the uncertainty region is decreased, θ←θ(1−s)); when certain, the region is widened (θ←θ(1+s)) and fewer queries fire.
- It uses least confidence, but adapts to any measure; entropy is often more robust with several classes (e.g., HAR).
Pool-based vs Stream-based
Pool-based (general ML)
- A large pool of unlabeled data; in an offline phase the system chooses the points maximizing uncertainty measures
Stream-based (AmI!)
- A human can't label a sensor sample long after it's collected → uncertainty is evaluated on each new sensor sample, and the query fires as soon as the model is uncertain
Acceptability — the real-world wall
- Most literature assumes users are constantly available — not realistic. Notifications at inappropriate moments cause stress and frustration, increase the feedback error rate, and create disaffection with the system.
- Fixes: trigger queries considering the user's context (current activity, location, social context, recent queries); interruptibility models (person + context descriptors → interruptibility level); incentivize users (gamification, economic rewards, promise of a better model). Still open, poorly investigated problems!
Activity Discovery & Evaluation
Activity discovery
- A fixed set of activities can't be foreseen in advance — different users perform different activities. Semi-supervised learning may enable activity discovery!
- Idea: pre-train on a dataset with fixed activities, then apply activity discovery methods to automatically find new patterns from unsupervised data points. New patterns may indicate new activities or different ways of performing previously considered activities (= concept evolution from §1!).
- The annotation of new clusters can be based on active learning.
- Incremental activity discovery is significantly harder: besides the pre-trained model, it must memorize the original data distribution to avoid catastrophic forgetting and find sensor patterns deviating from the original distribution (new concepts). A replay approach: predict & discover outliers → in-memory buffer → semi-supervised clustering + label propagation → extend the classifier, create class prototypes, update the replay memory.
Evaluating incremental classifiers
- Since they continuously learn, evaluate the evolution of the recognition rate. For each stream sample: classify with the current model, apply the semi-supervised method (e.g., check if an AL query is needed), store prediction + ground truth + pseudo-labels, memorize triggered queries, update the model when needed (with CL strategies).
- Visualize with a sliding window of the predictions: per window compute the metrics of interest (e.g., F1 score, number of queries). The window size sets the evaluation periodicity — it should be high enough that captured samples are representative.
Final Quiz — Exam Style
12 MCQs + 2 open-ended. Three are the actual simulation questions (Q11, Q12, Q4) verbatim; the rest are built with the professor's own recipe — sibling distractors, one-word swaps, a 2×2 grid, and one FALSE question hiding an absolutizer.
How does the replay mechanism mitigate catastrophic forgetting in continual incremental learning?
Which one of the following sentences on semi-supervised learning is FALSE?
How does Active Learning select the samples to be labeled?
a) Catastrophic forgetting: neural networks forget old concepts as new ones are learned — learning on new data would likely override the weights learned in the past, significantly degrading the recognition rate on past concepts. The stability-plasticity dilemma: stability is the capability of retaining old knowledge, plasticity the capability of learning and adapting on new knowledge; the dilemma is achieving "stability without rigidity and plasticity without chaos" — a trade-off between continuously learning and maintaining old information (an open problem in AI, which Continual Learning addresses by balancing the two). Re-training from scratch is in principle possible but not scalable in large real-world scenarios: it requires continuously storing labeled data (original training set + new data) and periodically re-training the model, which is computationally costly and cannot adapt quickly to changes in the stream. Incremental learning instead updates the classifier with mini-batches of new data.
b) Replay-based: maintain a limited subset of stored samples (replay memory) representing old knowledge, provided to the classifier along with new data at each incremental update. Variants: random replay (buffer filled by randomly sampling past labeled points), exemplars replay (representative samples, e.g., close to the feature mean of each class), constrained-based replay (last K samples + a loss on old data that can decrease but cannot increase), generative replay (when storing samples is unfeasible: previous model versions with synthetic input — e.g., a GAN — approximate old knowledge; complexity significantly increased). Pros: low-cost and easy to implement; cons: extra storage, the model may overfit on the buffer, buffer data may not be representative. Regularization-based: avoid storing raw inputs (prioritizing privacy) and alleviate memory requirements via extra regularization terms in the loss. Data-focused (LwF): knowledge distillation — the previous model (teacher) generates pseudo-labels on incoming data related to old tasks; a specific loss makes the new model (student) output a similar probability distribution on old knowledge while learning the new tasks. Prior-focused (EWC, VCL): estimate an importance value for each weight; a regularization term penalizes significant changes to important weights, so knowledge is maintained. Trade-off: replay is simple but stores raw data; regularization is privacy-friendly and memory-light but relies on approximations of past knowledge.
a) In real-life streams it is unlikely to obtain new labels (labeled data scarcity). Semi-supervised learning: a small labeled training set initializes the model before deployment; the pre-trained model then provides labels (pseudo-labels) to unlabeled data points, and the pseudo-labels are used to update the model — aiming at results close to fully supervised approaches (it also enables personalization: a collaboratively trained model is downloaded by the target user and adapted locally). Self-training: the model predicts pseudo-labels with a confidence level on unlabeled data; only reliable (high-confidence) predictions are kept and used to update the model. A common implementation is label propagation: data points close in space likely share the label — build a graph whose nodes are labeled+unlabeled points and whose edges are weighted by similarity; labels spread from labeled to unlabeled nodes based on a similarity threshold, repeated until convergence. Limits: errors in label spreading are amplified with time; it is hard to spread labels to data "different" from the training data; storing unlabeled points is impossible incrementally (periodic runs on a limited buffer). Co-training: an ensemble of classifiers, each trained on a different view of the dataset (feature subsets, portions); each model uses the most confident predictions of the other models as additional pseudo-labels on its own view. Limits: less effective with non-complementary views (designing views is not trivial) and errors propagate between models.
b) Active learning — the most effective semi-supervised strategy — requires interaction with the user: when the model is uncertain about an output, it triggers a query; the user provides the ground truth and the new labeled point updates the model; the number of queries must be minimized by targeting only the most informative points (where the classifier is most uncertain). Uncertainty measures: least confidence φLC=1−P(y*|x) (query when top-class confidence is below threshold), smallest margin φM=P(y₁*|x)−P(y₂*|x) (gap between the two most confident classes), entropy −ΣP log P (whole distribution; high entropy = several classes with similar probabilities → query above a threshold; entropy is often more robust with many classes). Thresholds can be adaptive (VAR-UNCERTAINTY: shrink the uncertainty region when uncertain to trigger more queries, widen it when certain). Pool-based vs stream-based: general ML picks the most uncertain points offline from a large unlabeled pool; in AmI a human can't label sensor data long after collection, so uncertainty is evaluated on each new sample and queries fire as soon as the model is uncertain (stream-based). Acceptability: assuming users are constantly available is not realistic — ill-timed notifications cause stress and frustration, increase the feedback error rate and cause disaffection. Mitigations: trigger queries based on the user's context (activity, location, social context, recent queries), use interruptibility models, and incentivize labeling (gamification, rewards, promise of a better model) — still open research problems.