Real-time Transfer Active Learning for Functional Regression and Prediction based on Multi-output Gaussian Process

Zengchenghao Xia; Zhiyong Hu; Qingbo He; Chao Wang

doi:10.1109/TSP.2024.3451412

Back

Real-time Transfer Active Learning for Functional Regression and Prediction based on Multi-output Gaussian Process

Journal article

Peer reviewed

Real-time Transfer Active Learning for Functional Regression and Prediction based on Multi-output Gaussian Process

Zengchenghao Xia, Zhiyong Hu, Qingbo He and Chao Wang

IEEE transactions on signal processing, Vol.72, pp.4163-4177

2024

DOI: 10.1109/TSP.2024.3451412

View Online

Abstract

Active learning provides guidance for the design and modeling of systems with highly expensive sampling costs. However, existing active learning approaches suffer from cold-start concerns, where the performance is impaired due to the initial few experiments designed by active learning. In this paper, we propose using transfer learning to solve the cold-start problem of functional regression by leveraging knowledge from related and data-rich signals to achieve robust and superior performance, especially when only a few experiments are available in the signal of interest. More specifically, we construct a multi-output Gaussian process (MGP) to model the between-signal functional relationship. This MGP features unique innovations that distinguish the proposed transfer active learning from existing works: i) a specially designed covariance structure is proposed for characterizing within-and between-signal inter-relationships and facilitating interpretable transfer learning, and ii) an iterative Bayesian framework is proposed to update the parameters and prediction of the MGP in real-time, which significantly reduces the computational load and facilitates the iterative active learning. The inter-relationship captured by this novel MGP is then fed into active learning using the integrated mean-squared error (IMSE) as the objective. We provide theoretical justifications for this active learning mechanism, which demonstrates the objective (IMSE) is monotonically decreasing as we gather more data from the proposed transfer active learning. The real-time updating and the monotonically decreasing objective together provide both practical efficiency and theoretical guarantees for solving the cold-start concern in active learning. The proposed method is compared with benchmark methods through various numerical and real case studies, and the results demonstrate the superiority of the method, especially when limited experiments are available at the initial stage of design.

Active learning

Bayesian analysis

cold-start problem

Data models

Field effect transistors

Gaussian processes

Iterative methods

multi-output Gaussian process

Real-time systems

Transfer learning

Uncertainty

Details

Title: Subtitle: Real-time Transfer Active Learning for Functional Regression and Prediction based on Multi-output Gaussian Process
Creators: Zengchenghao Xia - Department of Industrial and Systems Engineering, University of Iowa, Iowa City, IA, USA
Zhiyong Hu - University of Science and Technology of China
Qingbo He - Shanghai Jiao Tong University
Chao Wang - University of Iowa
Resource Type: Journal article
Publication Details: IEEE transactions on signal processing, Vol.72, pp.4163-4177
Publisher: IEEE
DOI: 10.1109/TSP.2024.3451412
ISSN: 1053-587X
eISSN: 1941-0476
Number of pages: 14
Grant note: National Science Foundation (NSF): CMMI-2436025
This work was partially supported by the National Science Foundation (NSF) under Grant CMMI-2436025.
Language: English
Electronic publication date: 08/30/2024
Date published: 2024
Academic Unit: Industrial and Systems Engineering
Record Identifier: 9984702833802771

Metrics

1 Record Views