Penalized likelihood estimation of a fixed-effect and a mixed-effect transfer function model

Motivated by the need of estimating the main spawning period of North Sea cod, we develop a common transfer function model with a panel of contemporaneously correlated times series data. This model incorporates (i) the smoothness on the parameters by assuming that the second differences are small and (ii) the contemporaneous correlation by assuming that the errors have a general variance-covariance matrix. Penalized likelihood estimation of this model requires an iterative procedure that is developed in this work. We develop three methods for determining confidence bands: frequentist, Bayesian, and bootstrap (both nonparametric and parametric). A simulation study on the frequentist and Bayesian confidence bands motivated by the cod spawning data is conducted and the results of those simulations are compared. The model is then used on the cod spawning data, with all confidence bands computed. The results of this analysis are discussed. We then delve further into our model by discussing the theory behind this model. We prove a theorem that shows that the estimated regression parameter vector is a consistent estimate of the true regression parameter. We further prove that this estimated regression parameter vector has an asymptotic normal distribution. Both theorems are proved while assuming mild conditions.

We further develop our model by incorporating between-series variation in the transfer function, with the random effect assumed to have a normal distribution with a "smooth" mean vector. We implement the EM algorithm to do the penalized likelihood estimation. We consider five different specifications of the variance-covariance matrix of the random transfer function model, namely, a general variance-covariance matrix, a diagonal matrix, a multiple of the identity matrix, an autoregressive matrix of order one, and a multiplicative error specification. Since the computation of confidence bands would lead to numerical problems, we introduce a bootstrap approach for estimating the confidence bands. We consider both the nonparametric and parametric bootstrap approaches. We then apply this model to estimate the cod spawning period, while also looking into the different specifications of the variance-covariance matrix of the random effect, the two types of bootstrapped confidence bands, and model checking.