Estimation of semiparametric mixed analysis of covariance model

A semiparametric mixed analysis of covariance model is postulated. This model is estimated by imbedding restricted maximum likelihood estimation and smoothing splines regression into the backfitting algorithm along with the bootstrap method. To mitigate overparameterization, the heterogeneous effect of covariates across groups of experimental units is assumed to affect the response through a nonparametric function. Simulation studies exhibited the capability of the postulated model (and estimation procedures) in increasing predictive ability and stabilizing variance components estimates even for small sample size and with minimal covariate effect, and regardless of the extent of misspecification error. The method also exhibits relative advantage even for unbalanced cases.