Signals, recorded over time, are often observed as mixtures of multiple source signals. To extract relevant information from such measurements one needs to determine the mixing coefficients. In case of weakly stationary time series with uncorrelated source signals, this separation can be achieved by jointly diagonalizing sample autocovariances at different lags, and several algorithms address this task. Often the mixing estimates contain close-to-zero entries and one wants to decide whether the corresponding source signals have a relevant impact on the observations or not. To address this question of model selection we consider the recently published second-order blind identification procedures SOBIdef and SOBIsym which provide limiting distributions of the mixing estimates. For the first time, such distributions enable informed decisions about the presence of second-order stationary source signals in the data. We consider a family of linear hypothesis tests and information criteria to perform model selection as second step after parameter estimation. In simulations we consider different time series models. We validate the model selection performance and demonstrate a good recovery of the true zero pattern of the mixing matrix.