SEM 2 Symposium 2017

  • Both models imply a variance-covariance matrix \(\hat{\pmb{\Sigma}}\), aimed to closely resemble the sample variance-covariance matrix \(\pmb{S}\) with positive degrees of freedom.

Structural Equation Modeling

\[ \hat{\pmb{\Sigma}} = \pmb{\Lambda} \left( \pmb{I} - \pmb{B} \right)^{-1} \pmb{\Psi} \left( \pmb{I} - \pmb{B} \right)^{-1\top} \pmb{\Lambda}^{\top} + \pmb{\Theta} \]

  • \(\hat{\pmb{\Sigma}}\): model implied variance-covariance matrix
  • \(\pmb{\Lambda}\): matrix containing factor loadings
  • \(\pmb{B}\): matrix containing structural relationships between latents
  • \(\pmb{\Psi}\): variance-covariance matrix of latents or latent residuals
  • \(\pmb{\Theta}\): matrix containing residual variances and covariances
    • Usually diagonal!

\[ \begin{aligned} \hat{\pmb{\Sigma}} &= \pmb{\Lambda} \pmb{\Psi} \pmb{\Lambda}^{\top} + \pmb{\Theta} \\ \begin{bmatrix} 2 & 1 & 1 & 1 \\ 1 & 2 & 1 & 1 \\ 1 & 1 & 2 & 1 \\ 1 & 1 & 1 & 2 \end{bmatrix} &= \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} \begin{bmatrix} 1 \end{bmatrix} \begin{bmatrix} 1 & 1 & 1 & 1 \end{bmatrix} + \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \end{aligned} \]

  • Degrees of freedom: 2

  • Inspiration and headache are independent after conditioning on attending SEM1&2

Gaussian Graphical Model

In network analysis, multivariate Gaussian data is modeled with the Gaussian Graphical Model (GGM): \[ \hat{\pmb{\Sigma}} = \pmb{\Delta} \left( \pmb{I} - \pmb{\Omega} \right)^{-1}\pmb{\Delta} \]

  • \(\pmb{\Delta}\) is a diagonal scaling matrix
  • \(\pmb{\Omega}\) is a symmetrical matrix with \(0\) on the diagonal and partial correlation coefficients on offdiagonal elements
    • \(\omega_{ij} = \omega_{ji} = \mathrm{Cor}\left( Y_i, Y_j \mid \pmb{Y}^{-(i,j)} \right)\)
    • Encodes a network; there is no edge between node \(Y_i\) and \(Y_j\) if \(\omega_{ij}=0\)
    • A GGM is saturated if all offdiagonal elements in \(\pmb{\Omega}\) are non-zero

\[ \boldsymbol{\Omega} = \begin{bmatrix} 0 & \\ \omega_{21} & 0 \\ 0 & \omega_{32} & 0\\ \end{bmatrix} \]

Sparse configurations of \(\pmb{\Omega}\) can often lead to dense configurations of \(\hat{\pmb{\Sigma}}\)

\[ \boldsymbol{\Omega} = \begin{bmatrix} 0 & 0.5 & 0\\ 0.5 & 0 & 0.5\\ 0 & 0.5 & 0\\ \end{bmatrix}, \pmb{\Delta} = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \]

Results in:

\[ \hat{\boldsymbol{\Sigma}} = \begin{bmatrix} 1.5 & 1 & 0.5 \\ 1 & 2 & 1\\ 0.5 & 1 & 1.5\\ \end{bmatrix} \]

  • If all nodes are connected, \(\hat{\pmb{\Sigma}}\) will be dense
  • Degrees of freedom in this example: 1
    • Parameters: 2 edges (\(\omega_{21}\) and \(\omega_{32}\)) and 3 scaling parameters (\(\delta_{11}\), \(\delta_{22}\) and \(\delta_{33}\)).

SEM problem 1: local indepence

  • Local independence is not plausible; psychological variables interact with each other
  • Thus, theoretically we should expect a saturated models

SEM problem 2: exploratory search

Networks problem 1: latent variables

  • If we could not condition on \(\eta\)

Networks problem 2: Measurement error

Generalized Network Modeling

Augment Structural Equation Models (SEM) by modeling either the residuals or latent covariances as a Gaussian Graphical model (GGM):

Implementing networks in SEM

The variance-covariance matrices in a SEM model can be modeled as a network.

\[ \hat{\pmb{\Sigma}} = \pmb{\Lambda} \left( \pmb{I} - \pmb{B} \right)^{-1} \pmb{\Psi} \left( \pmb{I} - \pmb{B} \right)^{-1\top} \pmb{\Lambda}^{\top} + \pmb{\Theta} \]

Residual networks: \[ \pmb{\Theta} = \pmb{\Delta}_{\pmb{\Theta}} \left( \pmb{I} - \pmb{\Omega}_{\pmb{\Theta}} \right)^{-1} \pmb{\Delta}_{\pmb{\Theta}} \]

Latent networks: \[ \pmb{\Psi} = \pmb{\Delta}_{\pmb{\Psi}} \left( \pmb{I} - \pmb{\Omega}_{\pmb{\Psi}} \right)^{-1} \pmb{\Delta}_{\pmb{\Psi}} \]

Residual Network Modeling (RNM)

\[ \hat{\pmb{\Sigma}} = \pmb{\Lambda} \left( \pmb{I} - \pmb{B} \right)^{-1} \pmb{\Psi} \left( \pmb{I} - \pmb{B} \right)^{-1\top} \pmb{\Lambda}^{\top} + \pmb{\Delta}_{\pmb{\Theta}} \left( \pmb{I} - \pmb{\Omega}_{\pmb{\Theta}} \right)^{-1} \pmb{\Delta}_{\pmb{\Theta}} \]

  • Network is formed at the residuals of SEM
  • Model a network while not assuming no unobserved common causes
  • Model a latent variable structure without the assumption of local independence

Latent Network Modeling (LNM)

\[ \hat{\pmb{\Sigma}} = \pmb{\Lambda} \pmb{\Delta}_{\pmb{\Psi}} \left( \pmb{I} - \pmb{\Omega}_{\pmb{\Psi}} \right)^{-1} \pmb{\Delta}_{\pmb{\Psi}} \pmb{\Lambda}^{\top} + \pmb{\Theta} \]

  • Models conditional independence relations between latent variables as a network
  • Model networks between latent variables
  • Exploratory search for conditional independence relationships between latents

Exploratory estimation

Two methods for exploratory estimation:

  • Stepwise model search
    • Start with empty residual network (RNM) or saturated latent network (LNM) and add/remove edges to improve fit
    • Works for ~8 variables
  • Penalized maximum likelihood estimation
    • LASSO regularization
    • Works for ~25 variables
    • Optimize penalized likelihood: \[ \min_{\hat{\pmb{\Sigma}}} \left[ \log \det \left( \hat{\pmb{\Sigma}}\right) + \mathrm{Trace}\left( \pmb{S} \hat{\pmb{\Sigma}}^{-1} \right) - \log \det \left( \hat{\pmb{S}}\right) - P + \nu \mathrm{Penalty} \right] \]

Performance

Simulation studies (included in paper) show:

  • High specificity (few false edges)
  • Sensitivity increases with sample size (correct edges are picked up)
  • Slow estimation with increasing number of variables

lvnet package

  • Package website: http://github.com/SachaEpskamp/lvnet
  • Uses OpenMx for estimating confirmatory latent variable network models
    • lvnet function
  • Includes exploratory search algorithms
    • lvnetSearch for step-wise search
    • lvnetLasso for penalized maximum likelihood
  • Also include earlier work on lvglasso lvglasso and EBIClvglasso functions

Installation:

library("devtools")
install_github("sachaepskamp/lvnet")

Load dataset:

# Load package:
library("lvnet")

# Load dataset:
library("lavaan")
data(HolzingerSwineford1939)
Data <- HolzingerSwineford1939[,7:15]

Setup lambda:

# Measurement model:
Lambda <- matrix(0, 9, 3)
Lambda[1:3,1] <- NA
Lambda[4:6,2] <- NA
Lambda[7:9,3] <- NA

Lambda
##       [,1] [,2] [,3]
##  [1,]   NA    0    0
##  [2,]   NA    0    0
##  [3,]   NA    0    0
##  [4,]    0   NA    0
##  [5,]    0   NA    0
##  [6,]    0   NA    0
##  [7,]    0    0   NA
##  [8,]    0    0   NA
##  [9,]    0    0   NA

Fit model:

# Fit CFA model:
CFA <- lvnet(Data, lambda = Lambda)
CFA
## 
## lvnet estimation completed.:
##      - Chi-square (24) = 85.31, p = 0
##      - RMSEA = 0.09 (95% CI: 0.07 - 0.11)
## 
##  Use summary(object) to inspect more fitmeasures and parameter estimates (see ?summary.lvnet) 
##  Use plot(object) to plot estimated networks and factor structures (see ?plot.lvnet) 
##  Use lvnetCompare(object1, object2) to compare lvnet models (see ?lvnetCompare)

CFA fit is comparable to lavaan:

HS.model <- ' visual  =~ x1 + x2 + x3
              textual =~ x4 + x5 + x6
              speed   =~ x7 + x8 + x9 '

cfa(HS.model, data=HolzingerSwineford1939)
## lavaan (0.5-23.1097) converged normally after  35 iterations
## 
##   Number of observations                           301
## 
##   Estimator                                         ML
##   Minimum Function Test Statistic               85.306
##   Degrees of freedom                                24
##   P-value (Chi-square)                           0.000

Latent network:

# Latent network:
Omega_psi <- matrix(c(
  0,NA,NA,
  NA,0,0,
  NA,0,0
),3,3,byrow=TRUE)
Omega_psi
##      [,1] [,2] [,3]
## [1,]    0   NA   NA
## [2,]   NA    0    0
## [3,]   NA    0    0

# Fit model:
LNM <- lvnet(Data, lambda = Lambda, omega_psi=Omega_psi)

# Compare fit:
lvnetCompare(cfa=CFA,lnm=LNM)
##           Df      AIC      BIC     EBIC    Chisq Chisq diff Df diff
## Saturated  0       NA       NA       NA  0.00000         NA      NA
## cfa       24 7517.490 7595.339 7835.038 85.30552  85.305522      24
## lnm       25 7516.494 7590.637 7818.921 86.31009   1.004565       1
##             Pr(>Chisq)
## Saturated           NA
## cfa       8.502553e-09
## lnm       3.162085e-01

Exploratory search for latent network:

Res <- lvnetSearch(Data, lambda = Lambda, 
            matrix = "omega_psi", verbose = FALSE)
Res$best$matrices$omega_psi
##           [,1]      [,2]      [,3]
## [1,] 0.0000000 0.4222516 0.4420685
## [2,] 0.4222516 0.0000000 0.0000000
## [3,] 0.4420685 0.0000000 0.0000000

Empirical Example

Dataset from the psych package on the Big 5 personality traits. This dataset consists of 2800 observations of 25 items designed to measure the 5 central personality traits with 5 items per trait:

library("psych")

# Load BFI data:
data(bfi)
bfi <- bfi[,1:25]

# Lavaan model:
Mod <- '
A =~ A1 + A2 + A3 + A4 + A5
E =~ E1 + E2 + E3 + E4 + E5
C =~ C1 + C2 + C3 + C4 + C5
N =~ N1 + N2 + N3 + N4 + N5
O =~ O1 + O2 + O3 + O4 + O5
'

Empirical Example

Dataset from the psych package on the Big 5 personality traits. This dataset consists of 2800 observations of 25 items designed to measure the 5 central personality traits with 5 items per trait:

library("psych")

# Load BFI data:
data(bfi)
bfi <- bfi[,1:25]

# Lavaan model:
Mod <- '
A =~ A1 + A2 + A3 + A4 + A5
E =~ E1 + E2 + E3 + E4 + E5
C =~ C1 + C2 + C3 + C4 + C5
N =~ N1 + N2 + N3 + N4 + N5
O =~ O1 + O2 + O3 + O4 + O5
'

# Transform lavaan model to lvnet model matrices:
mod <- lav2lvnet(Mod, bfi)

# Best RNM model:
RNM <- lvnetLasso(bfi, lambda = mod$lambda, 
                  lassoMatrix = "omega_theta", 
                  nCores = 8,
                  tuning.max = 0.5, nTuning = 100, 
                  criterion = "ebic")
RNM_best <- RNM$best

  • Included in my dissertation (chapter 7)
  • sachaepskamp.com/Dissertation
  • Other topics:
    • LASSO estimation of GGM models
    • Stability checks in network models
    • Network models in applied clinical practice
    • Time-series analysis
    • The Ising model in psychometrics
    • Pretty pictures

  • Guyon, H., Falissard, B., & Kop, J. L. (2017). Modelling mental attributes in Psychology–an epistemological discussion: Network Analysis vs Latent Variables. Frontiers in Psychology, 8, 798-. Chicago

  • Forbes, M. K., Wright, A. G., Markon, K., & Krueger, R. (2017). Evidence That Psychopathology Symptom Networks do not replicate Have Limited Replicability. Journal of Abnormal Psychology (https://osf.io/5dfc4/)

Conclusion

  • Generalizing SEM and GGM leads to two frameworks:
    • Residual network model (RNM)
    • Latent network model (LNM)
  • RNM and LNM allow for specifying many models
    • CFA with residual network
    • Graphical VAR
    • Network with (some) nodes latent
  • Confirmatory fit and exploratory search are possible
  • Future directions:
    • Exploratory search algorithms need to be optmized
    • Estimation needs to be fleshed out
    • SEM extensions need to be included (e.g., ordinal data, FIML)

Thank you for your attention!