政务大数据应用与分析 (80700673)
清华大学社会科学学院
理解网络
目的:描述网络结构,识别网络特征
分析网络
目的:揭示网络构成因素和原因
A、B、C三方,存在几种建立关系的方式?
(22)3 = 64
A→B→C + A→C
定义网络结构平衡(Structural Balance)
Top Down:
Cores → factions → modularity
Bottom Up:
Cliques → n-cliques → n-clans
The k-core of a network graph G is the maximal subgraph H ⊆ G in which all vertices have degree at least k
邻国相望,鸡犬之声相闻,民至老死,不相往来。 (王弼 and 楼宇烈 2011)
Aka., Cluster analysis
A.k.a., 凝聚力
高 ⇒
低 ⇒
基于网络的分析可以:
主动机制
被动机制
统计推断流程
传统计量的前提假定
网络分析:研究个体 → 研究关系
⇓
随机网络分析
通盘考虑:
Interdependence
Dyadic Covariate
Structural
\[ \begin{align} P(N, \boldsymbol{\theta}) =& \frac{exp(\sum_i\theta_iz_i(N))}{\kappa(\theta)}.\\ \downarrow& \\ P(N, \boldsymbol{\theta}) =& \frac{exp(\boldsymbol{\theta'h}(N))}{\sum_{N^*\in N}exp(\boldsymbol{\theta'h}(N^*))}. \end{align} \]
ERGM Assumptions
Edge Effect (邻居效应)
Diffusion Effect (扩散分析)
Complete Network analysis (全网分析)
常见分类(叉)linkage
分类(叉)标准 (\(d(a,b)\) \(a\) 和 \(b\) 之间的距离)
基本思路
\[ Max(\frac{类内相似性}{类间相似性}) \]
步骤
优点:计算速度快。易于理解
缺点:
使用中心点而非平均值
优点
缺点
基于谱分解(spectral decomposition)提取数据特征,然后根据特征聚类。1
步骤:
通常对于多维数据采用Mixture of models, 比如Gaussian Mixture Models (GMM)
\[ L(\boldsymbol{\mu_1}, \dots, \boldsymbol{\mu_k}, \boldsymbol{\Sigma_1}, \dots, \boldsymbol{\Sigma_k} | \boldsymbol{x_1}, \dots, \boldsymbol{x_n}). \]
定义密度,然后寻找最大化密度的点
常见方法:
优点
缺点
“Baseline network”比较分析的一种
\[ Network_{obs}\quad vs.\quad Network_{ran} \]
观测网络与随机网络在这些方面是否相异
通过permutation test实现:
OV: 网络
EV: 解释变量adjacency matrices
\[ E(\text{Advice}_{ij}) = \beta_0 + \beta_1\text{Reports}_{ij} + \beta_2\text{Friends}_{ij} +... \]
Network Logit Model
Coefficients:
Estimate Exp(b) Pr(<=b) Pr(>=b) Pr(>=|b|)
(intercept) -0.4251826 0.6536504 0.4 0.6 0.4
x1 0.5794452 1.7850479 1.0 0.0 0.1
x2 3.0589826 21.3058690 1.0 0.0 0.0
Goodness of Fit Statistics:
Null deviance: 582.2436 on 420 degrees of freedom
Residual deviance: 548.1452 on 417 degrees of freedom
Chi-Squared test of fit improvement:
34.09844 on 3 degrees of freedom, p-value 1.888613e-07
AIC: 554.1452 BIC: 566.266
Pseudo-R^2 Measures:
(Dn-Dr)/(Dn-Dr+dfn): 0.07509042
(Dn-Dr)/Dn: 0.05856387
Contingency Table (predicted (rows) x actual (cols)):
Actual
Predicted 0 1
0 188 122
1 42 68
Total Fraction Correct: 0.6095238
Fraction Predicted 1s Correct: 0.6181818
Fraction Predicted 0s Correct: 0.6064516
False Negative Rate: 0.6421053
False Positive Rate: 0.1826087
Test Diagnostics:
Null Hypothesis: qap
Replications: 10
Distribution Summary:
(intercept) x1 x2
Min -4.50574 -2.69571 -1.40610
1stQ -3.92006 -0.19556 -0.04667
Median -3.25925 0.08638 0.52835
Mean -3.28825 0.08504 0.33152
3rdQ -2.73010 0.67105 0.86806
Max -2.09563 2.38630 1.15204
Simple Random Graph Model
↓
P1 Model
↓
P* Model
↓
ERGM &rarr GEGM/TEGM…
\[ P(X = x) = \frac{exp(\theta_LL(x))}{k(\theta)}. \]
Bernoulli Assumption
\[ P(X = x) = \frac{exp(\theta_LL(x) + \color{red}{\theta_MM(x) + \sum_i\alpha_iy_{i+} + \sum_j\beta_iy_{+j}})}{\kappa(\theta)}. \]
Bernoulli Dyad Independence Assumption
\[ P(X = x) = \frac{exp(\sum_i\theta_i\color{red}{z_i(x)})}{\kappa(\theta)}. \]
zi: any network statistics加在一起
Dyad Independence Markov assumption
\[ P(N, \boldsymbol{\theta}) = \frac{exp(\boldsymbol{\theta'h}(N))}{\sum_{N^*\in N}exp(\boldsymbol{\theta'h}(N^*))}. \]
诊断
Super important!
局限
发展
……
Modeling Peer Influence
\[\begin{align} \boldsymbol{Y}^{(1)} &= \boldsymbol{XB},\\ \boldsymbol{Y}^{(t)} &= \alpha\boldsymbol{WY}^{t} + (1 - \alpha) Y^{(1)}. \end{align}\]