BART (Bayesian Additive Regression Trees)

Definition

A Bayesian ensemble method that models the outcome as a sum of many trees

$$Y = \sum_{k=1}^{K} g_k(X, W; T_k, M_k) + \epsilon, \quad \epsilon \sim N(0, \sigma^2)$$

where:

• $g_k$: the $k$-th tree
• $T_k$: tree structure
• $M_k$: leaf node values
• Prior on $(T_k, M_k)$

---

Application to Causal Inference

Potential Outcome Modeling

$$\mu(X, W) = E[Y \mid X, W] = \sum_k g_k(X, W)$$

CATE Estimation

$$\hat{\tau}(X) = \hat{E}[Y \mid X, W=1] - \hat{E}[Y \mid X, W=0]$$

Estimated by sampling from the posterior:

$$\hat{\tau}(X) = \frac{1}{S}\sum_{s=1}^{S} \left[\sum_k g_k^{(s)}(X, 1) - \sum_k g_k^{(s)}(X, 0)\right]$$

---

Model Structure

Prior Specification

Tree structure $T$ prior:

• Node split probability: $P(\text{split at depth } d) = \alpha(1+d)^{-\beta}$
• Typically $\alpha = 0.95, \beta = 2$

Leaf value $M$ prior:

$$\mu_{kl} \sim N(0, \sigma^2_\mu)$$

Variance prior:

$$\sigma^2 \sim \text{Inverse-Gamma}$$

MCMC Sampling

Posterior estimation via Gibbs Sampling:

1. $T_k \mid \text{others}$: MH step
2. $M_k \mid T_k, \text{others}$: Conjugate update
3. $\sigma^2 \mid \text{others}$: Conjugate update

---

Advantages and Disadvantages

Advantages

Advantage	Description
Uncertainty quantification	Posterior → confidence/credible intervals
Flexible nonlinearity	Captures complex interactions
Regularization	Prior prevents overfitting
Continuous/binary treatment	Both can be handled
Automatic variable selection	Discovers important variables

Disadvantages

Disadvantage	Description
Computational cost	Slow due to MCMC
Convergence diagnostics	Requires checking MCMC convergence
Hyperparameters	Sensitive to prior specification
Large-scale data	Hard to scale

---

Causal BART Variants

BCF (Bayesian Causal Forests)

Hahn et al. (2020): separates the treatment effect

$$Y = \mu(X) + \tau(X) \cdot W + \epsilon$$

• $\mu(X)$: Prognostic function (BART)
• $\tau(X)$: Treatment effect function (BART)

ps-BART

Includes the propensity score as a covariate:

$$Y = f(X, e(X), W) + \epsilon$$

---

Implementation

R (dbarts, bartCause)

library(dbarts)

# Basic BART
bart_fit <- bart(x.train = cbind(X, W),
                 y.train = Y,
                 ntree = 200)

# CATE estimation
X1 <- cbind(X, W = 1)
X0 <- cbind(X, W = 0)

pred1 <- predict(bart_fit, X1)
pred0 <- predict(bart_fit, X0)

cate <- colMeans(pred1 - pred0)

# Credible interval
cate_samples <- pred1 - pred0
ci <- apply(cate_samples, 2, quantile, c(0.025, 0.975))

R (bartCause)

library(bartCause)

# Causal BART
fit <- bartc(y = Y, z = W, x = X,
             method.rsp = "bart",
             method.trt = "bart")

# CATE
cate <- predict(fit)

---

Comparison with Causal Forest

Property	BART	Causal Forest
Inference	Bayesian	Frequentist
Uncertainty	Posterior	Bootstrap/Asymptotic
Honest	No	Yes
Speed	Slow (MCMC)	Fast
Theory	Posterior consistency	$\sqrt{n}$-normality

---

Related Concepts

• Tree-based Methods Overview - integration of tree-based methods
• Causal Forest - Frequentist alternative
• Honest Estimation - overfitting prevention
• HTE - estimation target

---

Key Papers

• Hill, J. L. (2011). Bayesian nonparametric modeling for causal inference. JCGS
• Chipman, H. A., George, E. I., & McCulloch, R. E. (2010). BART: Bayesian additive regression trees. Annals of Applied Statistics
• Hahn, P. R., Murray, J. S., & Carvalho, C. M. (2020). Bayesian regression tree models for causal inference. Bayesian Analysis
• yaoSurveyCausalInference2021 - Section 3.4.3