Influence Function

Definition

If an estimator $\hat\psi$ of a functional parameter $\psi:\mathcal{P}\to\mathbb{R}$ is asymptotically linear, then an influence function (IF) $\phi$ exists such that $\sqrt{n}\big(\hat\psi-\psi(P)\big)=\frac{1}{\sqrt{n}}\sum_{i=1}^n \phi(O_i;P)+o_p(1),\quad E_P[\phi]=0.$ The asymptotic variance is $\mathrm{Var}=E_P[\phi^2]$. Equivalently, the IF is the Gateaux derivative along the path $P_\varepsilon=(1-\varepsilon)P+\varepsilon\delta_o$: $\phi(o)=\frac{d}{d\varepsilon}\psi(P_\varepsilon)\big|_{\varepsilon=0}$. von Mises expansion: $\psi(\bar P)-\psi(P)=\int\phi(o;P)\,d(\bar P-P)(o)+R_2$.

Intuitive Understanding

How much does a single observation perturb the estimate — the “leverage” of each data point. Because the remainder $R_2$ is a product (second-order) of nuisance estimation errors, the first-order bias of the plug-in estimator can be corrected by the mean of the IF (One-step Estimator · AIPW).

Related Concepts

• Efficient Influence Function — the minimum-variance IF (efficiency lower bound)
• One-step Estimator · AIPW · TMLE — IF-based correction and estimation
• Neyman-Orthogonal Score · Double Machine Learning — orthogonality / cross-fitting

Key Papers

• Hines, Dukes, Diaz-Ordaz & Vansteelandt, “Demystifying Statistical Learning Based on Efficient Influence Functions”, The American Statistician 76(3):292–304, 2022 — introduction to the EIF, start here
• Kennedy, “Semiparametric doubly robust targeted DML: a review”, arXiv:2203.06469, 2022
• van der Vaart, Asymptotic Statistics, 1998, Ch.25