Study and research notes — causal inference, decision-making under uncertainty, and personalization, across clinical and industry domains.https://lowellth.com/https://lowellth.com/notes/dunnhumby-track1-segmentation/https://lowellth.com/notes/dunnhumby-track1-segmentation/NMF latent factors (92.44% explained variance) + K-Means yield 7 stable behavioral segments (Bootstrap ARI 0.77) with per-segment marketing actions. Illustrative case study on the public Dunnhumby retail dataset.Sat, 13 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/dunnhumby-track2-causal-targeting/https://lowellth.com/notes/dunnhumby-track2-causal-targeting/Meta-learner / Causal Forest CATE under severe positivity violation (PS AUC 0.989); an OPE-validated policy targets ~31% of customers and surfaces counter-intuitive negative-CATE segments. Hypothesis-generating on public data.Sat, 13 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/causal-pricing-cate-scm-framework/https://lowellth.com/notes/causal-pricing-cate-scm-framework/An applied case study using only public datasets (LendingClub, iPinYou) that combines CATE estimation for price-sensitivity heterogeneity with SCM-based moderator analysis to design individual-level, risk-based pricing and RTB bidding policies — all findings illustrative and projected, not proprietary.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/customer-segmentation-causal-targeting-case-study/https://lowellth.com/notes/customer-segmentation-causal-targeting-case-study/An end-to-end applied case study on the public Dunnhumby dataset — NMF latent factors and K-Means segmentation feeding meta-learner / Causal Forest HTE and an OPE-validated optimal targeting policy, with a candid look at positivity violation and counter-intuitive "sleeping dog" segments.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/customer-segmentation/https://lowellth.com/notes/customer-segmentation/Customer Segmentation is the unsupervised task of partitioning customers into a finite set of segments by similarity in behavior, value, and preference. A common recipe is latent-factor decomposition followed by clustering: behavioral features → NMF (non-negative, parts-based decomposition) → factor scores → K-Means → segments.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/hte-to-policy-through-line/https://lowellth.com/notes/hte-to-policy-through-line/One methodological spine — estimate heterogeneous treatment effects and turn them into individual-level policies — powers both clinical sequential treatment decisions and industrial targeting, pricing, and recommendation.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/llm-multilayer-attributes-recsys/https://lowellth.com/notes/llm-multilayer-attributes-recsys/A case study on extracting a 3-layer attribute taxonomy (product / perceptual / theory-grounded) with LLM/VLM pipelines, turning it into user profiles and a mixture-of-experts adaptor, and plugging it into standard recommenders across two public domains (fashion + music).Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/marketing-attribution-simulation-to-scale/https://lowellth.com/notes/marketing-attribution-simulation-to-scale/A case study comparing 10+ multi-touch attribution methods against a known-ground-truth simulator, then scaling them on the public Criteo dataset, closing the loop with budget off-policy evaluation for channel allocation.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/optimal-targeting-policy/https://lowellth.com/notes/optimal-targeting-policy/An Optimal Targeting Policy maps covariates $x$ to a treatment decision $\pi(x)\in\{0,1\}$ so as to maximize policy value:Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/partial-id-sensitivity-bridge/https://lowellth.com/notes/partial-id-sensitivity-bridge/When real-world data fail strong ignorability, point identification gives way to bounds, proxies, and sensitivity analysis — an honest hierarchy of evidence that connects credible causal claims to semiparametric efficiency.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/rtb-causal-bidding-strategy/https://lowellth.com/notes/rtb-causal-bidding-strategy/A five-stage case study on the public iPinYou RTB dataset that moves from pCTR/pCVR prediction through causal effect estimation (CATE, SCM) to budget-constrained optimal bidding and off-policy policy evaluation.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/sequential-adaptive-decisions/https://lowellth.com/notes/sequential-adaptive-decisions/A synthesis essay tracing one methodological spine through sequential decision-making under uncertainty — exploration–exploitation in bandits, off-policy evaluation, and optimal/dynamic treatment regimes — that powers clinical adaptive trials and real-time bidding alike.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/targeting-overview/https://lowellth.com/notes/targeting-overview/Targeting & Profiling is the industrial face of personalization. The same methodological core (heterogeneous effect estimation → individual-level optimal policy; MOC-Personalization) appears in clinical settings as "optimal treatment assignment per patient," and in industry as "optimal campaign/exposure assignment per customer." This domain answers who…Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/uplift-modeling/https://lowellth.com/notes/uplift-modeling/Uplift is the causal increment that a treatment (campaign exposure, coupon, recommendation) induces in an individual's outcome (purchase, conversion). For binary treatment $W\in\{0,1\}$, outcome $Y$, and covariates $X$.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/user-profiling/https://lowellth.com/notes/user-profiling/User Profiling is the task of inferring a personal preference profile (taste, context, latent patterns) from a customer's behavioral history and representing it as a vector. It is the shared input layer for targeting, segmentation, and recommendation — the industry-side counterpart to patient covariate/multimodal representations (Multimodal Clinical Data) in the clinical domain.Fri, 12 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/anytime-valid-ope/https://lowellth.com/notes/anytime-valid-ope/Anytime-valid off-policy evaluation that provides time-uniform off-policy value confidence sequences valid at any stopping time; based on e-processes/confidence sequences.Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/anytime-valid-inference-overview/https://lowellth.com/notes/anytime-valid-inference-overview/Game-theoretic statistics that resolves the "peeking" problem of fixed-sample hypothesis testing. The mathematical foundation for real-time monitoring of identification-validity drift.Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/confidence-sequence/https://lowellth.com/notes/confidence-sequence/A confidence sequence (CS) $(C_t){t\ge1}$ is a sequence of confidence intervals with time-uniform coverage:Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/decision-making-overview/https://lowellth.com/notes/decision-making-overview/Methods for (sequential) decision-making under uncertainty — from bandit regret to RL, off-policy evaluation, and dynamic/optimal treatment regimes. Underpins both clinical (DTR/OTR) and industrial (targeting, bidding) personalization.Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/dynamic-treatment-regimes/https://lowellth.com/notes/dynamic-treatment-regimes/A DTR is a sequence of decision rules $\{d_t(H_t)\}_{t=1}^T$ mapping the accumulated history $H_t$ (covariates, prior treatments, intermediate outcomes) to a treatment. The optimal treatment regime (OTR) maximizes the expected long-term outcome $E[Y^{d}]$. Estimation:Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/efficient-influence-function/https://lowellth.com/notes/efficient-influence-function/Among the regular asymptotically linear (RAL) estimators of a (semi)parametric model, the IF with the smallest variance is the efficient influence function (EIF), and its variance equals the semiparametric efficiency bound (the supremum of the Cramér-Rao bounds over all parametric submodels)…Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/e-process/https://lowellth.com/notes/e-process/An e-value $E$ is a nonnegative random variable with $EP[E]\le 1$ ($\forall P\in H0$) under the null $H0$. An e-process $(Et)$ is a nonnegative process such that $E\tau$ is an e-value at any stopping time $\tau$ ($E[E\tau]\le1$) — typically a nonnegative supermartingale under the null.…Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/influence-function/https://lowellth.com/notes/influence-function/If an estimator $\hat\psi$ of a functional parameter $\psi:\mathcal{P}\to\mathbb{R}$ is asymptotically linear, then an influence function (IF) $\phi$ exists such thatThu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/negative-control-outcome/https://lowellth.com/notes/negative-control-outcome/An NCO is an outcome variable guaranteed a priori to be unaffected by the treatment's causal influence, yet still cast in the shadow of the same confounder $U$. By contrast, an NCE (negative control exposure) is an exposure with no causal effect on the outcome. If the "apparent effect" on an NCO is nonzero → a signal of unmeasured confounding (detection) → correct for it via proximal methods.Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/multi-armed-bandits/https://lowellth.com/notes/multi-armed-bandits/$K$ arms; at each round $t$ pull $A_t$ and observe a reward. Minimize cumulative regret:Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/off-policy-evaluation/https://lowellth.com/notes/off-policy-evaluation/Estimate the value $V(\pie)=E{\pie}[\sum r]$ of a target policy $\pie$ from logs collected under a different behavior policy $\pib$.Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/one-step-estimator/https://lowellth.com/notes/one-step-estimator/Corrects first-order bias by adding the empirical mean of the estimated EIF to the plug-in $\psi(\hat P)$:Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/partial-identification/https://lowellth.com/notes/partial-identification/When point identification is impossible due to a lack of assumptions, we only know that the parameter lies in the identified set $\ThetaI$ (often an interval $[\thetaL,\thetaU]$) compatible with the data plus assumptions. Manski's assumption-free / worst-case bounds are the starting point. sharp bounds =…Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/proximal-causal-inference/https://lowellth.com/notes/proximal-causal-inference/When unmeasured confounding $U$ is present, the causal effect is identified using two types of proxies:Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/tmle/https://lowellth.com/notes/tmle/A procedure that corrects (targets) a plug-in estimator toward the target parameter:Thu, 11 Jun 2026 00:00:00 GMThttps://lowellth.com/notes/escm2/https://lowellth.com/notes/escm2/A model that integrates a counterfactual risk regularizer based on the Inverse Propensity Score (IPS) and the Doubly Robust estimator into ESMM, in order to address ESMM's two theoretical limitations — Inherent Estimation Bias (IEB) and Potential Independence Priority (PIP).Wed, 25 Mar 2026 00:00:00 GMThttps://lowellth.com/notes/esmm/https://lowellth.com/notes/esmm/A multi-task model that addresses CVR's Sample Selection Bias and Data Sparsity problems simultaneously by exploiting the sequential user behavior $\text{impression} \to \text{click} \to \text{conversion}$ to learn CVR indirectly over the entire impression space.Wed, 25 Mar 2026 00:00:00 GMThttps://lowellth.com/notes/deepfm/https://lowellth.com/notes/deepfm/DeepFM (Guo et al., 2017) is a CTR prediction model that combines an FM component and a Deep component in parallel, jointly learning low-order (explicit) and high-order (implicit) feature interactions.Fri, 13 Feb 2026 00:00:00 GMThttps://lowellth.com/notes/factorization-machine/https://lowellth.com/notes/factorization-machine/The Factorization Machine (FM) is a general-purpose prediction model proposed by Rendle (2010) that models interactions between all pairs of features as inner products of latent factor vectors.Fri, 13 Feb 2026 00:00:00 GMThttps://lowellth.com/notes/pnn/https://lowellth.com/notes/pnn/PNN (Qu et al., 2016) is a CTR prediction model that introduces a product layer between the embedding layer and the DNN hidden layers, explicitly capturing the interactions among feature embeddings before passing them to the DNN.Fri, 13 Feb 2026 00:00:00 GMThttps://lowellth.com/notes/wide-and-deep/https://lowellth.com/notes/wide-and-deep/Wide & Deep (Cheng et al., 2016) is a CTR prediction model that combines a linear wide component (memorization) with a DNN deep component (generalization). It was first deployed for Google Play app recommendation.Fri, 13 Feb 2026 00:00:00 GMThttps://lowellth.com/notes/multi-task-learning/https://lowellth.com/notes/multi-task-learning/A learning paradigm that jointly trains several related tasks, improving generalization through a shared representation.Thu, 29 Jan 2026 00:00:00 GMThttps://lowellth.com/notes/aipw/https://lowellth.com/notes/aipw/- $\hat{\mu}_t(X)$: Outcome model ($E[Y|T=t, X]$)Wed, 29 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/att/https://lowellth.com/notes/att/Average treatment effect for the group that actually received treatmentTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/back-door-criterion/https://lowellth.com/notes/back-door-criterion/The Back-door Criterion (Pearl, 1993) is a graphical criterion for identifying a causal effect from observational data. It determines whether a set of variables $Z$ is sufficient to identify the causal effect of $X \rightarrow Y$.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/bart/https://lowellth.com/notes/bart/A Bayesian ensemble method that models the outcome as a sum of many treesTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/a-b-testing/https://lowellth.com/notes/a-b-testing/A/B testing is the online application of the randomized controlled trial (RCT), estimating causal effects by randomly exposing two or more variants to users.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/cate/https://lowellth.com/notes/cate/The Conditional Average Treatment Effect (CATE) is the average treatment effect given covariates $X=x$:Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/causal-forest/https://lowellth.com/notes/causal-forest/Causal Forest is a causal-inference application of the Generalized Random Forest (GRF) proposed by Athey, Tibshirani, and Wager (2019), splitting so as to maximize the heterogeneity of treatment effects.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/cevae/https://lowellth.com/notes/cevae/A method that uses a VAE to infer latent confounders and estimate causal effects.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/cfr/https://lowellth.com/notes/cfr/A deep learning method that learns balanced representations via IPM (Integral Probability Metric) regularizationTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/collider/https://lowellth.com/notes/collider/A collider is a variable affected by both the treatment (X) and the outcome (Y) (a common effect). In the structure X → C ← Y, C is a collider.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/confounder/https://lowellth.com/notes/confounder/A confounder is a variable that affects both the treatment (X) and the outcome (Y) (a common cause), creating a spurious (non-causal) association between X and Y.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/constraint-based-methods-overview/https://lowellth.com/notes/constraint-based-methods-overview/Constraint-based methods recover the causal graph by testing conditional independence (CI) relations in the data. Under the faithfulness assumption, they exploit the correspondence between CI relations and d-separation.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/contextual-bandits/https://lowellth.com/notes/contextual-bandits/Contextual Bandits are a multi-armed bandit problem in which the optimal action (arm) varies depending on the context.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/cuped/https://lowellth.com/notes/cuped/CUPED (Controlled-experiment Using Pre-Experiment Data) is a technique that leverages pre-experiment data to reduce the variance of A/B tests.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/d-separation/https://lowellth.com/notes/d-separation/d-separation (directional separation) is a graphical criterion in a DAG for determining whether two sets of variables are conditionally independent given a third set.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/dag/https://lowellth.com/notes/dag/A DAG (Directed Acyclic Graph) is a graph that visually represents the causal relationships among variables. It is a core tool in causal inference for grasping confounding structure and deciding an identification strategy.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/design-effect/https://lowellth.com/notes/design-effect/The Design Effect (DEFF) measures the impact of a complex sampling design on variance relative to simple random sampling.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/double-debiased-ml/https://lowellth.com/notes/double-debiased-ml/A methodology for performing valid statistical inference on a low-dimensional parameter of interest $\theta0$ in the presence of a high-dimensional nuisance parameter $\eta0$.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/do-operator/https://lowellth.com/notes/do-operator/The do-operator is Pearl's formalization of intervention.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/doubly-robust-estimator/https://lowellth.com/notes/doubly-robust-estimator/The Doubly Robust (DR) Estimator combines an outcome-regression model and a propensity-score model, remaining consistent as long as just one of the two is correctly specified.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/dr-learner/https://lowellth.com/notes/dr-learner/The DR-Learner is a two-stage doubly robust estimator for CATE that regresses a pseudo-outcome on the covariates.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/endogeneity/https://lowellth.com/notes/endogeneity/Endogeneity is the problem that arises when an explanatory variable is correlated with the error term.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/fundamental-problem-of-causal-inference/https://lowellth.com/notes/fundamental-problem-of-causal-inference/The problem that, for the same individual, the outcomes under treatment (W=1) and control (W=0) cannot be observed simultaneouslyTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/hte/https://lowellth.com/notes/hte/The phenomenon in which the treatment effect varies with an individual's characteristicsTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/instrumental-variables/https://lowellth.com/notes/instrumental-variables/Instrumental variables (IV) are exogenous variables used to address the problem of endogeneity.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/ipw/https://lowellth.com/notes/ipw/Estimating treatment effects by using the inverse of the propensity score as weightsTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/ite/https://lowellth.com/notes/ite/The treatment effect for individual $i$Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/mdp/https://lowellth.com/notes/mdp/A Markov Decision Process (MDP) is a mathematical framework for sequential decision-making problems.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/mediator/https://lowellth.com/notes/mediator/A mediator is an intermediate variable lying on the causal pathway through which a treatment (X) affects an outcome (Y). In the structure X → M → Y, M is the mediator.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/meta-learners/https://lowellth.com/notes/meta-learners/Meta-learners are a general term for algorithms that estimate the CATE by leveraging existing supervised learning methods (base learners).Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/policy-trees/https://lowellth.com/notes/policy-trees/Policy Trees, proposed by Athey & Wager (2021), are an interpretable policy-learning method.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/positivity/https://lowellth.com/notes/positivity/The probability of receiving treatment lies strictly between 0 and 1 for every covariate valueTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/propensity-score-matching/https://lowellth.com/notes/propensity-score-matching/Matching treated and control individuals with similar propensity scoresTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/r-learner/https://lowellth.com/notes/r-learner/R-Learner (Residualized Learner) is a meta-learner that estimates the CATE using residualized outcomes and residualized treatments based on the Robinson Transformation.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/representation-learning-overview/https://lowellth.com/notes/representation-learning-overview/Methods for learning representations that are independent of treatment while remaining useful for outcome prediction.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/s-learner/https://lowellth.com/notes/s-learner/The S-Learner (Single Learner) is a Meta-learner that estimates the response function with a single model including the treatment indicator as a feature, then computes the CATE.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/scm/https://lowellth.com/notes/scm/An SCM (Structural Causal Model) is a framework for mathematically expressing the causal relationships among variables. It is the core of Pearl's causal inference framework.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/score-based-methods-overview/https://lowellth.com/notes/score-based-methods-overview/Score-based methods assign a score function to each graph and search for the graph that best fits the data. Unlike constraint-based methods, they optimize model fit without CI tests.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/statistical-power/https://lowellth.com/notes/statistical-power/Statistical power is the probability of detecting an effect when it truly exists.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/strong-ignorability/https://lowellth.com/notes/strong-ignorability/An assumption combining Ignorability and PositivityTue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/t-learner/https://lowellth.com/notes/t-learner/The T-Learner (Two Learner) is a Meta-learner that estimates the CATE by training separate models for the treatment group and the control group.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/sutva/https://lowellth.com/notes/sutva/The potential outcome of one unit is not affected by the treatment assignment of other units, and only a single version exists for each treatment level.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/thompson-sampling/https://lowellth.com/notes/thompson-sampling/Thompson Sampling is a Bayesian approach that balances exploration and exploitation.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/treatment-effects-overview/https://lowellth.com/notes/treatment-effects-overview/A systematic overview of the treatment effects that serve as the estimands in the Potential Outcome Framework.Tue, 28 Jan 2025 00:00:00 GMThttps://lowellth.com/notes/x-learner/https://lowellth.com/notes/x-learner/The X-Learner is a three-stage algorithm that leverages imputed treatment effects, a meta-learner that effectively exploits group imbalance and the structural properties of the CATE.Tue, 28 Jan 2025 00:00:00 GMT