Statistics (Machine Learning) Papers@StatsPapers
12m
Kernel-Based Learning of Analytic Operators and Parametric PDEs

Giovanni S. Alberti, Markus Holzleitner, Lorenzo Rosasco, Matteo Santacesaria
arxiv.org/abs/2611.09612 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 πš–πšŠπšπš‘.πš‚πšƒ]
We study kernel-based learning of analytic operators from finite samples, with a focus on how the regularity of the target should determine the choice of the kernel. We establish embeddings of analytic operators into vector-valued reproducing kernel Hilbert spaces and derive finite-sample, high-probability guarantees that quantify the uncertainty due to random sampling: in the noiseless setting, suitable analytic kernels yield rates exponential in a fractional power of the sample size, while Sobolev-type kernels yield polynomial rates. We apply the theory to the solution operator of a parametric elliptic PDE, where the training data are themselves finite-element approximations, and obtain bounds that separate the sampling and discretization contributions to the error and prescribe how the mesh size should be coupled to the sample size. The results are illustrated numerically.
0
0
0
18
Statistics (Machine Learning) Papers@StatsPapers
1h
Geometry-Aware Score-Repellent Monte Carlo

Jie Hu, Lingyun Chen, Do Young Eun
arxiv.org/abs/2611.08909 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
πŸ’¬To appear at NeurIPS 2026
The recently proposed Score-Repellent Monte Carlo (SRMC) method improves MCMC sampling by tilting the target density Ο€ in mathbbR^d with the factor exp(-Ξ± ΞΈ_n^β†’p s(x)), where the score function s(x) = βˆ‡_x log Ο€(x), and ΞΈ_n is the score average from past samples. However, a single scalar repellence strength Ξ±β‰₯0 cannot handle both the steep and the flat directions in anisotropic energy landscapes at once. We introduce Geometry-Aware SRMC (GA-SRMC), which generalizes the Euclidean alignment to ΞΈ^β†’p M s(x) for any symmetric positive-definite matrix M. A stochastic-approximation central limit theorem under a trace constraint on M identifies the trace-normalized inverse Fisher M_optpropto S^-1 as the unique minimizer of the worst-direction covariance bound, elevating the Fisher information S=CovΟ€(s,s) from a proposal-side preconditioner (e.g., FisherMALA, natural gradient) to the bound-optimal repulsion-side matrix, with a single online estimate SΜ‚_n serving both roles at no additional compu
0
0
1
73
Statistics (Machine Learning) Papers@StatsPapers
2h
Beyond the Lower-Star Filtration: Alternative Filtration Methods for Persistent Homology

Innocent Abaa, Umar Islambekov
arxiv.org/abs/2611.08833 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 πš–πšŠπšπš‘.π™°πšƒ 𝚜𝚝𝚊𝚝.𝙰𝙿]
In the lower-star filtration (LSF), simplices are assigned filtration values according to the maximum-vertex rule, with each simplex entering the filtration when its highest-valued vertex appears. A limitation of this construction is that some higher-dimensional simplices may enter simultaneously with their boundary faces, causing certain topological features to be filled immediately upon formation and thus not appear in the resulting persistence diagram. To address this limitation, we propose three alternative filtration methods: the Ξ΅-lower-star filtration (ELSF), the Lβ‚š filtration (LpF), and the softmax filtration (SMF). ELSF introduces a dimension-dependent delay to the maximum-vertex rule, allowing higher-dimensional simplices to enter the filtration after their boundary faces, whereas LpF and SMF use aggregation functions that depend on all vertex values rather than solely on their maximum. By modifying how filtration values are assigned, these methods allow simplices to enter th
0
0
0
106
Statistics (Machine Learning) Papers@StatsPapers
2h
High-Dimensional Off-Policy Evaluation without Sample Splitting using an Onsager Correction

Saksham Jain, Guillaume Braun, Masaaki Imaizumi
arxiv.org/abs/2611.08794 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 πš–πšŠπšπš‘.πš‚πšƒ]
Reward models used for off-policy evaluation (OPE) are commonly selected using held-out data. When sample size and context dimension grow proportionally, the difference in test risk between models fitted on the full sample and on a fixed fraction of it need not vanish. Held-out evaluation can therefore consistently estimate the test risk of the model trained on the smaller sample, but because the relative ranking of candidate reward models can change after full-data refitting, held-out selection can be suboptimal. We develop an Onsager-corrected, target-policy-weighted criterion that estimates the full-data reward model's out-of-sample risk under the target policy using only its training observations. This enables early stopping along the full-data gradient-descent trajectory without sample splitting. For neural networks with fixed depth and hidden width under Gaussian contexts, single-index rewards, context-dependent multi-index target policies, and known context-independent logging,
0
0
1
135
Statistics (Machine Learning) Papers@StatsPapers
3h
Blind Noise Equalization for Large Data Matrices under General Variance Profiles

Boris Landa, Francesco Strino, Fabio Parisi, Ruiqi Li, Yuval Kluger
arxiv.org/abs/2611.08784 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 𝚎𝚎𝚜𝚜.πš‚π™Ώ πš–πšŠπšπš‘.πš‚πšƒ 𝚜𝚝𝚊𝚝.𝙼𝙴]
Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We p
0
0
1
168
Statistics (Machine Learning) Papers@StatsPapers
4h
pESTO: Iterative p-greedy kernel regression with risk-driven Early STOpping

L. Bruni Bruno, P. Massa, E. Perracchione, M. Trombini
arxiv.org/abs/2611.08555 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
Kernel-based greedy algorithms select suitable approximation nodes by minimizing point-wise error indicators, such as the power function; in this context, they are usually referred to as p-greedy approaches. In the presence of noisy data, deterministic criteria may fail, as the discrepancy between the true solution and its estimate is influenced by both approximation error and noise amplification, and the balance between the two is unknown. This study explicitly separates these two contributions, which are governed by the power function and a Lebesgue-type term, respectively. These bounds motivate the use of the Stein's unbiased risk estimator (SURE) as a proxy of the mean squared error, which in turn provides a principled and computable stopping criterion for the number of nodes to select in the p-greedy procedure. Numerical experiments demonstrate that this approach effectively balances deterministic accuracy and statistical noise contributions.
0
0
1
114
Statistics (Machine Learning) Papers@StatsPapers
4h
ELPAC: Endpoint-Anchored Latent Progression with Stage-Varying Multimodal Coordination

Shengxian Ding, Yifei Zhang, Xinyuan Tian, Yuanshuang Guo, Yize Zhao
arxiv.org/abs/2611.08541 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 𝚜𝚝𝚊𝚝.𝙼𝙴]
πŸ’¬NeurIPS 2026
Aging cohorts increasingly combine imaging, plasma, and cognitive measurements, but most subjects provide only a partially observed cross-sectional snapshot, leaving latent stage, progression subtype, and stage-varying multimodal coordination to be inferred jointly. Existing methods estimate the cohort-level aging process, model covariance under observed covariates, or fuse modalities for prediction. However, they rarely target the joint problem of endpoint-anchored staging and progression-resolved residual coordination across modalities, even though subject staging, subtype assignment, and multimodal interplay are mutually dependent. We introduce ELPAC, a Bayesian framework with two interlocking constructions. The first integrates subtype trajectories backward from clinically reliable endpoint groups along biologically structured velocity components encoding modality-specific monotone direction and optional sequential ordering priors. The second factorizes residual multimodal variatio
0
0
1
228
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Unifying Central Flow, Rod Flow, and the Edge Coupling

Eric Regis, Sinho Chewi
arxiv.org/abs/2611.08358 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
At the edge of stability, the iterates of gradient descent oscillate along the sharp directions of the Hessian. This poses a significant challenge for continuous-time models as the usual continuous-time approximation – the gradient flow – admits no oscillations. Several continuous-time models have been proposed to capture the behavior at the edge of stability, among them central flow and rod flow. Although both models can be interpreted as performing a coarse-graining in time, the connection between them remains poorly understood. Here we show that both flows can be derived from a third object, the edge coupling: a functional on consecutive iterates whose critical points are the fixed points and period-two orbits of gradient descent. It is revealed that the two flows correspond to coarse-grainings at different time scales. Rod flow coarse-grains over only the fastest time scale, while central flow coarse-grains over both the fast and intermediate time scales. Furthermore, using the edg
0
0
5
430
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Series-wise Cross-validation for Global Time Series Forecasting

Vitor Cerqueira, Mariana Oliveira, Luis Roque
arxiv.org/abs/2611.08252 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
We consider the problem of cross-validation (CV) in time series forecasting, with a focus on global forecasting models based on deep learning. The prevailing CV strategy in forecasting is to partition data across time with the last part of each series used for testing. With global models trained on collections of time series, however, a fundamentally different approach becomes possible: partitioning across series and time. We formalize series-wise CV for global forecasting models and benchmark several instantiations, including K-fold CV, bootstrapping, and Monte Carlo CV, on 11 datasets with 10 neural network architectures. We evaluate CV procedures along five dimensions: performance estimation, overoptimism rate, model selection accuracy, regret, and downstream forecasting performance. We withhold a random fraction of series from training so that they appear only in the test set. This protocol simulates the emergence of new series not seen during training, a common scenario in global
0
0
0
240
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting

Junyu Chen, Likai Chen, Weining Wang
arxiv.org/abs/2611.08160 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 πšŽπšŒπš˜πš—.𝙴𝙼]
Attention-based time series forecasters commonly group consecutive observations into patches. Patch length is known to have a substantial impact on forecasting performance, yet its selection remains largely empirical. We develop a theoretical framework that explains why patch length matters for forecasting accuracy and provides a principled basis for its selection. For linear state-space models, we derive an exact factorization of the forecasting coefficient matrix that separates forecast dynamics from filtering dynamics. We show that dominance of a single forecast component at long horizons does not generally imply a rank-one representation. This explains why longer horizons alone need not reduce representational complexity. We combine the factorization with estimation bounds to develop a patch-selection criterion that balances representation and estimation errors under temporal dependence. Our analysis establishes conditions under which balanced patching is justified and clarifies ho
0
1
4
250
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Identifying Neural Dynamics Using Nonlinear Interventional State Space Models

Hanna M. Tolle, Carles Balsells-Rodas, Yixin Wang, Pedro A. M. Mediano, Amin Nejatbakhsh
arxiv.org/abs/2611.08113 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
State-space models (SSMs) recover low-dimensional latent dynamics from high-dimensional observations, but latents are generally not identifiable from observational data alone. Interventional state-space models (iSSMs) use perturbations to break observational symmetries and identify parameters up to permutation, scaling, and shift. However, prior work restricts identifiability to linear latent dynamics and piecewise-linear emissions, excluding many behaviors characteristic of neural circuits. We extend iSSMs to a substantially broader model class: we prove that (i) affine identifiability holds under general injective emission functions and Gaussian latent transitions, and (ii) with interventions, identifiability up to permutation, scaling, and shift is preserved under general nonlinear latent dynamics. We validate our theory on simulated FitzHugh-Nagumo dynamics, and then demonstrate two applications: adjudicating between competing models of decision-making; and predicting responses to
0
0
5
266
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Joint Modeling of Conditional Mean and Variability in Multilevel Functional Data: Application to Postprandial Glucose Trajectories in the AEGIS Study

Yang Xiao, Francisco Gude, Marcos Matabuena
arxiv.org/abs/2611.07937 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 𝚜𝚝𝚊𝚝.𝙰𝙿 𝚜𝚝𝚊𝚝.𝙼𝙴]
Functional data analysis provides a framework for studying time-varying associations between covariates and high-frequency biological signals. In digital health, participants often contribute repeated functional trajectories, yet existing multilevel regression methods largely focus on the conditional mean, overlooking variability and its role in biological heterogeneity. We propose a heteroskedastic functional mixed-effects regression framework that jointly models the conditional mean and variance of repeated trajectories, with participant-specific random effects in both components. We also develop a participant-level bootstrap to quantify uncertainty in estimated functional parameters. We apply the method to postprandial glucose trajectories from the AEGIS study. Premeal glucose, carbohydrate intake, and meal type show distinct time-dependent associations with glucose variability, whereas HbA1c is more strongly associated with the conditional mean than with variability. Substantial be
0
1
2
232
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Network Effect Decomposition for Causal Effect Estimation under Network Interference

Zhirong Huang, Wenzhen Zhang, Guangquan Lu, Boyan Chen, Debo Cheng, Shichao Zhang
arxiv.org/abs/2611.07879 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙰𝙸]
Estimating causal effects from observational network data requires accounting for both individual treatment and neighborhood exposure. Existing outcome models commonly incorporate these intervention variables into a joint predictor and estimate causal effects through prediction contrasts, leaving the corresponding response components implicit. We propose the Network Effect Decomposition Network (NEDNet), which parameterizes the conditional outcome response as a query-invariant pre-treatment baseline plus separate treatment- and exposure-response pathways. Path-specific neighborhood contextualization supplies local pre-treatment information to the two pathways. We characterize the fitted response through a unique reference-normalized decomposition, which is equivalent to a zero mixed treatment–exposure contrast. Experiments on semi-synthetic benchmarks derived from BlogCatalog and Flickr show that NEDNet achieves the lowest PEHE among the compared methods for individual, peer, and total
0
0
4
248
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Task-Calibrated Random Features via Influence-Based Error Cancellation

Shinto Eguchi
arxiv.org/abs/2611.07209 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
Random features approximate kernel integrals by Monte Carlo, yet only selected frequency-sampling errors affect a downstream task. We propose Task-Calibrated Random Features (TCRF), which balance frequency moments constructed from task-specific influence using weights shared across kernel entries. Closed-form weights coincide with classical minimum-distance calibration; the contribution is a reusable, task-derived calibration space. We show that first-order variance equals the residual variance after projection onto this space. For kernel ridge regression, complete cancellation yields a non-Gaussian second-order limit. For kernel Stein discrepancy, a subset identity and disjoint-block variance decomposition eliminate the linear component of pilot noise. Positive exponential-tilting weights preserve the positive-semidefinite feature representation and support higher feasible calibration ranks. Paired experiments show improved approximation to exact kernel ridge regression at larger freq
0
0
0
188
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
ShapleyOG: Fast Adaptive Shapley Value Estimation with Odd Gaussian Processes

David Rundel, R. Teal Witter, Santo M. A. R. Thies, Matthias Feurer, Fabian Fumagalli, Bernd Bischl
arxiv.org/abs/2611.07173 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
Shapley values are a central tool for attribution in machine learning. Since their exact computation requires evaluating the cooperative game to be explained on exponentially many player coalitions, they are typically estimated from a limited number of evaluations. Recently, ShaplEIG has shown that adaptively selecting the coalitions to evaluate substantially improves sample efficiency over classical estimators. This, however, comes at the cost of a sequential procedure that, in every iteration, refits a Gaussian process surrogate of the game and selects the coalition with the largest expected information gain about the Shapley values. The resulting computational overhead limits its applicability to costly games with low evaluation budgets. We present fast adaptive Shapley value estimation with Odd Gaussian Processes (ShapleyOG) to overcome this limitation. ShapleyOG leverages the fact that Shapley values depend only on the odd component of the game, and we show that under paired sampl
0
0
1
236
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
Bayesian Tensor Train Kernel Machines with Automatic Model Complexity Inference

Afra Kilic, Kim Batselier
arxiv.org/abs/2611.07129 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
πŸ’¬Code: github.com/afrakilic/BTTKM
Tensor Network (TN) kernel machines accelerate supervised learning by using low-rank TNs to reduce computation and memory. However, most TN-based methods are deterministic with manually tuned ranks and feature dimensions, while the existing Bayesian TN kernel machine is restricted to the CANDECOMP/PARAFAC decomposition (CPD), whose single rank is shared by all modes. We extend this framework to the tensor train (TT) and propose Bayesian Tensor Train Kernel Machines (BTT-KM), whose hierarchical ARD-type priors on the TT cores infer the TT ranks and feature dimensions from data. Since each TT core is coupled to the others only through its neighbors, its closed-form posterior update requires propagating the contributions of all other cores along the chain. Mean-field variational inference yields a Bayesian alternating least squares algorithm with the same computational complexity as its deterministic counterpart and a closed-form predictive distribution. A synthetic experiment illustrates
0
1
5
419
Statistics (Machine Learning) Papers@StatsPapers
Nov 6
ToroFit: Fitting Wedge Sums of Tori to High-Dimensional Data

Tanishq Bhatia, Jose A. Perea, Tarek M. Zikry
arxiv.org/abs/2611.06990 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢 πš–πšŠπšπš‘.π™°πšƒ]
Data arising in scientific applications often exhibits complex nonlinear structure. Identifying and parameterizing said nonlinearities can enable better modeling, enhance predictive analysis, data interpolation, denoising, and anomaly detection. In this paper, we develop theoretical and computational methods to detect and parametrize Euclidean point cloud data lying near a wedge sum of tori. Specifically, we use persistent cohomology and cup products to identify the distinct toroidal components, and on each component, we optimize over low-energy multivariate trigonometric polynomials for best data fit. The ensuing optimization problem reduces to generalized ridge regression, which can be solved efficiently via standard techniques. We demonstrate the utility of the proposed framework in both synthetic and real datasets, including interpolation of low frame-rate recurrent videos, attractor recovery in quasiperiodic dynamical systems, and the parametrization of gene-space cycle data for p
0
0
3
229
Statistics (Machine Learning) Papers@StatsPapers
Nov 5
Alignment Myopia in Generative AI

Remi Denton, Aida Davani, Cynthia L Bennett, Michael Madaio, Roma Patel, Himabindu Lakkaraju
arxiv.org/abs/2611.06830 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.π™²πšˆ 𝚌𝚜.𝙻𝙢]
Current generative AI alignment relies on a myopic paradigm that prioritizes immediate conversational satisfaction. By relying on short-horizon feedback, this approach ignores the profound, longitudinal externalities of persistent human-AI interaction, which can lead to adverse outcomes such as cognitive atrophy, social erosion, and diminished human agency. To address this, we propose a paradigm shift toward long-horizon alignment that accounts for how AI reshapes human experience over time. To this end, we map the tensions between near-term utility and long-term costs of AI use across four facets of human well-being to reveal outcomes that current reward models fail to capture. Ultimately, we advocate for moving beyond simple preference optimization toward alignment strategies that actively foster long-term human well-being by navigating these complex temporal scales.
0
0
0
187
Statistics (Machine Learning) Papers@StatsPapers
Nov 5
TestBED: An Open-Source Toolkit for Bayesian Experimental Design

Gavin Kerrigan, Marcel Hedman, Sahel Iqbal, Tom Rainforth
arxiv.org/abs/2611.06633 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
Bayesian experimental design (BED) is a principled framework for intelligent data acquisition with applications across a wide range of scientific and industrial domains. However, the development and adoption of BED remains hindered by a lack of standardised software and benchmarks. Existing methods are typically implemented in bespoke codebases and evaluated on a small subset of tasks, making methods difficult to apply, compare, or extend. To address this, we present TestBED, a general-purpose, open-source toolkit for Bayesian experimental design. TestBED decomposes the BED process into modular, reusable components spanning tasks, objectives, training mechanisms, policy architectures, and evaluation procedures, together with scalable implementations of core primitives such as expected information gain estimators. We provide a wide range of these components, which can be freely composed, allowing existing methods to be reproduced within a common framework, while also enabling previously
0
0
3
221
Statistics (Machine Learning) Papers@StatsPapers
Nov 5
Risk-Sensitive Non-Parametric and Kernel Bandits

Devdan Dey, Sujoy Bhore, Avishek Ghosh
arxiv.org/abs/2611.06489 [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙢]
We study non-parametric bandits in which arms are evaluated using Cumulative Prospect Theory (CPT) rather than expected reward. We consider continuous action spaces with arm-dependent noise and propose Non-Parametric CPT Bandits (mathttNPCB), an optimistic algorithm that decouples mean estimation from CPT-based decision-making. We prove CPT regret of order OΜƒ(√(T dim_E(β„±,T^{-2)) logN(β„±,T^-2,β€–β‹…β€–_∞)}), where dim_E is the Eluder dimension of the function class β„±, T is the time horizon, and N is its covering number in the sup norm. This matches the order-wise regret of classical non-parametric bandits, showing that CPT risk sensitivity incurs no additional order-wise regret cost. We extend mathttNPCB to unknown noise CDFs, retaining the known-CDF mean-learning term and adding a CDF-learning cost governed by a distributional Eluder dimension. We also study Kernel-CPT bandits and obtain information-gain-based regret bounds for a broad range of kernels, matching the corresponding risk-neutral
0
0
3
251
Load more