publications

2026

1. arXiv

   Checkable NTK Positivity and Finite-Width Gradient Descent for Scalar- and Vector-Valued PINNs with Strong-Form, Weak-Form, and Nonlocal Linear Constraints

   Zifan Lyu

   arXiv preprint arXiv:2610.04357. Under review at ICLR 2027 , 2026

   Abs arXiv Bib PDF Code

   We give checkable positive-definiteness criteria for the limiting neural tangent kernel (NTK) and high-probability finite-width gradient-descent guarantees for scalar- or vector-valued physics-informed neural networks (PINNs) with linear constraints. The constraints may be strong-form differential rows of any fixed finite order, including coupled systems, with any linear initial or boundary conditions; weak-form residual and boundary functionals; or finite-measure nonlocal observations such as integral, nonlocal-diffusion, and Dirac-data rows, all in any dimension. For each class, positive definiteness of the limiting NTK is equivalent to a rank condition on a finite coefficient, functional, or moment matrix of the fixed design: a certificate computed before training that detects structural zero modes. The model hypotheses are those of an ordinary two-layer PINN, a smooth nonpolynomial activation with bounded symmetric initialization, and are met by standard choices such as tanh with uniform initialization. Given a certificate, explicit width and step-size conditions ensure, with high probability, that the empirical NTK retains at least half the limiting gap and that full-batch gradient descent decreases the training loss geometrically at every iteration. Controlled experiments check the certificates and the finite-width mechanism.

   @article{lyu2026ntkpinn,
     title = {Checkable {NTK} Positivity and Finite-Width Gradient Descent for Scalar- and Vector-Valued {PINNs} with Strong-Form, Weak-Form, and Nonlocal Linear Constraints},
     author = {Lyu, Zifan},
     journal = {arXiv preprint arXiv:2610.04357},
     year = {2026},
   }

2. ICML

   Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm that Provably Exploits Model Similarity

   Zifan Lyu, Chahine Nejma, Tobias Wegel, and 2 more authors

   In International Conference on Machine Learning (ICML), 2026

   Abs arXiv Bib PDF Code

   Large Language Models are typically benchmarked by evaluating every model on every test query. For practitioners seeking the best model to deploy, this is often wasteful: if a model clearly performs worse than others, there is no need to precisely estimate its performance. Best-arm identification algorithms can be naturally applied to drastically reduce costs by adaptively allocating evaluation budget. Further, language models often respond similarly to the same prompt-a property previous work has tried to leverage with mixed success. We propose Synchronized Successive Rejects (SySRs), augmenting the classical Successive Rejects algorithm with paired comparisons. Unlike prior attempts to leverage model similarity in best-model identification, our approach is hyperparameter-free and enjoys performance guarantees that improve with the degree of similarity between evaluated models. Empirically, our method outperforms all baselines in terms of average error rate across 15 standard benchmarks, and in terms of worst-case budget for reliably identifying the best model.

   @inproceedings{lyu2026sysrs,
     title = {Cutting {LLM} Evaluation Costs with {SySRs}: A Bandit Algorithm that Provably Exploits Model Similarity},
     author = {Lyu, Zifan and Nejma, Chahine and Wegel, Tobias and Yang, Fanny and Dorner, Florian E.},
     booktitle = {International Conference on Machine Learning (ICML)},
     year = {2026},
   }