arXiv Machine Learning

Shifting-based Optimizable Linear Relaxations for General Activation Functions

arXiv:2606. 20292v1 Announce Type: new Abstract: The use of neural networks (NNs) is rapidly increasing, including in safety- and security-critical domains.

arXiv Machine Learning
Aug 19

Certified but Private: Scalable Zero-Knowledge Proofs for Neural Network Guarantees

PANDA is a scalable system that uses zero‑knowledge proofs to certify the robustness and fairness of neural networks without revealing their private parameters. Built on the CROWN robustness framework, PANDA introduces a novel algorithm for proving linear relaxation bounds on non‑linear activation layers, producing lightweight proofs. The system can generate proofs for networks with over 2.9 million parameters in just five minutes and verify them in ten seconds, scaling polynomially with network size and enabling verification of models four orders of magnitude larger than prior ZKP‑based approaches.

By Youwei Zhong, Ben Merbaum, Timos Antonopoulos, Ning Luo, Charalampos Papamanthou, Katerina Sotiraki, Ruzica Piskac
arXiv Machine Learning
1d ago

Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning

The paper introduces ZFO, a lightweight framework that separates direction selection from step-size determination in large‑scale neural network optimization. ZFO uses a trusted first‑order optimizer to pick a search direction and then performs only two additional objective evaluations to build a local curvature‑aware model, selecting an adaptive step within a bounded interval. The authors provide theoretical guarantees for reliable curvature estimation, near‑optimal step selection, and convergence to a stationary point, and demonstrate that ZFO improves optimization and final performance over fixed‑step first‑order baselines on language‑model fine‑tuning tasks.

By Cristian McGee, El Houcine Bergou, Aritra Dutta
arXiv Machine Learning
1d ago

HUANet: Hard-Constrained Unrolled ADMM for Constrained Convex Optimization

HUANet is a deep neural network architecture that unrolls the Alternating Direction Method of Multipliers (ADMM) into a trainable model for accelerating parametric constrained convex optimization. It embeds a hard‑constrained neural network in each ADMM iteration, using a differentiable correction stage to enforce affine equalities of the primal subproblem. The method also incorporates first‑order optimality conditions into a self‑supervised training loss, and numerical experiments on benchmark problems and a control application demonstrate its effectiveness in speeding up constrained convex optimization.

By Trinh Tran, Binh Nguyen, Truong X. Nghiem