Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This work characterizes the finite-sample behavior of the maximum likelihood estimator (MLE) in well-specified Poisson regression under Gaussian design. The authors reveal that while MLE existence exhibits a sharp phase transition at $n > d$, achieving the optimal $\mathcal{O}(d/n)$ excess risk requires a sample size that scales exponentially with $\|\beta\|$. Between these bounds lies an intermediate regime where the MLE exists but fails to achieve optimal rates, which the authors bound whenever $n > d^{1+\epsilon}$ by developing new PAC-Bayes bounds for sub-Gamma random vectors and matrices.
Poisson MLE existence is deceptively cheap: estimators appear as soon as $n > d$, but recovering the optimal $\mathcal{O}(d/n)$ convergence rate requires sample sizes that explode exponentially with the true parameter norm.
We study the maximum likelihood estimation of the coefficient β in well-specified Poisson regression. Using tools from empirical process theory and random conic geometry, we show that the probability of existence of the maximum likelihood estimator (MLE) exhibits a sharp phase transition at the threshold n > d. We then determine a minimum threshold exponential in the norm of β on the sample size n to guarantee with high probability an excess risk of the asymptotic order d/n. We reveal the existence of an intermediate regime in Poisson regression, when n is larger than d but smaller than this exponential threshold, where the MLE exists but does not achieve the optimal rate d/n. We close the gap between the two regimes up to a d^{1+ε} term with ε \in (0, 1) by providing an upper bound on the distance between the MLE and β whenever n > d^{1+ε}. Along the way, we provide two generalizations of well-known PAC-Bayes inequalities regarding sub-Gamma random vectors and sub-Gamma random matrices that are of independent interest and that we use extensively to prove the main results of the present paper.