Online Resource Allocation with an Endogenous Markov State: Fewer LP Solves Earn More

Key points

• In this problem, a transient fluid LP benchmark upper bounds the expected reward of every nonanticipating policy, while a stationary LP supplies randomized state-dependent controls.1
• We assume that the stationary LP has a unique optimum and identify primal nondegeneracy and irreducibility of the optimal induced kernel as important regularity conditions in this framework.1
• With a known request prior, we show that, under nondegeneracy and irreducibility, both frequent and infrequent re-solving attain $O(1)$ regret.1
• With an unknown request prior, we develop a three-phase U-shaped infrequent re-solving policy that coordinates learning and inventory correction with $O(\log\log T)$ LP solves.1

Sources (1)

• [1]Online Resource Allocation with an Endogenous Markov State: Fewer LP Solves Earn More
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 07:21 AM

  We study finite-horizon online resource allocation with i.i.d. requests and an endogenous Markov state on a finite state space: each action affects the transition of the state that governs future rewards and resource consumption.

  In this problem, a transient fluid LP benchmark upper bounds the expected reward of every nonanticipating policy, while a stationary LP supplies randomized state-dependent controls.

Extractive summary: sentences quoted from the sources.