Marginal Coverage Credit Reduces Redundant Exploration in Parallel State-Entropy Optimization
Marginal Coverage Credit Reduces Redundant Exploration in Parallel State-Entropy Optimization
arXiv:2608.27507v1 Announce Type: new Abstract: Policy Gradient for Parallel State Entropy maximization (PGPSE) expands state-space coverage by training independently parameterized policies in replicated copies of the same environment. However, its pooled team-entropy score measures only collective exploration and cannot identify policies that contribute non-redundant coverage. We introduce Marginal Coverage Credit for PGPSE (MCC-PGPSE), which combines leave-one-policy-out coverage with state-owner specialization to estimate policy-specific credit. MCC-PGPSE preserves PGPSE's pooled objective and redistributes non-negative auxiliary intrinsic rewards according to these credits without changing their total mass. This redistribution is designed to discourage redundant visitation and promote complementary coverage. We evaluated MCC-PGPSE in controlled environments, seven public discrete-state benchmarks, and representative Room and Maze settings from the original PGPSE protocol. Across all tested settings, MCC-PGPSE produced positive final window gains in normalized team state entropy and state support over the Entropy baseline. Controlled-task comparisons and the fixed-suite public aggregate were significant, whereas five-seed original-protocol comparisons were directionally consistent. Ablations and credit alignment controls indicate that most gains arise from leave-one-policy-out coverage rather than non-uniform weighting, mismatched credit, or neural novelty alone. These results support contribution-conditioned auxiliary reward allocation as an interpretable approach to improving complementary coverage among parallel policies in discrete state spaces.
📖 Cet article provient d'une source externe.
🔗 Lire l'article complet sur la source →200 mots extraits · Source originale
Ordinateur portable 10,1 pouces, 1 Go + 8 Go, Android 6.0 A33 Dual-Core ARM Cortex-A9 jusqu'à 1,5 GHz, WiFi, carte SD, disque U (rose)
✅ Découvrez toutes les caractéristiques.
✅ Vérifiez la disponibilité actuelle.
✅ Consultez les avis des acheteurs.
