A Deeper Analysis of Block-Sparse Featurizers
A Deeper Analysis of Block-Sparse Featurizers
arXiv:2608.27515v1 Announce Type: new Abstract: The recently introduced block-sparse featurizer (BSF; Fel et al., 2026) is similar to a sparse autoencoder (SAE), but its atomic unit is a small subspace (a block of directions) rather than a single direction. It is designed for features that live on low-dimensional manifolds, which are especially frequent in vision. This work studies the BSF's strengths and weaknesses, finding how it still somewhat suffers from classic SAE failure modes, like feature splitting and composition. We propose several architectural changes to the BSF, including a Tournament Top-K selection rule that significantly reduces feature splitting, and we also extend the block paradigm to the crosscoder.
📖 Cet article provient d'une source externe.
🔗 Lire l'article complet sur la source →108 mots extraits · Source originale
Xiaomi POCO M7 Global, 6 Go + 128 Go, lecteur d'empreintes digitales latéral, écran 6,9 pouces, Xiaomi HyperOS 2, Snapdragon 685 4G Octa Core, NFC, réseau 4G (bleu)
✅ Découvrez toutes les caractéristiques.
✅ Vérifiez la disponibilité actuelle.
✅ Consultez les avis des acheteurs.
