| Naslov: | Prunability of multi-layer perceptrons trained with the forward-forward algorithm |
|---|
| Avtorji: | ID Nikov, Mitko (Avtor)
ID Strnad, Damjan (Avtor)
ID Podgorelec, David (Avtor) |
|---|
| Datoteke: |  mathematics-13-02668-v2.pdf (7,01 MB)
MD5: C04BE66BFE440B30671ECBB8D216ACF6
|
|---|
| Jezik: | Angleški jezik |
|---|
| Vrsta gradiva: | Članek v reviji |
|---|
| Tipologija: | 1.01 - Izvirni znanstveni članek |
|---|
| Organizacija: | FERI - Fakulteta za elektrotehniko, računalništvo in informatiko
|
|---|
| Opis: | We explore the sparsity and prunability of multi-layer perceptrons (MLPs) trained using the Forward-Forward (FF) algorithm, an alternative to backpropagation (BP) that replaces the backward pass with local, contrastive updates at each layer. We analyze the sparsity of the weight matrices during training using multiple metrics, and test the prunability of FF networks on the MNIST, FashionMNIST and CIFAR-10 datasets. We also propose FFLib—a novel, modular PyTorch-based library for developing, training and analyzing FF models along with a suite of FF-based architectures, including FFNN, FFNN+C and FFRNN. In addition to structural sparsity, we describe and apply a new method for visualizing the functional sparsity of neural activations across different architectures using the HSV color space. Moreover, we conduct a sensitivity analysis to assess the impact of hyperparameters on model performance and sparsity. Finally, we perform pruning experiments, showing that simple FF-based MLPs exhibit significantly greater robustness to one-shot neuron pruning than traditional BP-trained networks, and a possible 8-fold increase in compression ratios while maintaining comparable accuracy on the MNIST dataset. |
|---|
| Ključne besede: | Forward-Forward, sparsity, pruning, model compression, machine learning, neural network |
|---|
| Status publikacije: | Objavljeno |
|---|
| Verzija publikacije: | Objavljena publikacija |
|---|
| Poslano v recenzijo: | 22.06.2025 |
|---|
| Datum sprejetja članka: | 15.08.2025 |
|---|
| Datum objave: | 19.08.2025 |
|---|
| Založnik: | MDPI |
|---|
| Leto izida: | 2025 |
|---|
| Št. strani: | 23 str. |
|---|
| Številčenje: | Vol. 13, iss. 16, [article no.] 2668 |
|---|
| PID: | 20.500.12556/DKUM-94526  |
|---|
| UDK: | 004.9 |
|---|
| COBISS.SI-ID: | 246013187 |
|---|
| DOI: | 10.3390/math13162668 |
|---|
| ISSN pri članku: | 2227-7390 |
|---|
| Avtorske pravice: | © 2025 by the authors |
|---|
| Datum objave v DKUM: | 20.08.2025 |
|---|
| Število ogledov: | 152 |
|---|
| Število prenosov: | 8 |
|---|
| Metapodatki: |    |
|---|
| Področja: | Ostalo
|
|---|
|
:
|
Kopiraj citat |
|---|
| | | | Skupna ocena: | (0 glasov) |
|---|
| Vaša ocena: | Ocenjevanje je dovoljeno samo prijavljenim uporabnikom. |
|---|
| Objavi na: |  |
|---|
Postavite miškin kazalec na naslov za izpis povzetka. Klik na naslov izpiše
podrobnosti ali sproži prenos. |
