| Naslov: | Controllable speech-driven gesture generation with selective activation of weakly supervised controls |
|---|
| Avtorji: | ID Crnek, Karlo (Avtor)
ID Rojc, Matej (Avtor) |
|---|
| Datoteke: |  applsci-15-09467-v2.pdf (1,63 MB)
MD5: C4928D336842855F4B0DD151617041C0
|
|---|
| Jezik: | Angleški jezik |
|---|
| Vrsta gradiva: | Članek v reviji |
|---|
| Tipologija: | 1.01 - Izvirni znanstveni članek |
|---|
| Organizacija: | FERI - Fakulteta za elektrotehniko, računalništvo in informatiko
|
|---|
| Opis: | Generating realistic and contextually appropriate gestures is crucial for creating engaging embodied conversational agents. Although speech is the primary input for gesture generation, adding controls like gesture velocity, hand height, and emotion is essential for generating more natural, human-like gestures. However, current approaches to controllable gesture generation often utilize a limited number of control parameters and lack the ability to activate/deactivate them selectively. Therefore, in this work, we propose the Cont-Gest model, a Transformer-based gesture generation model that enables selective control activation through masked training and a control fusion strategy. Furthermore, to better support the development of such models, we propose a novel evaluation-driven development (EDD) workflow, which combines several iterative tasks: automatic control signal extraction, control specification, visual (subjective) feedback, and objective evaluation. This workflow enables continuous monitoring of model performance and facilitates iterative refinement through feedback-driven development cycles. For objective evaluation, we are using the validated Kinetic–Hellinger distance, an objective metric that correlates strongly with the human perception of gesture quality. We evaluated multiple model configurations and control dynamics strategies within the proposed workflow. Experimental results show that Feature-wise Linear Modulation (FiLM) conditioning, combined with single-mask training and voice activity scaling, achieves the best balance between gesture quality and adherence to control inputs. |
|---|
| Ključne besede: | gesture generation, objective evaluation, selective control activation, transformers, weakly supervised learning |
|---|
| Status publikacije: | Objavljeno |
|---|
| Verzija publikacije: | Objavljena publikacija |
|---|
| Poslano v recenzijo: | 10.08.2025 |
|---|
| Datum sprejetja članka: | 26.08.2025 |
|---|
| Datum objave: | 28.08.2025 |
|---|
| Založnik: | MDPI |
|---|
| Leto izida: | 2025 |
|---|
| Št. strani: | 21 str. |
|---|
| Številčenje: | Vol. 15, iss. 17, [article no.] 9467 |
|---|
| PID: | 20.500.12556/DKUM-95219  |
|---|
| UDK: | 004.8 |
|---|
| COBISS.SI-ID: | 248164099 |
|---|
| DOI: | 10.3390/app15179467 |
|---|
| ISSN pri članku: | 2076-3417 |
|---|
| Avtorske pravice: | © 2025 by the authors |
|---|
| Datum objave v DKUM: | 09.09.2025 |
|---|
| Število ogledov: | 251 |
|---|
| Število prenosov: | 11 |
|---|
| Metapodatki: |    |
|---|
| Področja: | Ostalo
|
|---|
|
:
|
Kopiraj citat |
|---|
| | | | Skupna ocena: | (0 glasov) |
|---|
| Vaša ocena: | Ocenjevanje je dovoljeno samo prijavljenim uporabnikom. |
|---|
| Objavi na: |  |
|---|
Postavite miškin kazalec na naslov za izpis povzetka. Klik na naslov izpiše
podrobnosti ali sproži prenos. |
