Izločanje značilk pogovornih gest iz video signala za rekonstrukcijo gest in njihovo uporabo v kinematičnem modelu virtualnega pogovornega agenta : doktorska disertacija

Močnik, Grega

| | SLO | ENG | Cookies and privacy

Bigger font | Smaller font

First page > Show document

Show document

Title:	Izločanje značilk pogovornih gest iz video signala za rekonstrukcijo gest in njihovo uporabo v kinematičnem modelu virtualnega pogovornega agenta : doktorska disertacija
Authors:	ID Močnik, Grega (Author) ID Kačič, Zdravko (Mentor) More about this mentor...
Files:	DOK_Mocnik_Grega_2023.pdf (2,61 MB) MD5: A52EA8283B2321D810D1AD415086C1BD
Language:	Slovenian
Work type:	Doctoral dissertation
Typology:	2.08 - Doctoral Dissertation
Organization:	FERI - Faculty of Electrical Engineering and Computer Science
Abstract:	Za poustvarjanje prepričljivih in človekovim podobnih pogovornih odzivov, mora umetna entiteta, tj. utelešeni pogovorni agent, v govorjeni socialni interakciji izražati korelirana govor (besedno) in geste (nebesedno). Večina obstoječih del se osredotoča na načrtovanje namena in načrtovanje vedenja. Realizacija je prepuščena omejenemu naboru statičnih 3D prikazov pogovornih izrazov. Poleg funkcionalne in pomenske sinhronosti med verbalnimi in neverbalnimi signali končno vernost izraza oblikuje fizična realizacija neverbalnih izrazov. Glavni izziv večine pogovornih sistemov, zmožnih reproduciranja gest, je raznolikost izraznosti. V doktorskem delu obravnavamo postopke za samodejno zajemanje gest iz videoposnetkov in njihovo preoblikovanje v 3D prikaze, shranjene kot del repozitorija motoričnih veščin pogovornega agenta. Posebna pozornost je posvečena izločanju značilk z dovolj kakovostno informacijo, da je rekonstruirano pogovorno izražanje naravno in čim vernejše. Predlagan je sistem za rekonstrukcijo pogovornih gest, ki zagotavlja naravnost gest utelešenega pogovornega agenta, posledica česar je bolj kakovostna interakcija med človekom in računalnikom. Predlagan sistem temelji na sledilniku Kanade–Lucas–Tomasi, filtru Savitzky–Golay, kinematičnem modelu na osnovi Denavit–Hartenberg in ogrodju EVA. Obravnavana je ocena sintetiziranega izraznega gibanja in namesto subjektivne ocene sintetiziranega gibanja podana objektivna metoda, temelječa na kosinusni podobnosti. Empirično je predstavljena tudi ocena gibanja, rekonstruiranega s predlaganim sistemom za rekonstrukcijo.
Keywords:	značilke pogovornih gest, rekonstrukcija, virtualni pogovorni agenti, Denavit-Hartenberg, Kanade–Lucas–Tomasi, filter Savitzky–Golay, video
Place of publishing:	Maribor
Place of performance:	Maribor
Publisher:	[G. Močnik]
Year of publishing:	2023
Number of pages:	V, 89 str.
PID:	20.500.12556/DKUM-83800
UDC:	004.934:159.925(043.3)
COBISS.SI-ID:	160865027
Publication date in DKUM:	03.08.2023
Views:	796
Downloads:	138
Metadata:
Categories:	KTFMB - FERI
:	Copy citation

Average score:	(0 votes)
Your score:	Voting is allowed only for logged in users.
Share:

Hover the mouse pointer over a document title to show the abstract or click on the title to get all document metadata.

Licences

License:	CC BY-NC-ND 4.0, Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Link:	http://creativecommons.org/licenses/by-nc-nd/4.0/
Description:	The most restrictive Creative Commons license. This only allows people to download and share the work for no commercial gain and for no other purposes.
Licensing start date:	11.02.2023

Secondary language

Language:	English
Title:	Feature extraction of conversational gestures from video signal for gesture reconstruction and their use in embodied conversational agent kinematic model
Abstract:	To recreate convincing and human-like conversational responses, an artificial entity, i.e. an embodied conversational agent, must express correlated speech (verbal) and gestures (nonverbal) in spoken social interaction. Most of the existing work focuses on intention planning and behaviour planning. Realization is left to a limited set of static 3D representations of conversational expressions. In addition to the functional and semantic synchronicity between verbal and nonverbal signals, the final believability of an expression is shaped by the physical realization of nonverbal expressions. The main challenge of most conversational systems capable of reproducing gestures is the diversity of expressiveness. In this doctoral thesis, we discuss procedures for automatically capturing gestures from videos and transforming them into 3D representations stored as part of the conversational agent's motor skills repository. Special attention is paid to selecting features with sufficient quality information so that the reconstructed conversational expressions are natural and as believable as possible. A conversational gesture reconstruction system is proposed, which ensures the naturalness of the gesture of an embodied conversational agent, resulting in a better quality of human-computer interaction. The proposed system is based on Kanade–Lucas–Tomasi tracker, Savitzky–Golay filter, Denavit–Hartenberg based kinematic model and EVA framework. The evaluation of synthesized expressive movement is discussed, and an objective method based on cosine similarity is given instead of a subjective evaluation of synthesized motion. The motion reconstruction estimation with the proposed reconstruction system is also presented empirically
Keywords:	conversational gesture features, reconstruction, embodied conversational agent, Denavit-Hartenberg, Kanade–Lucas–Tomasi, filter Savitzky–Golay, video

Comments

Leave comment

You must log in to leave a comment.

Comments (0)

0 - 0 / 0

There are no comments!

Back