20.500.12556/DKUM-83800 Izločanje značilk pogovornih gest iz video signala za rekonstrukcijo gest in njihovo uporabo v kinematičnem modelu virtualnega pogovornega agenta doktorska disertacija Feature extraction of conversational gestures from video signal for gesture reconstruction and their use in embodied conversational agent kinematic model Za poustvarjanje prepričljivih in človekovim podobnih pogovornih odzivov, mora umetna entiteta, tj. utelešeni pogovorni agent, v govorjeni socialni interakciji izražati korelirana govor (besedno) in geste (nebesedno). Večina obstoječih del se osredotoča na načrtovanje namena in načrtovanje vedenja. Realizacija je prepuščena omejenemu naboru statičnih 3D prikazov pogovornih izrazov. Poleg funkcionalne in pomenske sinhronosti med verbalnimi in neverbalnimi signali končno vernost izraza oblikuje fizična realizacija neverbalnih izrazov. Glavni izziv večine pogovornih sistemov, zmožnih reproduciranja gest, je raznolikost izraznosti. V doktorskem delu obravnavamo postopke za samodejno zajemanje gest iz videoposnetkov in njihovo preoblikovanje v 3D prikaze, shranjene kot del repozitorija motoričnih veščin pogovornega agenta. Posebna pozornost je posvečena izločanju značilk z dovolj kakovostno informacijo, da je rekonstruirano pogovorno izražanje naravno in čim vernejše. Predlagan je sistem za rekonstrukcijo pogovornih gest, ki zagotavlja naravnost gest utelešenega pogovornega agenta, posledica česar je bolj kakovostna interakcija med človekom in računalnikom. Predlagan sistem temelji na sledilniku Kanade–Lucas–Tomasi, filtru Savitzky–Golay, kinematičnem modelu na osnovi Denavit–Hartenberg in ogrodju EVA. Obravnavana je ocena sintetiziranega izraznega gibanja in namesto subjektivne ocene sintetiziranega gibanja podana objektivna metoda, temelječa na kosinusni podobnosti. Empirično je predstavljena tudi ocena gibanja, rekonstruiranega s predlaganim sistemom za rekonstrukcijo. To recreate convincing and human-like conversational responses, an artificial entity, i.e. an embodied conversational agent, must express correlated speech (verbal) and gestures (nonverbal) in spoken social interaction. Most of the existing work focuses on intention planning and behaviour planning. Realization is left to a limited set of static 3D representations of conversational expressions. In addition to the functional and semantic synchronicity between verbal and nonverbal signals, the final believability of an expression is shaped by the physical realization of nonverbal expressions. The main challenge of most conversational systems capable of reproducing gestures is the diversity of expressiveness. In this doctoral thesis, we discuss procedures for automatically capturing gestures from videos and transforming them into 3D representations stored as part of the conversational agent's motor skills repository. Special attention is paid to selecting features with sufficient quality information so that the reconstructed conversational expressions are natural and as believable as possible. A conversational gesture reconstruction system is proposed, which ensures the naturalness of the gesture of an embodied conversational agent, resulting in a better quality of human-computer interaction. The proposed system is based on Kanade–Lucas–Tomasi tracker, Savitzky–Golay filter, Denavit–Hartenberg based kinematic model and EVA framework. The evaluation of synthesized expressive movement is discussed, and an objective method based on cosine similarity is given instead of a subjective evaluation of synthesized motion. The motion reconstruction estimation with the proposed reconstruction system is also presented empirically značilke pogovornih gest rekonstrukcija virtualni pogovorni agenti Denavit-Hartenberg Kanade–Lucas–Tomasi filter Savitzky–Golay video conversational gesture features reconstruction embodied conversational agent Denavit-Hartenberg Kanade–Lucas–Tomasi filter Savitzky–Golay video true true false [G. Močnik] Slovenski jezik Angleški jezik Doktorsko delo/naloga 2023-02-11 17:50:52 2023-08-03 11:32:45 2023-08-09 03:06:40 0000-00-00 00:00:00 2023 0 Maribor 0 Maribor V, 89 str. 0000-00-00 NiDoloceno NiDoloceno NiDoloceno 0000-00-00 0000-00-00 0000-00-00 004.934:159.925(043.3) 160865027 DOK_Mocnik_Grega_2023.pdf DOK_Mocnik_Grega_2023.pdf 1 A52EA8283B2321D810D1AD415086C1BD 61ab760209f7b9be40d905bea4c3c3f47d3aa2f0309fe49d60862da8be0a9d3e 9fa3fa98-aa2d-11ed-83ce-00155d000105 https://dk.um.si/Dokument.php?lang=slv&id=165669 Fakulteta za elektrotehniko, računalništvo in informatiko 0 0 0