Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

The eyes play important roles in human communication. In this study, a robot tells a user it is watching her/him in a shopping scenario . First, we conducted experiments to determine the parameters of the eyes on the screen of the robot . Next, we conducted a scenario experiment, assuming a shopping scene, to demonstrate the effectiveness of the eye-based interaction compared to common push-type interaction. The results showed that the robot achieved modest and casual interaction in a shopping scene.

DOI： 10.1007/978-3-030-78465-2_15

Web of Science

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2021-2.html#AoyagiSY21

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

DOI： 10.1007/978-3-030-78361-7_33

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2021-5.html#SejimaSW21

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

Pupils give favorable impressions to humans depending on their size. Focusing on effects of the pupil size, various pupil sizes were evaluated using human images as stimulations in previous researches. However, it is difficult to express an extreme pupil size in a face image due to a limited area of eyes. In addition, the evaluation of pupil size itself is hard because impressions in human images are formed depending on not only the pupil size but also other facial parts. In order to evaluate impressions of the pupil size itself, it is desirable to extract only the pupil and exclude other facial parts. In this study, we analyzed impressions on various pupil sizes that combined the dilation / contraction using two kinds of pupil expression media that have been developed so far. In the analysis, the impression evaluation obtained by the SD method was factor-analyzed. As the result of factor analysis, three factors were extracted: acceptability, reliability, and curiosity. Acceptability and reliability factors had a certain peak, and curiosity factor was proportional to the pupil size. These analysis results demonstrated that a certain dilated area of pupil could form a favorable impression regardless of pupil expression media.

DOI： 10.1299/transjsme.21-00187

researchmap

Language：Japanese   Publisher：日本福祉工学会  

CiNii Books

researchmap

Other Link： https://search.jamas.or.jp/link/ui/2021102526

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER INTERNATIONAL PUBLISHING AG  

© 2020, Springer Nature Switzerland AG. Pupil response is closely related to human affects and emotions. Focusing on the pupil response in human- robot interaction, we developed a pupil response interface using hemisphere displays for enhancing affective expression. This interface can generate pupil response like human by speech input and enhance affective expression. In this study, for the basic research of forming an intimate communication between human and pet-robot, we analyzed the pupil response during his or her body contact stroking forearm or head by using a pupil measurement device. Based on the analysis, we developed an advanced pupil response system for enhancing intimacy. This system generates the empathy expression when the talker touches any surface of hemisphere displays. The effectiveness of the system was confirmed experimentally.

DOI： 10.1007/978-3-030-20441-9_11

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ahfe/ahfe2019-1.html#SejimaSW19

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

Human eyes are related to own affects and emotions, and seem to twinkle when human has positive affects and emotions such as a strong interest and eagerness. Human is subconsciously attracted to the twinkling eyes. In this study, aiming at the development of a novel communication interface that attracts human, we focused on the reflected glare on the surface of eyeballs and proposed an expression method that superimposes human shadows on pupils. We conducted an impression experiment to evaluate the proposed method and demonstrated that the method was effective for attracting human. Based on the result, we developed an advanced interface that expresses twinkling eyes for attracting human. The developed interface generates a pseudo reflected glare by superimposing human shadow on each pupil as a pseudo-self-image.

DOI： 10.1007/978-3-030-60117-1_20

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2020-42.html#SejimaNW20

Language：English   Publishing type：Research paper (scientific journal)   Publisher：JAPAN SOC MECHANICAL ENGINEERS  

Pupil response plays an important role in expression of talker's affect in an embodied interaction and communication. Focusing on the pupil response in human voice communication, we analyzed the pupil response during utterance, and demonstrated that the pupil enlarges and contracts in synchronization with the burst-pause (ON-OFF) of the utterance. In addition, it was confirmed that the pupil response is effective for enhancing affective conveyance by using the developed system in which an interactive CG character generates the pupil response based on the synchronization with the burst-pause of utterance. In this study, we developed a video communication system with a virtual pupil CG superimposed on the partner's pupil for enhancing affective conveyance. This system generates a virtual pupil response in synchronization with the talker's burst-pause of utterance. We performed a communication experiment under the condition that the virtual pupil CG is generated by being synchronized with the talker's burst-pause of utterance. The effectiveness of the system was demonstrated by means of sensory evaluations of 12 pairs of participants in the video communication.

DOI： 10.1299/jamdsm.2020jamdsm0091

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2018-4.html#SejimaMHSW18

researchmap

Language：Japanese   Publisher：Japan Society of Kansei Engineering  

In human face-to-face communication, not only verbal messages but also non-verbal behaviors such as facial expressions, body movements, and gazes are conveyed; these non-verbal behaviors can express human affect. In addition, it leads to sharing of empathy unconsciously in human interaction and enhancing intimacy between humans. In particular, cognitive empathy as well as emotional empathy play an important role in sharing of empathy. Therefore, it is expected to evaluate emotional empathy based on the features of affect for enhancing intimacy. In this study, for the basic research of evaluating emotional empathy, a method that estimates the emotional centroid based on the coordinate system in the Russell’s circumplex model was proposed. In addition, the experiment was conducted to evaluate the proposed method. The results demonstrated that the method has a possibility to estimate the features of affect.

DOI： 10.5057/jjske.tjske-d-18-00069

researchmap

Other Link： https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-16K01560/

Language：English   Publishing type：Research paper (scientific journal)   Publisher：JAPAN SOC MECHANICAL ENGINEERS  

© 2019 The Japan Society of Mechanical Engineers In recent years, video communication systems have found wide application along with cellphones and e-mails for remote communication. Using video communication, partner’s expression and personal space, which are beneficial in providing smooth communication between two remote locations, can be seen. However, it is difficult to perceive depth, since the projected images are only able to display two-dimensional information. Furthermore, during video communication, while user might move around, the view of partner remains unchanged. It is important to see an image including depth information for communication. It is expected that people will be able to communicate smoothly using images including depth information because feelings of shared place are increased by it. Therefore, in this paper, we develop a virtual window communication system by which an image of a remote location through a virtual window is projected based on the head position of the user. In this system, when the positional relationship between the head position of the user and the window changes, the projected image will also change accordingly. Thus, the user can feel as though he or she is looking directly at his or her conversational partner and the surrounding environment through the virtual window. Furthermore, remote pointing is realized using a pointing function in this communication system. This pointing function does not require calibration and can be placed at an arbitrary position using image processing. Then, sensory evaluations were performed to demonstrate the effectiveness of the developed virtual window communication system.

DOI： 10.1299/JAMDSM.2019JAMDSM0030

Web of Science

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2018 IEEE. We developed an expressible pupil response interface using hemispherical displays for enhancing human-robot communication. This interface looks like robot's eyeballs and expresses vivid pupil response by speech input. In particular, the interface can express exaggerated pupil response that human cannot express. In this study, for the basic research of realizing affinitive interaction in human-robot communication, we analyzed the pupil response with affective expression during utterance using a pupil measurement device. On the basis of the analysis result, we developed a speech-driven pupil response system with affective expression using hemispherical displays. This system expresses pupil response with various affective expression by speech input. We carried out an evaluation experiment by using a sensory evaluation with the system acting as the speaker. The result demonstrated that the system with affective expression is effective for enhancing affinitive interaction.

DOI： 10.1109/ROMAN.2018.8525764

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2018.html#SejimaEMSW18

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2018 IEEE. Robots are expected to work in fields involving direct interaction with human. However, in order to work in these fields, the robot has to be accepted favorably by humans. Therefore, these robots need to start communication smoothly. In the previous research, hand-up response motion of robot has been discussed. The effectiveness of this model was demonstrated by experiments using a hand-up robot system that employed the developed model. On the other hand, robots should also be able to generate active hand-up motion in order to promote its embodied interaction with humans. However, request side of hand-up greeting interaction of robot has not been discussed. Therefore, in this paper, a hand-up request motion model was proposed for generation of hand-up greeting with voice greeting with human. Initially, hand-up greeting between mutual human is measured and analyzed. Based on the motion characteristics obtained from the analysis result, a hand-up request motion model in which a robot requests hand-up greeting with voice greeting from human is proposed. The effectiveness of the developed request motion model is demonstrated through sensory evaluation experiments.

DOI： 10.1109/IRC.2018.00048

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/irc/irc2018.html#OtaYJS18

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

© Springer International Publishing AG, part of Springer Nature 2018. Pupil response plays an important role in expression of talker’s affect. Focusing on the pupil response in human voice communication, we analyzed the pupil response in embodied interaction, and demonstrated that the speaker’s pupil was clearly dilated during the burst-pause of utterance. In addition, it was confirmed that the pupil response is effective for enhancing affective conveyance by using the developed system in which an interactive CG character generates the pupil response based on the burst-pause of utterance. In this study, we develop a video communication system with a virtual pupil CG superimposed on the partner’s pupil for enhancing affective conveyance. This system generates a virtual pupil response in synchronization of the talker’s utterance. The effectiveness of the system is demonstrated by means of sensory evaluations of 12 pairs of subjects in video communication.

DOI： 10.1007/978-3-319-92043-6_28

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2018-4.html#SejimaMHSW18

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2017 IEEE. We have developed a pupil response robot called, Pupiloid, that generates pupil responses that are closely related to human emotions as well as gaze. Pupiloid can express human-like pupil response by using a mechanism that rotates feathers sterically. In this study, in order to create a smooth interaction between human and robot, we performed an analysis of pupil response during utterance by using a pupil measurement device. Based on the results, we propose a method in which the robot's pupil is dilated by being synchronized with the burst-pause of utterance. Then, we developed an advanced communication robot that was used with the Pupiloid in order to enhance the robot's affect during utterance. This advanced robot generates a vivid pupil response via mechanical structures based on the burst-pause of utterance. We carried out a sensory evaluation experiment under the condition that the robot speaks. The results demonstrated that the developed robot effectively enhances affect.

DOI： 10.1109/ROMAN.2017.8172339

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2017.html#SejimaEMSW17

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2017 IEEE. In human face-to-face communication, embodied sharing using the synchrony of embodied rhythms is promoted by embodied interactions. Therefore, embodied interactions are important for smoothly initiating coexistence and communication. When there is embodied interactions with direct contact, it plays to synchronize embodied rhythms effectively. Hug behavior is one of the types of embodied interactions that involve direct contact. In this type interactions, humans whole-body contact with each other. In the case of interaction between a human and a robot, robot synchronizes embodied rhythms effectively using hug behaviors. Furthermore, embodied interactions are expected to be promoted by hug behaviors, in which robot actively approaches and requests to the human. Therefore, in this paper, we develop a hug request motion model of robot during active approach to human. At first, mutual hug behaviors between humans are analyzed in an environment with a voice greeting. Then, on the basis of this analysis results, a hug request motion model during active approach to human is developed. This model generates a hug behavior in which a robot active approaches and requests a hug behavior to a human. In addition, we developed a hug robot system that uses the developed hug request motion model. Using this robot system, the effectiveness of the proposed model is demonstrated by a sensory evaluation.

DOI： 10.1109/SMC.2017.8122674

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/smc/smc2017.html#JindaiOYSS17

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2017 The Society of Instrument and Control Engineers - SICE. In order to communicate smoothly, human does not only use verbal information but also use non-verbal information such as nodding. Nodding can be defined as the action of moving the head vertically up-and-down rhythmically. Nodding plays a huge role as a form of non-verbal information to show approval, agreement or understanding during communications. However, a system to detect nodding action where the head motion and voice rhythm are integrated has not been proposed. Therefore, in this paper, we developed a nodding detection using Neural Network based on the communication characteristics of the head motion and voice rhythm. Initially, the voice data of the speaker side and the head motion of the listener side are measured. The human nodding is detected by measured data using NN.

DOI： 10.23919/SICE.2017.8105683

Web of Science

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2017 The Society of Instrument and Control Engineers - SICE. In our previous study, we performed the analysis of pupil response during his or her utterance by using a pupil measurement device and demonstrated that speaker's pupil response dilates with synchronization to the burst-pause of utterance. In addition, we developed a pupil response robot called 'Pupiloid' that generates the pupil response with mechanical structure, and demonstrated that the pupil response is effective to express own affect. In this paper, in order to enhance affective conveyance, we propose a pupil response model synchronized with burst-pause of utterance based on the heat conduction equation in human-robot interaction. This model estimates the degree of affective conveyance and generates the pupil response based on the estimated value.

DOI： 10.23919/SICE.2017.8105598

Web of Science

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2016 IEEE. Laughing response plays an important role in supporting human interaction and communication, and enhances empathy by sharing laughter each other. Therefore, in order to develop communication systems which enhance empathy, it is desired to design the media representation using the pupil response which is related to affective response such as pleasure-unpleasure. In this paper, we aim to enhance empathy during human and robot interaction and communication, and develop a pupil response system for inducing empathy by laughing response using hemispherical display. In addition, we evaluate the pupil response with the laughing response by using the developed system. The results demonstrate that the dilated pupil response with laughing response is effective for enhancing empathy.

DOI： 10.1109/SII.2016.7844051

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2016.html#EgawaSSW16

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2016 IEEE. An increase of half-hearing person caused by progressive aging of society in our country leads to an increase in demand for a digital hearing aid with a DSP. Because of a hard physical limit for battery capacity which stems from its wearing form, the battery life of an existing digital hearing aid comes up to only about few days. In this paper, we proposed an implementation for a bit-serial multiplier for DSP in a hearing aid with high working frequency and low power consumption. To reduce the power consumption associated with clock generation, we use a ring oscillator to dynamically generate clock pulse only in the period of calculation. In addition, we adopt the Booth encoding to reduce the number of partial products in multiplication and reduce the calculation time and power consumption associated with it. We implement the proposed multiplier and show the effectiveness of it through the comparison experiments.

DOI： 10.1109/CANDAR.2016.105

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ic-nc/candar2016.html#OkamotoKYSAS16

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER INTERNATIONAL PUBLISHING AG  

© Springer International Publishing AG 2017. Line-of-sight such as gaze and eye-contract plays an important role to enhance the embodied interaction and communication through avatars. In addition, many gaze models and communication systems with the line-of-sight using avatars have been proposed and developed. However, the gaze behaviors by generating the above-mentioned models are not considered to enhance the embodied interaction such as activated communication, because the models stochastically generate the eyeball movements based on the human gaze behavior. Therefore, we analyzed the interaction between the human gaze behavior and the activated communication by using line-of-sight measurement devices. Then, we proposed an eye gaze model based on the above-mentioned analysis. In this study, we develop an advanced avatar-mediated communication system in which the proposed eye gaze model is applied to speech-driven embodied entrainment characters called “InterActor.” This system generates the avatar’s eyeball movements such as gaze and looking away based on the activated communication, and provides a communication environment wherein the embodied interaction is promoted. The effectiveness of the system is demonstrated by means of sensory evaluations of 24 pairs of subjects involved in avatar-mediated communication.

DOI： 10.1007/978-3-319-58521-5_48

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2017-3.html#SejimaOW17

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>We have already developed an embodied communication system with the pupil response and demonstrated that the pupil response plays an important role in realizing smooth human interaction and communication. Therefore, the pupil response has possibilities to enhance a sharing of empathy and to convey rich affects such as a pleasure with laugh. Hence, in order to develop communication systems which enhance empathy, it is desired to design the media representation of pupil response. In this paper, focusing on the laugh with pleasure emotion as a typical pleasure affect, we analyzed the relation between laugh and pupil response using a pupil measurement device, and developed a pupil response system for inducing empathy by laugh response based on speech input. In addition, we evaluated the pupil response with the laugh by using the developed system. The results demonstrated that the dilated pupil response with the laugh is effective for enhancing empathy.</p>

DOI： 10.1299/transjsme.17-00076

researchmap

Other Link： https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-16K01560/

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>We have already developed a pupil response robot called "Pupiloid" that generates the robot's gaze as well as pupil response which are closely related to human emotions. This robot can express pupil response like human by using the mechanism which rotates feathers sterically. In this study, for the basic research of realizing smooth communication during embodied interaction between human and robot, we perform the analysis of pupil response during his or her utterance by using a pupil measurement device. On the basis of the analysis, we propose a concept in which the robot's pupil is dilated by synchronizing with the burst-pause of utterance. Then, we develop an advanced communication robot that is used with the Pupiloid for enhancing the robot's affects such as enthusiasm in utterance. This advanced robot generates a vivid pupil response via mechanical structures based on the burst-pause of utterance. The effectiveness of the developed robot is demonstrated by a sensory evaluation in the communication experiment for 20 subjects.</p>

DOI： 10.1299/jsmermd.2017.1P2-L03

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computing Machinery, Inc  

Copyright © 2016 ACM. In reality, shadows are usually natural and unintentional. In virtual reality, however, they play an important role in three-dimensional effects and the perceived reality of the virtual space. An avatar's shadow can have interactive effects with the avatar itself in the virtual space. In this study, we develop an embodied avatar system using avatar-shadow color expressions with an interaction-activated communication model. This model is based on the heat conduction equation in heat-transfer engineering, and has been developed to enhance empathy during embodied interaction in avatar-mediated communication. A communication experiment is performed with 12 pairs of participants to confirm the effectiveness of the system. The results of the sensory evaluation show that interaction activation is visualized by changing avatar-shadow color.

DOI： 10.1145/2974804.2980487

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/hai/2016

Language：English   Publisher：Japan Society of Kansei Engineering  

Face color plays an important role in enhancing empathy in human face-to-face communication, and supports recognition of human emotions such as delight and sorrow. However, these effects cannot be applied to people with a color vision deficiency, who differ in color vision compared to people with normal color vision. Therefore, it is essential to investigate differences in the role of face color in perceiving human emotions. This paper focuses on face color in color vision deficiency, and the experiments evaluate the influence of face color on perception in people with normal color vision and a color vision deficiency. In addition, we evaluated differences in emotion perception between people with normal color vision and people with a color vision deficiency. The results demonstrate that orange improves perception of delight.

DOI： 10.5057/jjske.TJSKE-D-15-00056

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：JAPAN SOC MECHANICAL ENGINEERS  

© 2019 The Japan Society of Mechanical Engineers. In human interaction and communication, not only verbal messages but also nonverbal behaviors such as facial expressions, body movements, gazes and pupil responses play an important role in expressions of talker’s affect. These expressions encourage to read the emotional cues and to cause the sharing of embodiment and empathy. We focused on the pupil response which is closely related to human affect, and developed an embodied communication system in which an interactive CG character generates the pupil response as well as communicative actions and movements such as nodding and body movements by speech input. In addition, it was confirmed that the pupil response is effective for supporting the embodied interaction and communication using the developed system. In this paper, in order to realize the smooth interaction between human and robot, we developed a pupil response system using hemispherical displays for enhancing affective conveyance. This system looks like robot’s eyeballs and expresses vivid pupil response by speech input. We carried out a sensory evaluation experiment under the condition that the developed system speaks. The results demonstrated that the system effectively enhances affective conveyance.

DOI： 10.1299/JAMDSM.2019JAMDSM0032

Web of Science

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：CHINA AVIATION INDUSTRY PRESS  

A handshake is an embodied interaction for displaying closeness using physical contact. Therefore, in our previous study, a small-size handshake robot system is developed. This robot system can shake hands with humans, and the generated handshake motion is preferred by humans. In this system, a magnetic sensor is used for measuring positions of hands and joint angles of human. Therefore, it is necessary that receivers of the magnetic sensor are attached to human arm. However, for a natural handshake interaction, hand position and postures of humans should be measured by non-contact. In this paper, we develop a small-size handshake robot system for generation of handshake response motion using non-contact measurements. In this system, Kinect and Leap Motion controller are installed. The human position at long range is measured by Kinect, and the hand position at close range is measured with high accuracy by Leap Motion controller. Furthermore, sensory evaluations are performed to confirm the hand motion which is generated by the robot system using non-contact measurement. From these results, the handshake robot system can generate a handshake response motion using non-contact measurements the same as the previous system. Additionally, the generated response motion is preferred by humans. Thus, the effectiveness of the system is demonstrated.

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：JAPAN SOC MECHANICAL ENGINEERS  

We have developed a speech-driven embodied entrainment character called "InterActor" that has functions of both speaker and listener for supporting human interaction and communication. This character would generate communicative actions and movements such as nodding, body movements, and eyeball movements by using only speech input. In this study, we focus specifically on the pupillary responses related to human emotions in embodied interaction and communication. Pupillary responses in human face-to-face communication are analyzed using an embodied communication system with a line-of-sight measurement device. Based on the analysis results, we enhance the functions of the character and develop an advanced speech-driven embodied entrainment character system for conveying empathy. Using only speech input, this system changes the pupil size of characters based on the analysis. Through sensory evaluation, we perform experiments to determine the effects of the developed system. The results reveal that the system is effective in human interaction and communication.

DOI： 10.1299/mej.15-00314

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：JAPAN SOC MECHANICAL ENGINEERS  

© 2016 The Japan Society of Mechanical Engineers. In human interaction and communication, not only verbal messages but also nonverbal behavior such as nodding and paralanguage are rhythmically related and mutually synchronized among speakers. This synchrony of embodied rhythms unconsciously enhances a sense of unification and causes an interaction-activated communication in which nonverbal behaviors such as body movements and speech activity increase, and the embodied interaction is activated. In this paper, we propose the concept of an estimation model of interaction-activated communication based on the heat conduction equation with the characteristics of precipitous speed fluctuation and develop a model that estimates the degree of interaction-activated communication by using speech input only. Further, we evaluate the developed model in estimating the period of the interaction-activated communication in an avatar-mediated communication. The results demonstrate that the developed model is effective in estimating the interaction-activated communication.

DOI： 10.1299/jamdsm.2016jamdsm0103

Web of Science

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2015 IEEE. We have analyzed the entrainment between a speaker's speech and a listener's nodding in face-to-face communication, and developed iRT (InterRobot Technology) to generate a variety of communicative actions and movements such as nodding and body movements by using a speech input based on the entrainment analysis. In this study, to conduct basic research for realizing smooth communication during embodied interactions between humans and robots, we focus on the pupil response, which is related to human emotions during such interactions. We analyze the pupil response in human face-to-face communication by using an embodied communication system with a line-of-sight measurement device. On the basis of this analysis, using hemispherical displays, we develop an expressible pupil response interface in which the iRT is applied to enhance embodied interaction between humans and robots. This system enables expression of the pupil response by using only speech input. In addition, the effectiveness of the developed system is demonstrated experimentally.

DOI： 10.1109/ROMAN.2015.7333618

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2015.html#SejimaSW15

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Springer Verlag  

© Springer International Publishing Switzerland 2015. We have developed a speech-driven embodied entrainment character called “InterActor” that had functions of both speaker and listener for supporting human interaction and communication. This character would generate communicative actions and movements such as nodding, body movements, and eyeball movements by using only speech input. In this paper, we analyze the pupil response during the face-to-face communication and non-face-to-face communication with the typical users of the character system. On the basis of the analysis results, we enhance the functionalities of the character and develop an advanced speech-driven embodied entrainment character system for expressing the pupil response.

DOI： 10.1007/978-3-319-20618-9_38

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2015-5.html#SejimaSWJ15

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

© 2014 IEEE. In a previous study, we developed an embodied virtual communication system for human interaction analysis by synthesis in avatar-mediated communication and confirmed the close relationship between speech overlap and the period for activating embodied interaction and communication through avatars. In this paper, we propose an interaction-Activated communication model based on the heat conduction equation in heat-Transfer engineering for enhancing empathy between a human and a robot during embodied interaction in avatar-mediated communication. Further, we perform an evaluation experiment to demonstrate the effectiveness of the proposed model in estimating the period of interaction-Activated communication in avatar-mediated communication. Results suggest that the proposed model is effective in estimating interaction-Activated communication.

DOI： 10.1109/ROMAN.2014.6926356

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2014.html#SejimaWJ14

Publishing type：Research paper (international conference proceedings)  

Copyright © IARIA, 2014. We have developed a speech-driven embodied group entrained communication system called "SAKURA" for enabling group interaction and communication. In this system, speech-driven computer-generated (CG) characters called InterActors with functions of both speakers and listeners are entrained as a teacher and some students in a virtual classroom by generating communicative actions and movements. In this study, for enhancing group interaction and communication, we analyze the eyeball movements of a lecturer communicating in a virtual group by using an embodied communication system with a line-of-sight measurement device. On the basis of the analysis results, we propose an eyeball movement model that consists of a saccade model and a model of the lecturer's gaze at the audience, called "group gaze model." The saccade model reveals eyeball movement with a delay of 0.20 s with respect to the lecturer's head movement. A group gaze model reveals the rate of the lecturer's gaze (Center: 60%, Left-side: 27%, Right-side: 13%). Then, we develop an advanced communication system in which the proposed model is used with SAKURA. Using this system, we perform experiments and carry out sensory evaluation for determining the effects of the proposed model. The results reveal that the proposed model is effective for group interaction and communication in the speech-driven embodied group entrainment characters system.

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)  

In this study, we performed an experiment for examining an advanced line-of-sight measurement method in human face-to-face communication to investigate eye-contact measurement method that supports mental health care. In this experiment, an acrylic board was arranged between two talkers as a virtual display, and the gaze points were analyzed in order to examine the effects of the presence of the acrylic board on the talkers. The result demonstrated that the talkers' lines of sight were focused on the acrylic board.

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

In our previous research, we proposed an eyeball movement model that consists of a saccade model and a group gaze model for enhancing group interaction and communication. In this study, in order to evaluate the effects of the proposed model, we develop an advanced communication system in which the proposed model is used with a speech-driven embodied group entrained communication system. The effectiveness of the proposed model is demonstrated for performing the communication0 experiments with a sensory evaluation using the developed system. © 2013 IEEE.

DOI： 10.1109/ISM.2013.99

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ism/ism2013.html#SejimaWJO13

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

DOI： 10.1109/ROMAN.2012.6343893

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2012.html#SejimaWJOZ12

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

In this paper, we propose a model for estimating conversational activity based on the analysis of enhanced embodied interaction, and develop a virtual audience system. The proposed model is applied to a speech-driven embodied entrainment wall picture, which is a part of the virtual audience system, for promoting enhanced embodied interaction. This system generates activated movements based on the estimated value of conversational activity in enhanced interaction and provides a communication environment wherein embodied interaction is promoted by the virtual audience. The effectiveness of the system was demonstrated by means of sensory evaluations and behavioral analysis of 20 pairs of subjects involved in avatar-mediated communication. © 2011 Springer-Verlag.

DOI： 10.1007/978-3-642-21669-5_22

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2011-12.html#SejimaIW11

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

A large network delay is likely to obstruct human interaction in telecommunication systems such as telephony or video conferencing systems. In spite of the extensive investigations that have been carried out on network delays of voice and image data, there have been few studies regarding support for embodied communication under the conditions of network delay. To maintain smooth human interaction, it is important that the various ways in which delay is manifested are understood. We have already developed an embodied virtual communication system that uses an avatar called "VirtualActor," in which speakers who are remotely located from one another can share embodied interaction in the same virtual space. Responses to a questionnaire that was used in a communication experiment confirmed that a fixed 500-ms network delay has no effect on interactions via VirtualActors. In this paper, we propose a method of presenting a speaker's voice and an avatar's motion feedback in the case of a 1.5-s network delay using VirtualActors. We perform two communication experiments under different conditions of network delay. The aim of the first experiment is to examine the effect of a random time delay on the conversation. The second experiment is conducted under the conditions of a free-form conversation that takes place in 5 scenarios - 1 real-time scenario without a network delay and 4 scenarios with network delay that involve a combination of a delay in the talker's voice and in his/her avatar's motion feedback. The subjects consisted of a total of 30 students who worked in 15 pairs and who were familiar with each other. A sensory evaluation shows the effects upon communication of delays in the avatar's motion feedback, from the viewpoint of supporting the interaction. ©2010 IEEE.

DOI： 10.1109/IUCS.2010.5666007

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iucs/iucs2010.html#IshiiSW10

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

We have developed a speech-driven embodied entrainment system called "InterPicture" and have demonstrated the effectiveness of the system using an embodied virtual communication system. InterPicture is an image containing flowers that react to the speech input of talkers. We confirmed the importance of providing a communication environment in which not only avatars but also CG objects placed around the avatars are related to virtual communication. In this study, we have developed an advanced speech-driven embodied entrainment system called "InterWall". This system projects wall picture widely onto the wall surrounding avatars and behaves as a listener by producing nodding and body movements on the basis of the speech input of a talker. Further, a communication experiment has been performed, and the effectiveness of "InterWall" has been demonstrated by carrying out a sensory evaluation and a speech-overlap analysis for 20 pairs of 40 talkers. Copyright 2009 ACM.

DOI： 10.1145/1667780.1667844

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iucs/iucs2009.html#SejimaW09

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We have already developed an embodied virtual communication system for human interaction analysis by synthesis. This system provides two remote talkers with a communication environment in which embodied interaction is shared by Virtual Actors including the talkers themselves through a virtual face-to-face scene. We confirmed the importance of embodied sharing in embodied communication by using the analysis-by-synthesis system. We have also demonstrated the effects of nodding responses for embodied interaction and communication support. In this paper, we develop an embodied virtual communication system with a speech-driven embodied entrainment picture "InterPicture" for supporting virtual communication. The effects of the developed system are demonstrated by a sensory evaluation and speech-overlap analysis in the communication experiment for 20 pairs of talkers.

DOI： 10.1109/ROMAN.2009.5326243

Web of Science

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ro-man/ro-man2009.html#SejimaW09

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE COMPUTER SOC  

In this study, we develop the embodied virtual communication system with the speech-driven nodding response model for the analysis by synthesis of embodied communication. Using the proposed system in embodied virtual communication, we perform experiments and carry out sensory evaluation and voice-motion analysis to demonstrate the effects of nodding responses on a talker's avatar called VirtualActor. The result of the study shows that superimposed nodding responses in a virtual space promote communication. © 2008 IEEE.

DOI： 10.1109/ISUC.2008.71

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iucs/isuc2008.html#SejimaWY08

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

DOI： 10.1299/jsmemecj.2020.s12101

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

Empathy is very important in the enhancement of rapport, and broadly divided into emotional empathy and cognitive empathy. However, there are few approaches to expressing emotional empathy that can be understood intuitively. In this study, we designed and developed a teary-eyed for displaying emotional empathy during human-robot interaction. This robot mimics the human lacrimal structure, and can generate teary eye by controlling the inflow of water. The effectiveness of the teary-eyed was confirmed experimentally.

DOI： 10.1299/jsmermd.2020.1a1-f03

researchmap

Language：Japanese  

CiNii Books

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

Shadows play an important role in the effects of three-dimension and the enhancement of reality in virtual space. On the other hand, addition of color change in human interaction and communication is effective for enhancing expressions such as affects and interactions. Therefore, by changing the color of avatar’s shadows based on the interaction, it is expected to activate and enhance an avatar-mediated interaction and communication. In this study, focusing on the effects of avatar-shadow’s color, we develop an embodied communication system which expresses the change of avatar-shadow color based on an interaction-activated communication model. The model estimates activation levels of embodied communication based on speech input, and the color of avatar-shadow is changed on the basis of the estimated values. A communication experiment is performed with 12 pairs of participants to evaluate the system. The results of the sensory evaluation demonstrate that the color expressions of avatar-shadow are effective for visualizing embodied interaction and communication.

DOI： 10.1299/transjsme.18-00074

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publisher：日本福祉工学会  

CiNii Books

J-GLOBAL

researchmap

Other Link： http://search.jamas.or.jp/link/ui/2018372157

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publisher：電子情報通信学会  

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>Line-of-sight such as gaze and eye-contract plays an important role to enhance the embodied interaction and communication through avatars. In addition, many eyeball movement models and communication systems using avatars with the line-of-sight have been proposed and developed. However, these eyeball movements by generating the above-mentioned models were not considered to enhance the embodied interaction such as activated communication, because the models stochastically generate the eyeball movements based on the human gaze behavior. Therefore, in order to enhance and promote the embodied interaction from the view point of line-of-sight, it is desired to design a line-of-sight interaction which relates to the interaction-activated communication. In this study, we analyze the interaction between human gaze behavior and the interaction-activated communication by using an embodied communication system with line-of-sight measurement devices. Then, we propose a line-of-sight model based on the above-mentioned analysis, and develop an advanced embodied communication system by applying the proposed model to the speech-driven embodied entrainment character. This system generates the eyeball movements such as gaze and looking away based on the proposed model, and provides a communication environment wherein the embodied interaction is promoted. The effectiveness of the system is demonstrated by means of sensory evaluations of 15 pairs of subjects involved in avatar-mediated communication.</p>

DOI： 10.1299/transjsme.16-00114

researchmap

Language：Japanese   Publisher：電子情報通信学会  

CiNii Books

researchmap

Language：Japanese   Publisher：電子情報通信学会  

CiNii Books

researchmap

Language：Japanese   Publisher：Japanese Society for Engineering Education  

DOI： 10.4307/jsee.63.5_122

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already analyzed enhanced interaction through avatars using an embodied virtual communication system and proposed an interaction-activated communication model based on the heat conduction equation. In this paper, we analyzed the interaction between the talker's gaze and the degree of interaction-activated communication in face-to-face communication, by using an embodied communication system with a line-of-sight measurement device. On the basis of this analysis, we proposed a line-of-sight model, which consists of an eyeball delay movement model and a look away model. Then, we developed an advanced communication system in which the proposed model is applied to speech-driven embodied entrainment character for enhancing embodied interaction and communication. This system generates the eyeball movement on the basis of the proposed model by using the speech input alone to realize smooth and enjoyable interaction and communication.

DOI： 10.1299/jsmedsd.2015.25._1110-1_

CiNii Books

researchmap

Language：Japanese   Publisher：Japanese Society for Engineering Education  

Project-based learning (PBL) is effective for teaching students about the engineering design process, as it leads to higher educational effectiveness among lower-grade students. In this paper, we introduced an original PBL program that involved the use of an autonomous mobile robot in a rescue robot contest, and we assessed its educational effectiveness among several categories of students who participated in the contest. The results demonstrated that essential abilities for information engineering could be acquired by participating in the contest, and both the PBL program and contest provided a good educational environment for lower-grade students.

DOI： 10.4307/jsee.63.5_87

researchmap

Language：Japanese   Publisher：日本福祉工学会  

CiNii Books

researchmap

Other Link： http://search.jamas.or.jp/link/ui/2017235199

Language：Japanese   Publisher：The Institute of Image Information and Television Engineers  

A previous study reported that the lecturer's avatar of a remote education communication support system with a group gaze model which, the avatar gazed the audience at a rate of 13% right, 60% center, 27% left in the virtual classroom, could provide effectiveness for group interaction and communication, especially sense of unity. We believe that a reason of the result is explained by an experimental result, lecturer in the real class room gazes longer duration to the left side than to the right side. In this study, we investigated relationship between duration time of gazing and experimental tasks assigned to the participants. Results show that gazing time depends on the experimental tasks, and two tasks brought left and right deviation. Furthermore, we discussed the reason that the group gaze model can improve the sense of unity in the virtual class room.

DOI： 10.11485/itetr.38.10.0_13

CiNii Books

researchmap

Other Link： http://www.lib.yamaguchi-u.ac.jp/yunoca/handle/2014010498

Language：Japanese   Publisher：Japanese Society for Engineering Education  

DOI： 10.20549/jseeja.2014.0_420

CiNii Books

researchmap

Language：Japanese   Publisher：Japan Society for Science Education  

DOI： 10.14935/jssep.37.0_172

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

Previously, we developed an embodied virtual communication system (EVCOS) for human interaction analysis by synthesis, and confirmed the importance of embodied sharing in avatar-mediated communication. In this paper, we analyze human interaction and communication in interaction-activated conversation for supporting avatar-mediated communication by using EVCOS, and we propose a model for estimating conversational activity on the basis of this analysis. Further, we develop an interaction-activated communication support system for enhancing embodied interaction and communication by applying the proposed model to a virtual audience comprising interactive CG objects. In this system, the virtual audience generates entrained nodding responses as well as dynamic movements based on the estimated conversational activity during the interaction-activated communication period. The effectiveness of the system is demonstrated by means of sensory evaluations and behavioral analysis of 20 pairs of subjects involved in avatarmediated communication. © 2013 The Japan Society of Mechanical Engineers.

DOI： 10.1299/kikaic.79.4095

Scopus

researchmap

Language：Japanese   Publisher：Japanese Society for Engineering Education  

DOI： 10.20549/jseeja.2013.0_680

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already developed a speech-driven embodied group entrained communication system called "SAKURA" for activating group interaction and communication in which speech-driven CG characters called InterActors with both functions of speaker and listener by generating communicative actions and motions are entrained to one another as a teacher and some students in the same virtual classroom. In this paper, the eyeball movement of lecturer in virtual group communication is analyzed by using an embodied communication system with a line-of-sight measurement device. On the basis of the analysis, we propose an eyeball movement model, which consists of a saccade model and a model of lecturer's gazing to audiences called "group gaze model." Then, an advanced communication system in which the proposed model is applied to SAKURA is developed for enhancing group interaction and communication. This system generates lecturer's eyeball movement based on the proposed model by using only speech input. By using the system, we perform experiments for the effects of the proposed model by sensory evaluation. The results demonstrate that the system with the model is effective in group interaction and communication. © 2013 The Japan Society of Mechanical Engineers.

DOI： 10.1299/kikaic.79.827

Scopus

CiNii Books

researchmap

Other Link： http://www.lib.yamaguchi-u.ac.jp/yunoca/handle/2013010010

Language：English   Publisher：Japan Society of Kansei Engineering  

We investigated the effects of pre-preferences for CG characters (avatar), gender of CG characters, exposure frequency, and the concordance between CG characters and the background on the impression formed about CG characters. During 8 sessions, 36 participants were shown a succession of CG characters presented against concordant or discordant backgrounds. During sessions 1, 3, 5, and 7, participants evaluated each CG character in terms of post-preference, intelligence, and reliability using a 7-point scale. Session 8 involved a delay condition. The results showed that post-preference evaluation scores in the high pre-preference condition were associated with better first impressions of CG characters, and increased with exposure frequency. However, background conformity influenced the impressions of CG characters when pre-preferences were medium or low. In addition, post-preference and reliability evaluation scores were higher for female than male characters.

DOI： 10.5057/jjske.12.67

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have developed a speech-driven embodied entrainment characters with eyeball movement model, which consists of an eyeball delay movement model and a gaze withdrawal model. This characters can support the embodied interaction and communication in avatar mediated communication and demonstrate that the line-of-sight plays an important role in enhancing the embodied interaction and communication. In this paper, we propose a concept of eye-contact measurement method and develop an eye-contact measurement system using a dichroic mirror m face-to-face communication for enhancing the embodied interaction and communication. In this system, a dichroic mirror which has a function that transmits visible rays, and reflects infrared rays is arranged between two talkers as a virtual display, and the gaze points on the virtual display are estimated. When the estimated gaze points are present m the eye area of opposite talker, an eye-contact is established in face-to-face communication.

DOI： 10.1299/jsmedsd.2013.23._3202-1_

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

Previously, we developed an embodied virtual communication system for human interaction analysis by synthesis in avatar-mediated communication and confirmed that the speech-overlap between talkers is closely related with the period in which embodied interaction and communication is activated through their avatars. In this paper, we propose an interaction-activated communication model based on the heat conduction equation for enhancing embodied interaction in avatar-mediated communication. Further, we perform the evaluation experiment and demonstrate that the proposed model is effective for estimating the period of interaction-activated communication in avatar-mediated communication.

DOI： 10.1299/jsmemecj.2013._S121013-1

CiNii Books

researchmap

Language：Japanese  

CiNii Books

researchmap

Language：Japanese   Publisher：Japanese Society for Engineering Education  

DOI： 10.20549/jseeja.2012.0_676

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already developed a speech-driven embodied group entrained communication system called SAKURA for activating group interaction and communication. In this system, a speech-driven computer graphics (CG) characters called InterActors with functions of both speaker and listener are entrained to one another as a teacher and some students in a virtual classroom by generating communicative actions and movements. In this study, for the basic research of realizing smooth communication during embodied interaction between human and robot, we analyzed the eyeball movements of a lecturer communicating in a virtual group by using an embodied communication system with a line-of-sight measurement device. On the basis of the analysis results, we propose an eyeball movement model that consists of a saccade model and a model of a lecturer's gaze at an audience, called group gaze model. Then, we developed an advanced communication system in which the proposed model was used with SAKURA for enhancing group interaction and communication. This advanced system generates a lecturer's eyeball movement on the basis of the proposed model by using only speech input. We used sensory evaluation in the experiments to determine the effects of the proposed model. The results showed that the system with the proposed model is effective in group interaction and communication. © 2012 IEEE.

DOI： 10.1109/ROMAN.2012.6343893

Scopus

researchmap

Language：Japanese   Publisher：Japan Society for Science Education  

DOI： 10.14935/jssep.36.0_353

CiNii Books

researchmap

Publisher：The Japan Society of Mechanical Engineers  

We have already confirmed the importance of embodied sharing in avatar-mediated communication and have demonstrated the effectiveness of the entrained nodding responses for human interaction and communication support. In this paper, in order to support the avatar-mediated communication, a virtual audience system is developed by using a speech-driven embodied entrainment picture in which interactive CG objects behave as listeners such as nodding responses. This system provides a communication environment in which not only avatars but also the CG objects placed around the avatars are related to embodied communication. By using this system, the effectiveness of the present system in supporting embodied communication has been demonstrated by carrying out a sensory evaluation and a speech-overlap analysis for 20 pairs of 40 talkers. © 2012 The Japan Society of Mechanical Engineers.

DOI： 10.1299/kikaic.78.523

Scopus

CiNii Books

researchmap

Language：Japanese   Publisher：Human Interface Society  

<p>A large network delay is likely to obstruct human interaction in telecommunication systems such as telephony or video conferencing systems. In spite of the extensive investigations that have been carried out on network delays of voice and image data, there have been few studies regarding support for embodied communication under the conditions of network delay. To maintain smooth human interaction, it is important that the various ways in which delay is manifested are understood. We have already developed an embodied virtual communication system that uses an avatar called "VirtualActor," in which speakers who are remotely located from one another can share embodied interaction in the same virtual space. Responses to a questionnaire that was used in a communication experiment confirmed that a fixed 500-ms network delay has no effect on interactions via VirtualActors. In this paper, we propose a method of presenting a speaker's voice and an avatar's motion feedback in the case of a 1.5-s network delay using VirtualActors. We perform two communication experiments under different conditions of network delay. The aim of the first experiment is to examine the effect of a random time delay on the conversation. The second experiment is conducted under the conditions of a free-form conversation that takes place in 5 scenarios-1 real-time scenario without a network delay and 4 scenarios with network delay that involve a combination of a delay in the talker's voice and in his/her avatar's motion feedback. The subjects consisted of a total of 30 students who worked in 15 pairs and who were familiar with each other. A sensory evaluation shows the effects upon communication of delays in the avatar's motion feedback, from the viewpoint of supporting the interaction.</p>

DOI： 10.11184/his.13.1_23

CiNii Books

researchmap

Language：Japanese   Publisher：Japanese Society for Engineering Education  

DOI： 10.20549/jseeja.2011.0_678

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already developed a speech-driven embodied group entrained communication system called "SAKURA" for activating group interaction and communication in which speech-driven CG characters called InterActors with both functions of speaker and listener by generating communicative actions and motions are entrained one another as a teacher and some students in the same virtual classroom. In this paper, we analyze the eyeball movement of lecturer in group communication, by using an embodied communication system with a line-of-sight measurement device. On the basis of this analysis, we propose an eyeball movement model, which consists of an saccade model and a group gaze model. Then, we develop an advanced communication system in which the proposed eyeball movement model is applied to a speech-driven embodied group entrained communication system. The communication system generates the eyeball movement of lecturer on the basis of the model, and it generates the head and body entrained motions of characters by using the speech input alone. The system would be effective for group embodied interaction and communication as well as remote interaction and communication support.

DOI： 10.1299/jsmedsd.2011.21.525

CiNii Books

researchmap

Language：Japanese   Publisher：Japan Society for Science Education  

本研究の最終目標は、地域社会構成員が互いに協働することで、明日の科学技術日本を担う理系人材を効果的に育成する手法を開発することである。ここではまずその第1歩として、山口県下の産学公民が連携した新しい科学教育システムの構築を目的とし、展開した長州科楽維新プロジェクトについて述べる。さらに、このような教育システムの構築、維持に有用な役割を果たす要因について検討する。

DOI： 10.14935/jssep.35.0_265

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

In this paper, we analyze human eyeball movement through their avatars, by using an embodied virtual communication system with a line-of-sight measurement device. On the basis of this analysis, we propose an eyeball movement model, which consists of an eyeball delay movement model and a gaze withdrawal model. Then, we develop an advanced communication system in which the proposed eyeball movement model is applied to speech-driven embodied entrainment characters called InterActors. The communication system generates the eyeball movement on the basis of model, and it generates the head and body entrained motions of InterActors by using the speech input alone. The effectiveness of the proposed eyeball movement model and the developed communication system is demonstrated by means of sensory evaluations in an avatar mediated communication system.

DOI： 10.1299/kikaic.76.340

Scopus

CiNii Books

researchmap

Language：Japanese   Publisher：Japanese Society for Engineering Education  

DOI： 10.20549/jseejaarc.2010.0_528

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have developed an embodied virtual communication system called EVCOS for human interaction analysis by synthesis. In this paper, focusing on the speech-overlap in enhanced interaction, the characteristics are analyzed in the virtual face-to-face communication using EVCOS, and a model estimating conversational activity on the basis of the analysis is proposed. In addition, an enhanced interaction support system is developed by applying the proposed model to an interactive wall as a virtual audience.

DOI： 10.1299/jsmemecjo.2010.4.0_15

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already developed an embodied virtual communication system (EVCOS) for human interaction analysis by synthesis. This system provides two remote talkers with a communication environment in which embodied interaction is shared by VirtualActors (VAs) including the talkers themselves through a virtual face-to-face scene. We confirmed the importance of head movements including nodding in embodied communication by using this system. In this paper, we develop the EVCOS with a speech-driven nodding response model for the analysis by synthesis of embodied communication under the promoted condition of superimposed nodding responses in a virtual space. By using the system, we perform experiments for the effects of superimposed nodding responses by the sensory evaluation and the voice-motion analysis by superimposing nodding responses on VA by the nodding response model. It is found that the cross-correlation between the talker's voice and the listener's head motion in the inconsistently activated condition increases at a significance level of 1% compared to that observed under normal condition. The result demonstrates that the system with the model promotes embodied interaction.

DOI： 10.1299/kikaic.75.2773

Web of Science

Scopus

CiNii Books

researchmap

Language：Japanese   Publisher：ヒュ-マンインタフェ-ス学会  

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

A large network delay would obstruct human interaction using telecommunication systems such as telephony or video conference. Studies on the embodied communication support in the network delay condition have been little undertaken despite considerable investigations of the network delay of voice and image data. It is important that the ways of revealing the delay are discussed for smooth human interaction. We have already developed an embodied virtual communication system using an avatar called "VirtualActor" in which remote talkers can share embodied interaction in the same virtual space. It has been confirmed that the fixed 500ms network delay has no effects on interaction via VirutalActors in the questionnaire in the communication experiment. In this paper, we propose the presentation method of talker's voice and avatar's motion feedback in the 1.5s network delay condition by using VirtualActors. The communication experiment is performed under the conditions of a free conversation by 5 scenes; the real-time scene without the network delay, and the 4 remained network-delay scenes for the combination of the delay of the talker's voice and his/her avatar's motion feedback. The subjects consisted of a total of 15 pairs of 30 students who are familiar with each other. Sensory evaluation shows communication effects of the delay of avatar's motion feedback on interaction support.

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

A large network delay would obstruct human interaction using telecommunication systems such as telephony or video conference. Studies on the embodied communication support in the network delay condition have been little undertaken despite considerable investigations of the network delay of voice and image data. It is important that the ways of revealing the delay are discussed for smooth human interaction. We have already developed an embodied virtual communication system using an avatar called "VirtualActor" in which remote talkers can share embodied interaction in the same virtual space. It has been confirmed that the fixed 500ms network delay has no effects on interaction via VirutalActors in the questionnaire in the communication experiment. In this paper, we propose the presentation method of talker's voice and avatar's motion feedback in the 1.5s network delay condition by using VirtualActors. The communication experiment is performed under the conditions of a free conversation by 5 scenes; the real-time scene without the network delay, and the 4 remained network-delay scenes for the combination of the delay of the talker's voice and his/her avatar's motion feedback. The subjects consisted of a total of 15 pairs of 30 students who are familiar with each other. Sensory evaluation shows communication effects of the delay of avatar's motion feedback on interaction support.

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

A large network delay would obstruct human interaction using telecommunication systems such as telephony or video conference. Studies on the embodied communication support in the network delay condition have been little undertaken despite considerable investigations of the network delay of voice and image data. It is important that the ways of revealing the delay are discussed for smooth human interaction. We have already developed an embodied virtual communication system using an avatar called "VirtualActor" in which remote talkers can share embodied interaction in the same virtual space. It has been confirmed that the fixed 500ms network delay has no effects on interaction via VirutalActors in the questionnaire in the communication experiment. In this paper, we propose the presentation method of talker's voice and avatar's motion feedback in the 1.5s network delay condition by using VirtualActors. The communication experiment is performed under the conditions of a free conversation by 5 scenes; the real-time scene without the network delay, and the 4 remained network-delay scenes for the combination of the delay of the talker's voice and his/her avatar's motion feedback. The subjects consisted of a total of 15 pairs of 30 students who are familiar with each other. Sensory evaluation shows communication effects of the delay of avatar's motion feedback on interaction support.

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already developed an embodied virtual communication system for human interaction analysis by synthesis. This system provides two remote talkers with a communication environment in which embodied interaction is shared by VirtualActors including the talkers themselves through a virtual face-to-face scene. We confirmed the importance of embodied sharing in embodied communication by using the analysis-by-synthesis system. We have also demonstrated the effects of nodding responses for embodied interaction and communication support. In this paper, we develop an embodied virtual communication system with a speech-driven embodied entrainment picture "InterPicture" for supporting virtual communication. The effects of the developed system are demonstrated by a sensory evaluation and speech-overlap analysis in the communication experiment for 20 pairs of talkers. © 2009 IEEE.

DOI： 10.1109/ROMAN.2009.5326243

Web of Science

Scopus

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have already developed an embodied virtual communication system for human interaction analysis by synthesis. This system provides two remote talkers with a communication environment in which embodied interaction is shared by VirtualActors including the talkers themselves through a virtual face-to-face scene. We confirmed the importance of embodied sharing in embodied communication by using the system. We have also demonstrated the effects of nodding responses for embodied interaction and communication support. In this paper, we develop a virtual communication support system using a speech-driven embodied entrainment wall picture focusing on the immersive effect which makes an impression visually. The effectiveness of the system is demonstrated by the sensory evaluation and speech-overlap analysis in the communication experiment for 20 pairs of 40 talkers.

DOI： 10.1299/jsmedsd.2009.19.279

CiNii Books

researchmap

Language：Japanese   Publisher：Information Processing Society of Japan (IPSJ)  

In this paper, an eye-movement model generating eyeball movement based on head movement is proposed on the basis of the analysis of eyeball movement characteristics in an avator communication experiment. An advanced embodied entraiment communication system is developed in which the proposed model is applied to the speech-driven embodied entraiment character InterActor. The effectiveness of the system is demonstrated by the sensory evaluation of an avator communication experiment.

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

DOI： 10.1299/jsmekyushu.2008.115

CiNii Books

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

We have developed the embodied virtual communication system for human interaction analysis by synthesis. In this paper, focusing on line-of-sight effected smoothly communication in face-to-face communication, embodied virtual communication system with line-of-sight information is developed. In this system, body motions and line-of-sight information of the avatar are represented based on the talker's ones which are measured by magnetic sensors and a developed line-of-sight measurement device. The effectiveness of the system is demonstrated by the sensory evaluation of a remote communication experiment.

DOI： 10.1299/jsmemecjo.2007.4.0_217

CiNii Books

researchmap

researchmap

researchmap

＜優秀プレゼンテーション賞＞

researchmap

researchmap

researchmap

Award type：Honored in official journal of a scientific society, scientific journal 

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Country：Japan

researchmap

Country：Japan

researchmap

Country：Japan

researchmap