Horisontaaliset ja vertikaaliset päänliikkeet suomalaisessa viittomakielessä

Tämä pro gradu -tutkimus tarkastelee suomalaisen viittomakielen päänliikkeitä. Päänliikkeet ovat osa viitottujen kielten ei-manuaalista (ei käsillä tuotettua) artikulointia ja niillä on tutkitusti monenlaisia funktioita viittomisessa yksittäisestä viittomasta aina tekstitasolle asti. Päänliikkeiden sisäisiä vaiheita ja päänliiketyyppien välisiä eroja on tutkittu vain vähän, ja foneettisesta näkökulmasta ei juuri lainkaan. Tämä tutkimus selvittää päänliikkeiden foneettista rakennetta, päänliikkeiden foneettis-fonologisia, kieliopillisia ja tekstuaalisia funktioita sekä pään ja käsien liikkeiden välistä suhdetta. Tutkimus on foneettiseen mittaustietoon perustuva ja foneettiseen deskriptioon p…

The more you move, the more action you construct : a motion capture study on head and upper-torso movements in constructed action in Finnish Sign Language narratives

Abstract This paper investigates, with the help of motion capture data processed on corpus principles, the characteristics of head and upper-torso movements in constructed action and regular narration (i.e., signing without constructed action) in FinSL. Specifically, the paper evaluates the validity of two arguments concerning constructed action: that constructed action forms a continuum with regular narration, and that constructed action divides into three subtypes (i.e., overt, reduced, and subtle). The results presented in the paper support the first argument but not directly the second one. Because of the ambiguous position of reduced constructed action in between subtle and overt const…

Head movements in Finnish Sign Language on the basis of Motion Capture data

This paper reports a study of the forms and functions of head movements produced in the dimension of depth in Finnish Sign Language (FinSL). Specifically, the paper describes and analyzes the phonetic forms and prosodic, grammatical, communicative, and textual functions of nods, head thrusts, nodding, and head pulls occurring in FinSL data consisting of a continuous dialogue recorded with motion capture technology. The analysis yields a novel classification of the kinematic characteristics and functional properties of the four types of head movement. However, it also reveals that there is no perfect correspondence between form and function in the head movements investigated.

Creating Corpora of Finland’s Sign Languages

This paper discusses the process of creating corpora of the sign languages used in Finland, Finnish Sign Language (FinSL) and Finland-Swedish Sign Language (FinSSL). It describes the process of getting informants and data, editing and storing the data, the general principles of annotation, and the creation of a web-based lexical database, the FinSL Signbank, developed on the basis of the NGT Signbank, which is a branch of the Auslan Signbank. The corpus project of Finland’s Sign Languages (CFINSL) started in 2014 at the Sign Language Centre of the University of Jyväskylä. Its aim is to collect conversations and narrations from 80 FinSL users and 20 FinSSL users who are living in different p…

The alignment of head nods with syntactic units in Finnish Sign Language and Swedish Sign Language

In this paper we examine the relationship between specific head movement events – head nods, often treated as prosodic boundary markers – and syntactic units in Finnish (FinSL) and Swedish Sign Language (SSL). In the study we investigated the alignment of head nods with syntactic units on the basis of a total of 20 (10+10) FinSL and SSL narratives. The results of the study show that in both languages head nods appeared similarly on syntactic boundaries and that the tendency was to align nods sentence-finally. However, not all head nods behaved this way: for example, a relatively large number of head nods were also found to occur sentence-initially or elsewhere in the sentence. Furthermore, …

Towards understanding nonmanuality : A semiotic treatment of signers’ head movements

This article discusses a certain type of nonmanual action, signers’ head movements, from a semiotic perspective. It presents a typology of head movements and their iconic, indexical and symbolic features based on Peircean and post-Peircean semiotics. The paper argues for the view that (i) indexical strategies are very prominent in head movements, (ii) iconic features are most evident in enacting, while non-enacting description is less common, (iii) symbolic types for tokens are infrequent, although some movements—such as nodding and shaking the head—may become more conventional or schematized, and (iv) different types of head movements involve different proportions of iconicity, indexicalit…

Suomen viittomakielten korpusta rakentamassa

Viittomakielikorpusten rakentaminen on lisääntynyt merkittävästi 2000-luvulla: ensimmäiset korpusprojektit käynnistyivät 2000-luvun alussa Australiassa ja Hollannissa, minkä myötä laajoja, koneluettavia aineistokokoelmia on ryhdytty rakentamaan useissa Euroopan maissa 2010-luvulla. Tässä artikkelissa tarkastellaan Suomen viittomakielten, suomalaisen ja suomenruotsalaisen viittomakielen, korpuksen syntyä. Artikkeli esittelee korpuksen rakennusvaiheita eli aineiston keräämistä, käsittelyä, annotointia, pitkäaikaissäilytystä sekä julkaisua tietosuojakysymyksineen. Lisäksi artikkelissa kuvaillaan, miten korpusaineistoa on käytetty ja voidaan hyödyntää viittomakielten tutkimuksessa sekä opetukse…

Synchronizing eye tracking and optical motion capture : How to bring them together

Both eye tracking and motion capture technologies are nowadays frequently used in human sciences, although both technologies are usually used separately. However, measuring both eye and body movements simultaneously would offer great potential for investigating cross- modal interaction in human (e.g. music and language-related) behavior. Here we combined an Ergoneers Dikablis head mounted eye tracker with a Qualisys Oqus optical motion cap- ture system. In order to synchronize the recordings of both devices, we developed a gener- alizable solution that does not rely on any (cost-intensive) ready-made / company-provided synchronization solution. At the beginning of each recording, the partic…

Miten viittomakielen korpusta luodaan ja mihin sitä tarvitaan? Viittomakielten korpukset ja niiden tehtävät

Artikkeli käsittelee suomalaisen ja suomenruotsalaisen viittomakielen korpusten luontia CFINSL-projektissa (Corpus project of Finland’s sign languages, Suomen viittomakielten korpusprojekti). Viittomakielillä ei ole kirjoitettua muotoa, joten korpusten laatiminen vaatii erilaista lähestymistä kuin korpusten luonti sellaisille puhutuille kielille, joilla on kirjoitettu muoto. Artikkelissa kuvataan ne menetelmät, joilla Jyväskylän yliopiston viittomakielen keskuksessa on koottu aineistoa suomalaisen ja suomenruotsalaisen viittomakielen korpukseen. Lisäksi kuvataan korpusaineiston teknistä käsittelyä, annotointia, metatietojen keruuta ja käsittelyä sekä aineiston säilytystä ja tutkijoiden käyt…

Annotated Video Corpus of FinSL with Kinect and Computer-Vision Data

This paper presents an annotated video corpus of Finnish Sign Language (FinSL) to which has been appended Kinect and computer-vision data. The video material consists of signed retellings of the stories Snowman and Frog, where are you?, elicited from 12 native FinSL signers in a dialogue setting. The recordings were carried out with 6 cameras directed toward the signers from different angles, and 6 signers were also recorded with one Kinect motion and depth sensing input device. All the material has been annotated in ELAN for signs, translations, grammar and prosody. To further facilitate research into FinSL prosody, computer-vision data describing the head movements and the aperture change…

Taking non-manuality into account in collecting and analyzing Finnish Sign Language video data

This paper describes our attention to research into non-manuals when collecting a large body of video data in Finnish Sign Language (FinSL). We will first of all give an overview of the data-collecting process and of the choices that we made in order for the data to be usable in research into non-manual activity (e.g. camera arrangement, video compression, and Kinect technology). Secondly, the paper will outline our plans for the analysis of the non-manual features of this data. We discuss the technological methods we plan to use in our investigation of non-manual features (i.e. computer-vision based methods) and give examples of the type of results that this kind of approach can provide us…

Maisteriksi viittomakielestä : 11 sukellusta suomalaiseen viittomakieleen

On the rhythm of head movements in Finnish and Swedish Sign Language sentences

This paper investigates, with the help of computer-vision technology, the similarities and differences in the rhythm of the movements of the head in sentences in Finnish (FinSL) and Swedish Sign Language (SSL). The results show that the movement of the head in the two languages is often very similar: in both languages, the instances when the movement of the head changes direction were distributed similarly with regard to clause-boundaries, and the contours of the roll (tilting-like) motion of the head during the sentences were similar. Concerning differences, direction changes were found to be used more effectively in the marking of clause-boundaries in FinSL, and in SSL the head moved near…

Variation in the use of constructed action according to discourse type and age in Finnish Sign Language

This paper presents a study of the use of constructed action (CA) in the stories and conversations of adult Finnish Sign Language (FinSL) signers of different ages. CA is defined here as a type of depiction in which a signer enacts the actions, feelings, thoughts and utterances of discourse referents with different parts of their body. Most studies on CA in sign languages have been done on the basis of signed storytelling, and little is known about how the use of CA varies in different discourse types. The use of CA has also been noted to vary between individual signers, but we do not yet know much about the socio-individual phenomena that may be linked to this variation. In the present stu…

The Contribution of Event-Related Potentials to the Understanding of Sign Language Processing and Production in the Brain : Experimental Evidence and Future Directions

Functional neuroimaging allows investigation of the timing properties of the brain mechanisms underlying covert language processing. This paper presents a review of the use of the neuroimaging technique called Event-Related Potentials (ERPs) in sign language (SL) research. In the field of neurolinguistics, ERPs have been widely used in the study of spoken language, but their use in SL is still rare. Studying the neurocognitive aspects of SL could lead to a better understanding of the specific processing of SL in the brain. This review outlines the basic theoretical and methodological principles of ERPs. We focus on three groups of ERPs that are particularly relevant to SL processing and pro…