Learning high-level tasks through imitation

This paper presents the cognitive architecture Con-SCIS (Conceptual Space based Cognitive Imitation System), which tightly links low-level data processing with knowledge representation in the context of imitation learning. We use the word imitate to refer to the paradigm of program-level imitation: we are interested in the final effects of actions on objects, and not on the particular kinematic or dynamic properties of the motion. The same architecture is used both to analyze and represent the task to be imitated, and to perform the imitation by generalizing in novel and different circumstances. The implemented experimental scenario is a simplified two-dimensional world populated with vario…

Prosthetic design and prototype development

Abstract In recent years, there has been worldwide interest in improvement of the mobility of people with lower-limb amputations. Despite significant development of new technologies over the last decade, commercial below-knee and above-knee prostheses are still energetically passive devices. However, many locomotive functions, like walking up stairs and slopes, need significant power in the knee and ankle joints. The additional power for carrying out these previously mentioned activities needs to be achieved by means of external energy sources, which should be integral prosthetic components. This chapter presents preliminary investigations and development towards an active robotic prosthesi…

Hydraulic power and control system

Abstract In recent years, there has been worldwide interest in the improvement of the mobility of people with lower-limb amputation. Despite significant developments in new technologies during the last decade, commercial below-knee and above-knee prostheses are still energetically passive devices. However, many locomotive functions, like walking up stairs and slopes, need significant power in the knee and ankle joints. The additional power for doing previously mentioned activities needs to be achieved by means of external energy sources, which should be integral prosthetic components. In this chapter the design and development of the hydraulic power and control system are described.

Human Sensorimotor Communication: A Theory of Signaling in Online Social Interactions

Although the importance of communication is recognized in several disciplines, it is rarely studied in the context of online social interactions and joint actions. During online joint actions, language and gesture are often insufficient and humans typically use non-verbal, sensorimotor forms of communication to send coordination signals. For example, when playing volleyball, an athlete can exaggerate her movements to signal her intentions to her teammates (say, a pass to the right) or to feint an adversary. Similarly, a person who is transporting a table together with a co-actor can push the table in a certain direction to signal where and when he intends to place it. Other examples of ``si…

Simulating music with associative self-organizing maps

Abstract We present an architecture able to recognise pitches and to internally simulate likely continuations of partially heard melodies. Our architecture consists of a novel version of the Associative Self-Organizing Map (A-SOM) with generalized ancillary connections. We tested the performance of our architecture with melodies from a publicly available database containing 370 Bach chorale melodies. The results showed that the architecture could learn to represent and perfectly simulate the remaining 20% of three different interrupted melodies when using a context length of 8 centres of activity in the A-SOM. These promising and encouraging results show that our architecture offers somethi…

Content-Based Image Retrieval as Validation for Defect Detection in Old Photos

Learning problem solving skills from demonstration: An architectural approach

We present an architectural approach to learning problem solving skills from demonstration, using internal models to represent problem-solving operational knowledge. Internal forward and inverse models are initially learned through active interaction with the environment, and then enhanced and finessed by observing expert teachers. While a single internal model is capable of solving a single goal-oriented task, it is their sequence that enables the system to hierarchically solve more complex task. Activation of models is goal-driven, and internal ”mental” simulations are used to predict and anticipate future rewards and perils and to make decisions accordingly. In this approach intelligent …

A System for Simultaneous People Tracking and Posture Recognition in the context of Human-Computer Interaction

The paper deals with an artificial-vision based system for simultaneous people tracking and posture recognition In the context of human-computer Interaction. We adopt no particular assumptions on the movement of a person and on Its appearance, making the system suitable to several real-world applications. The system can be roughly subdivided Into two highly correlated phases: tracking and recognition. The tracking phase Is concerned with establishing coherent relations of the same subject between frames. We adopted the Condensation algorithm due to Its robustness In highly cluttered environments. The recognition phase adopts a modified elgenspace technique In order to classify between sever…

Texture Synthesis for Digital Restoration in the Bit-Plane Representation

In this paper we propose a new approach to handle the problem of restoration of grayscale textured images. The purpose is to recovery missing data of a damaged area. The key point is to decompose an image in its bit-planes, and to process bits rather than pixels. We propose two texture synthesis methods for restoration. The first one is a random generation process, based on the conditional probability of bits in the bit-planes. It is designed for images with stochastic textures. The second one is a best-matching method, running on each bit-plane, that is well suited to synthesize periodic patterns. Results are compared with a state-of-the-art restoration algorithm.

A probabilistic approach to learning a visually grounded language model through human-robot interaction

A Language is among the most fascinating and complex cognitive activities that develops rapidly since the early months of infants' life. The aim of the present work is to provide a humanoid robot with cognitive, perceptual and motor skills fundamental for the acquisition of a rudimentary form of language. We present a novel probabilistic model, inspired by the findings in cognitive sciences, able to associate spoken words with their perceptually grounded meanings. The main focus is set on acquiring the meaning of various perceptual categories (e. g. red, blue, circle, above, etc.), rather than specific world entities (e. g. an apple, a toy, etc.). Our probabilistic model is based on a varia…

Action simulation in the human brain: Twelve questions

Although the idea of action simulation is nowadays popular in cognitive science, neuroscience and robotics, many aspects of the simulative processes remain unclear from empirical, computational, and neural perspectives. In the first part of the article, we provide a critical review and assessment of action simulation theories advanced so far in the wider literature of embodied and motor cognition. We focus our analysis on twelve key questions, and discuss them in the context of human and (occasionally) primate studies. In the second part of the article, we describe an integrative neuro-computational account of action simulation, which links the neural substrate (as revealed in neuroimaging …

Dynamics-based action recognition for motor intention prediction

Abstract Powered lower-limb prostheses presented in the previous chapter require a natural and easy-to-use interface for communicating amputee’s motor intention in order to select the appropriate motor program in a given context or simply to commute from an active (powered) to a passive mode of functioning. To be accepted by amputees, such an interface should (1) not put additional cognitive load on the end-user, (2) be reliable and (3) be minimally invasive. In this chapter we present one possible solution for achieving that goal: a robust method for autonomously detecting and recognizing motor intents from a wearable sensor network mounted on a sound leg. The sensor network provides a rea…

An adaptive probabilistic approach to goal-level imitation learning

Imitation learning has been recognized as a promising technique to teach robots advanced skills. It is based on the idea that robots could learn new behaviors by observing and imitating the behaviors of other skilled actors. We propose an adaptive probabilistic graphical model which copes with three core issues of any imitative behavior: observation, representation and reproduction of skills. Our model, Growing Hierarchical Dynamic Bayesian Network (GHDBN), is hierarchical (i.e. able to characterize structured behaviors at different levels of abstraction), and growing (i.e. skills are learned or updated incrementally - and at each level of abstraction - every time a new observation sequence…

Recognizing actions with the associative self-organizing map

When artificial agents interact and cooperate with other agents, either human or artificial, they need to recognize others’ actions and infer their hidden intentions from the sole observation of their surface level movements. Indeed, action and intention understanding in humans is believed to facilitate a number of social interactions and is supported by a complex neural substrate (i.e. the mirror neuron system). Implementation of such mechanisms in artificial agents would pave the route to the development of a vast range of advanced cognitive abilities, such as social interaction, adaptation, and learning by imitation, just to name a few. We present a first step towards a fully-fledged int…

A knowledge based architecture for the virtual restoration of ancient photos

Abstract Historical images are essential documents of the recent past. Nevertheless, time and bad preservation corrupt their physical supports. Digitization can be the solution to extend their “lives”, and digital techniques can be used to recover lost information. This task is often difficult and time-consuming, if commercial restoration tools are used for the purpose. A new solution is proposed to help non-expert users in restoring their damaged photos. First, we defined a dual taxonomy for the defects in printed and digitized photos. We represented our restoration domain with an ontology and we created some rules to suggest actions to perform in case of some specific events. Classes and …

A System for simultaneous People Tracking and Posture Recognition in the context of Human-Robot Interaction

Sing with the Telenoid

We introduce a novel research proposal project aimed to build a robotic setup in which the Telenoid learns to improvise jazz singing in a duet with a human singer. In the proposed application, the Telenoid acts in teleoperated mode during the learning phase, while it becomes more and more autonomous during the working phase. A goal of the research is to investigate the essence of human communication which is based on gestures and prosody. We will employ an architecture for imitation learning that incrementally learns from demonstrations sequences of internal model activations, based on the idea of coupled forward- inverse internal models for representing musical phrases and the body sequenc…

Bounded Seed-AGI

Four principal features of autonomous control systems are left both unaddressed and unaddressable by present-day engineering methodologies: (1) The ability to operate effectively in environments that are only partially known at design time; (2) A level of generality that allows a system to re-assess and re-define the fulfillment of its mission in light of unexpected constraints or other unforeseen changes in the environment; (3) The ability to operate effectively in environments of significant complexity; and (4) The ability to degrade gracefully—how it can continue striving to achieve its main goals when resources become scarce, or in light of other expected or unexpected constraining fact…

A Neuro-Genetic Approach to Real-Time Visual Grasp Synthesis

Grasping is an essential prerequisite for an agent, either human or robotic, to manipulate various kinds of objects present in the world. It is a fact that we would like robots to have the same skills as we do. However, despite the construction of human-hand-like robotic effectors, much work is still to be done in order to give robots the capability to grasp and manipulate objects. The goal of this work is to automatically perform grasp synthesis of unknown planar objects. In other words, we must compute points on the object's boundary to be reached by the robotic fingers such that the resulting grasp, among infinite possibilities, optimizes some given criteria. The space of possible config…

A cognitive approach to goal-level imitation

Imitation in robotics is seen as a powerful means to reduce the complexity of robot programming. It allows users to instruct robots by simply showing them how to execute a given task. Through imitation robots can learn from their environment and adapt to it just as human newborns do. Despite different facets of imitative behaviours observed in humans and higher primates, imitation in robotics has usually been implemented as a process of copying demonstrated actions onto the movement apparatus of the robot. While the results being reached are impressive, we believe that a shift towards a higher expression of imitation, namely the comprehension of human actions and inference of its intentions…

Anchoring by Imitation Learning in Conceptual Spaces

In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. The purpose of the following discussion is to show how the same conceptual representation can be used both in a bottom-up approach, in order to learn sequences of actions by imitation learning paradigm, and in a top-down approach, in order to anchor the symbolical representations to the perceptu…

A Cognitive Framework for Learning by Imitation

Experimental validation of the prosthetic leg

Abstract Studies of populations whose ability to perform voluntary movements is impaired due to natural reasons (e.g. aging), genetic anomaly (e.g. Down syndrome), trauma (e.g. spinal cord injury, amputation) or illness (e.g. Parkinson’s disease) frequently result in a basic question: are the observed motor patterns, which may be rather different from those seen in the general population, actually abnormal? In other words, should they be corrected? This question is important, not only for understanding the mechanisms of control over normal and disordered movements, but also for assessing the effectiveness of existing therapeutic approaches and providing focus for developing new therapies an…

Multidirectional Scratch Detection and Restoration in Digitized Old Images

Line scratches are common defects in old archived videos, but similar imperfections may occur in printed images, in most cases by reason of improper handling or inaccurate preservation of the support. Once an image is digitized, its defects become part of that image. Many state-of-the-art papers deal with long, thin, vertical lines in old movie frames, by exploiting both spatial and temporal information. In this paper we aim to face with a more challenging and general problem: the analysis of line scratches in still images, regardless of their orientation, color, and shape. We present a detection/restoration method to process this defect.

What should I do next? Using shared representations to solve interaction problems

Studies on how “the social mind” works reveal that cognitive agents engaged in joint actions actively estimate and influence another’s cognitive variables and form shared representations with them. (How) do shared representations enhance coordination? In this paper, we provide a probabilistic model of joint action that emphasizes how shared representations help solving interaction problems. We focus on two aspects of the model. First, we discuss how shared representations permit to coordinate at the level of cognitive variables (beliefs, intentions, and actions) and determine a coherent unfolding of action execution and predictive processes in the brains of two agents. Second, we discuss th…

Real-Time Visual Grasp Synthesis Using Genetic Algorithms and Neural Networks

This paper addresses the problem of automatic grasp synthesis of unknown planar objects. In other words, we must compute points on the object’s boundary to be reached by the robotic fingers such that the resulting grasp, among infinite possibilities, optimizes some given criteria. Objects to be grasped are represented as superellipses, a family of deformable 2D parametric functions. They can model a large variety of shapes occurring often in practice by changing a small number of parameters. The space of possible grasp configurations is analyzed using genetic algorithms. Several quality criteria from existing literature together with kinematical and mechanical considerations are considered.…

Imitation Learning and Anchoring through Conceptual Spaces

In order to have a robotic system able to effectively learn by imitation and not merely reproduce the movements of a human teacher, the system should have the capability to deeply understand the perceived actions to be imitated. This paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. The purpose of the following discussion is to show how the same conceptual representation can be used both in a bottom-up approach, in order to learn sequences of actions by imitation learning paradigm, and in a top-down approach, in order to anchor the symbolical representations to the perceptual act…

Hierarchies of Self-Organizing Maps for action recognition

We propose a hierarchical neural architecture able to recognise observed human actions. Each layer in the architecture represents increasingly complex human activity features. The first layer consists of a SOM which performs dimensionality reduction and clustering of the feature space. It represents the dynamics of the stream of posture frames in action sequences as activity trajectories over time. The second layer in the hierarchy consists of another SOM which clusters the activity trajectories of the first-layer SOM and learns to represent action prototypes. The third - and last - layer of the hierarchy consists of a neural network that learns to label action prototypes of the second-laye…

Intentional strategies that make co-actors more predictable: The case of signaling

AbstractPickering & Garrod (P&G) explain dialogue dynamics in terms of forward modeling and prediction-by-simulation mechanisms. Their theory dissolves a strict segregation between production and comprehension processes, and it links dialogue to action-based theories of joint action. We propose that the theory can also incorporate intentional strategies that increase communicative success: for example, signaling strategies that help remaining predictable and forming common ground.

Simulating Actions with the Associative Self-Organizing Map

We present a system that can learn to represent actions as well as to internally simulate the likely continuation of their initial parts. The method we propose is based on the Associative Self Organizing Map (A-SOM), a variant of the Self Organizing Map. By emulating the way the human brain is thought to perform pattern recognition tasks, the A- SOM learns to associate its activity with di erent inputs over time, where inputs are observations of other's actions. Once the A-SOM has learnt to recognize actions, it uses this learning to predict the continuation of an observed initial movement of an agent, in this way reading its intentions. We evaluate the system's ability to simulate actions …

Learning high-level manipulative tasks through imitation

This paper presents ConSCIS, Conceptual Space based Cognitive Imitation System, which tightly links low-level data processing with knowledge representation in the context of robot imitation. Our focus is on the program-level imitation: we are interested in the final effects of actions on objects, and not on the particular kinematic or dynamic properties of the motion. The same architecture is used both to analyze and represent the task to be imitated, and to perform the imitation by generalizing in novel and different circumstances. The implemented experimental scenario is a two dimensional world populated with various objects in which observation/imitation takes place. To validate our appr…

A Knowledge Based Model for Digital Restoration and Enhancement of Images Concerning Archaeological and Monumental Heritage of Mediterranean Coast

Representation, Recognition and Generation of Actions in the Context of Imitation Learning

The paper deals with the development of a cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. We adopt the paradigm of conceptual spaces, in which static and dynamic entities are employed to efficiently organize perceptual data, to recognize positional relations, to learn movements from human demonstration and to generate complex actions by combining and sequencing simpler ones. The aim is to have a robotic system able to effectively learn by imitation and which has the capabilities of deeply understanding the perceived actions to be imitated. Experimentation has been performed on a robotic system composed of a PUMA 20…

An architecture for observational learning and decision making based on internal models

We present a cognitive architecture whose main constituents are allowed to grow through a situated experience in the world. Such an architectural growth is bootstrapped from a minimal initial knowledge and the architecture itself is built around the biologically-inspired notion of internal models. The key idea, supported by findings in cognitive neuroscience, is that the same internal models used in overt goal-directed action execution can be covertly re-enacted in simulation to provide a unifying explanation to a number of apparently unrelated individual and social phenomena, such as state estimation, action and intention understanding, imitation learning and mindreading. Thus, rather than…

Action Recognition based on Hierarchical Self-Organizing Maps

We propose a hierarchical neural architecture able to recognise observed human actions. Each layer in the architecture represents increasingly complex human activity features. The first layer consists of a SOM which performs dimensionality reduction and clustering of the feature space. It represents the dynamics of the stream of posture frames in action sequences as activity trajectories over time. The second layer in the hierarchy consists of another SOM which clusters the activity trajectories of the first-layer SOM and thus it learns to represent action prototypes independent of how long the activity trajectories last. The third layer of the hierarchy consists of a neural network that le…

Resolving ambiguities in a grounded human-robot interaction

In this paper we propose a trainable system that learns grounded language models from examples with a minimum of user intervention and without feedback. We have focused on the acquisition of grounded meanings of spatial and adjective/noun terms. The system has been used to understand and subsequently to generate appropriate natural language descriptions of real objects and to engage in verbal interactions with a human partner. We have also addressed the problem of resolving eventual ambiguities arising during verbal interaction through an information theoretic approach.

A Dual Taxonomy for Defects in Digitized Historical Photos

Old photos may be affected by several types of defects. Manual restorers use their own taxonomy to classify damages by which a photo is affected, in order to apply the proper restoration techniques for a specific defect. Once a photo is digitally acquired, defects become part of the image, and their aspect change. This paper wants to be a first attempt to correlate real defects of printed photos, and digital defects of their digitized versions. A dual taxonomy is proposed, for real and digital defects, and used to classify an image dataset, for a posteriori comparative study. Furthermore, a set of digital features is analyzed for digitized images, to identify which of them could be useful f…

Motor simulation via coupled internal models using sequential Monte Carlo

We describe a generative Bayesian model for action understanding in which inverse-forward internal model pairs are considered 'hypotheses' of plausible action goals that are explored in parallel via an approximate inference mechanism based on sequential Monte Carlo methods. The reenactment of internal model pairs can be considered a form of motor simulation, which supports both perceptual prediction and action understanding at the goal level. However, this procedure is generally considered to be computationally inefficient. We present a model that dynamically reallocates computational resources to more accurate internal models depending on both the available prior information and the predic…

Discriminating and simulating actions with the associative self-organising map

We propose a system able to represent others’ actions as well as to internally simulate their likely continuation from a partial observation. The approach presented here is the first step towards a more ambitious goal of endowing an artificial agent with the ability to recognise and predict others’ intentions. Our approach is based on the associative self-organising map, a variant of the self-organising map capable of learning to associate its activity with different inputs over time, where inputs are processed observations of others’ actions. We have evaluated our system in two different experimental scenarios obtaining promising results: the system demonstrated an ability to learn discrim…

Learning of Actions and Goals through Observation and Imitation

A Cognitive Framework for Imitation Learning

Abstract In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. The purpose of the following discussion is to show how this Conceptual Area can be employed to efficiently organize perceptual data, to learn movement primitives from human demonstration and to generate complex actions by combining and sequencing simpler ones. The proposed architecture ha…

The role of synergies within generative models of action execution and recognition: A computational perspective. Comment on "Grasping synergies: A motor-control approach to the mirror neuron mechanism" by A. D'Ausilio et al

Controlling the body – given its huge number of degrees of freedom – poses severe computational challenges. Mounting evidence suggests that the brain alleviates this problem by exploiting “synergies”, or patterns of muscle activities (and/or movement dynamics and kinematics) that can be combined to control action, rather than controlling individual muscles of joints [1–10]. D’Ausilio et al. [11] explain how this view of motor organization based on synergies can profoundly change the way we interpret studies of action recognition in humans and monkeys, and in particular the controversy on the “granularity” of the mirror neuron system (MNs): whether it encodes either (lower) kinematic aspects…

Restoration of Digitized Damaged Photos using Bit-Plane Slicing

Digital image restoration aims to recover damaged zones of a digital image, using surrounding information. In this paper we propose a novel approach, based on bit-plane slicing decomposition, with the purpose to make information analysis and reconstruction process easy, fast and effective. Tests have been made on digitized damaged old photos to restore several classes of typical defects in old photographic prints.

Human motor system

Abstract This chapter deals with the general issues of motor control and coordination rather than with neurophysiological mechanisms that form the basis for natural, coordinated movements. It is useful that, before we consider the basics of motor behaviours and disorders, we introduce a general theoretical framework adequate to consider issues of control and coordination in biological systems. However, it is impossible to separate issues of control from issues of coordination during natural human movements. Hence, this chapter will also deal with coordination, exploring how individual effectors (such as muscles, joints and limbs) are made to act together in a task-specific way. Ultimately, …

Visually-Grounded Language Model for Human-Robot Interaction

Visually grounded human-robot interaction is recognized to be an essential ingredient of socially intelligent robots, and the integration of vision and language increasingly attracts attention of researchers in diverse fields. However, most systems lack the capability to adapt and expand themselves beyond the preprogrammed set of communicative behaviors. Their linguistic capabilities are still far from being satisfactory which make them unsuitable for real-world applications. In this paper we will present a system in which a robotic agent can learn a grounded language model by actively interacting with a human user. The model is grounded in the sense that meaning of the words is linked to a…

Hankelet-based action classification for motor intention recognition

Powered lower-limb prostheses require a natural, and an easy-to-use, interface for communicating amputee’s motor intention in order to select the appropriate motor program in any given context, or simply to commute from active (powered) to passive mode of functioning. To be widely accepted, such an interface should not put additional cognitive load at the end-user, it should be reliable and minimally invasive. In this paper we present a one such interface based on a robust method for detecting and recognizing motor actions from a low-cost wearable sensor network mounted on a sound leg providing inertial (accelerometer, gyrometer and magnetometer) data in real-time. We assume that the sensor…

The intentional stance as structure learning: a computational perspective on mindreading

Recent theories of mindreading explain the recognition of action, intention, and belief of other agents in terms of generative architectures that model the causal relations between observables (e.g., observed movements) and their hidden causes (e.g., action goals and beliefs). Two kinds of probabilistic generative schemes have been proposed in cognitive science and robotics that link to a "theory theory" and "simulation theory" of mindreading, respectively. The former compares perceived actions to optimal plans derived from rationality principles and conceptual theories of others' minds. The latter reuses one's own internal (inverse and forward) models for action execution to perform a look…

SmartLeg: An intelligent active robotic prosthesis for lower-limb amputees

In recent years, there has been a worldwide interest in improvement of mobility of people with lower limb amputation. In spite of significant development of new technologies during the last decade, commercial below-knee and above-knee prostheses are still energetically passive devices. However, many locomotive functions, like walking up stairs and slopes, need significant power in knee and ankle joints. The additional power for doing previously mentioned activities needs to be achieved by means of external energy sources, which should be integral prosthetic components. This paper presents preliminary investigations towards an active robotic prosthesis that could potentially enable people wi…

Prosthetic modelling and simulation

Abstract Modelling of physical systems can be divided into two categories: physical and mathematical modelling. Physical modelling is a process in which we construct tangible scale models that look very much like the real system. In the past century, consider the animal models that have significantly influenced the development of disease treatment and artificial joints. However, scale models require a great deal of time and resources to develop and there are limits to what can be learned from them. Mathematical or behavioural modelling is a more abstract system used for studying a research question that does not necessarily lend itself to physical modelling. In these models, the system is s…

What Will You Do Next? A Cognitive Model for Understanding Others’ Intentions Based on Shared Representations

Goal-directed action selection is the problem of what to do next in order to progress towards goal achievement. This problem is computationally more complex in case of joint action settings where two or more agents coordinate their actions in space and time to bring about a common goal: actions performed by one agent influence the action possibilities of the other agents, and ultimately the goal achievement. While humans apparently effortlessly engage in complex joint actions, a number of questions remain to be solved to achieve similar performances in artificial agents: How agents represent and understand actions being performed by others? How this understanding influences the choice of ag…

Avoiding accidents at the champagne reception: A study of joint lifting and balancing

Using a lifting and balancing task, we contrasted two alternative views of planning joint actions: one postulating that joint action involves distinct predictions for self and other, the other postulating that joint action involves coordinated plans between the coactors and reuse of bimanual models. We compared compensatory movements required to keep a tray balanced when 2 participants lifted glasses from each other’s trays at the same time (simultaneous joint action) and when they took turns lifting (sequential joint action). Compared with sequential joint action, simultaneous joint action made it easier to keep the tray balanced. Thus, in keeping with the view that bimanual models are reu…

The challenges of prosthetic design and control

Abstract During the last decade, there has been significant interest – both in academia and in industry – in devising technologically advanced solutions for the improvement of mobility of people with a lower-limb amputation. This is partly due to the fact that the number of lower-limb amputees is constantly increasing. The majority of current prosthetic solutions are energetically passive devices, meaning that these devices can only react, while an active one can both act and react. Hence, they are unable to restore full mobility to lower-limb amputees. Many common everyday activities, such as walking up a slope or ascending and descending stairs, require the exertion of large forces and mo…

Sensorimotor Coarticulation in the Execution and Recognition of Intentional Actions

Humans excel at recognizing (or inferring) another's distal intentions, and recent experiments suggest that this may be possible using only subtle kinematic cues elicited during early phases of movement. Still, the cognitive and computational mechanisms underlying the recognition of intentional (sequential) actions are incompletely known and it is unclear whether kinematic cues alone are sufficient for this task, or if it instead requires additional mechanisms (e.g., prior information) that may be more difficult to fully characterize in empirical studies. Here we present a computationally-guided analysis of the execution and recognition of intentional actions that is rooted in theories of m…

You cannot speak and listen at the same time: a probabilistic model of turn-taking.

Turn-taking is a preverbal skill whose mastering constitutes an important precondition for many social interactions and joint actions. However, the cognitive mechanisms supporting turn-taking abilities are still poorly understood. Here, we propose a computational analysis of turn-taking in terms of two general mechanisms supporting joint actions: action prediction (e.g., recognizing the interlocutor's message and predicting the end of turn) and signaling (e.g., modifying one's own speech to make it more predictable and discriminable). We test the hypothesis that in a simulated conversational scenario dyads using these two mechanisms can recognize the utterances of their co-actors faster, wh…

Resilient hexapod robot

In this paper, we present a method of learning desired behaviour of the specific robotic system and transfer of the existing knowledge in the event of partial system failure. Six-legged robot (hexapod) built on top of the Bioloid platform is used for the method verification. We use genetic algorithms to optimize the hexapod's gait, after which we simulate physical damage caused to the robot. The goal of this method is to optimize the gait in accordance with the actual robot morphology, instead of the assumed one. Also, knowledge that was previously gained will be transferred in order to improve the results. Nonstandard genetic algorithm with the specific mixed population is used for this.

The role of synergies within generative models of action execution and recognition: A computational perspective

Controlling the body – given its huge number of degrees of freedom – poses severe computational challenges. Mounting evidence suggests that the brain alleviates this problem by exploiting “synergies”, or patterns of muscle activities (and/or movement dynamics and kinematics) that can be combined to control action, rather than controlling individual muscles of joints [1–10]. D’Ausilio et al. [11] explain how this view of motor organization based on synergies can profoundly change the way we interpret studies of action recognition in humans and monkeys, and in particular the controversy on the “granularity” of the mirror neuron system (MNs): whether it encodes either (lower) kinematic aspects…

A Cognitive Architecture for Robotic Hand Posture Learning

This paper deals with the design and implementation of a visual control of a robotic system composed of a dexterous hand and video camera. The aim of the proposed system is to reproduce the movements of a human hand in order to learn complex manipulation tasks or to interact with the user. A novel algorithm for robust and fast fingertips localization and tracking is presented. A suitable kinematic hand model is adopted to achieve a fast and acceptable solution to an inverse kinematics problem. The system is part of a cognitive architecture for posture learning that integrates the perceptions by a high-level representation of the scene and of the observed actions. The anthropomorphic robotic…

Scratches Removal in Digitised Aerial Photos Concerning Sicilian Territory

In this paper we propose a fast and effective method to detect and restore scratches in aerial photos from a photographic archive concerning Sicilian territory. Scratch removal is a typical problem for old movie films but similar defects can be seen in still images. Our solution is based on a semiautomatic detection process and an unsupervised restoration algorithm. Results are comparable with those obtained with commercial restoration tools.

Simulation and anticipation as tools for coordinating with the future

A key goal in designing an artificial intelligence capable of performing complex tasks is a mechanism that allows it to efficiently choose appropriate and relevant actions in a variety of situations and contexts. Nowhere is this more obvious than in the case of building a general intelligence, where the contextual choice and application of actions must be done in the presence of large numbers of alternatives, both subtly and obviously distinct from each other. We present a framework for action selection based on the concurrent activity of multiple forward and inverse models. A key characteristic of the proposed system is the use of simulation to choose an action: the system continuously sim…

The future of sensorimotor communication research