Proxectos

Description

Este proyecto profundizará en la investigación y el desarrollo de técnicas que permitan allanar el camino hacia una plena comprensión y explotación de los canales de comunicación verbal y no verbal que utilizan la expresión facial, el lenguaje corporal y la expresividad de las manos como sus fuentes de señales. El equipo está formado por miembros del GTM y una persona del grupo UVIGO GRADES (Gramática, Discurso y Sociedad).

La comunicación no verbal es un área interdisciplinaria en la que lingüistas, psicólogos, antropólogos, sociólogos y neurocientíficos desarrollaron una plétora de teorías. Puede definirse como la transferencia e intercambio de mensajes en todas y cada una de las modalidades que no implican palabras. Las modalidades son tan diversas como las expresiones faciales, los gestos y los movimientos corporales, las locuciones no verbales, el comportamiento en el espacio interpersonal e incluso la fisonomía (cara, cuerpo, ropa). Las lenguas de signos se basan principalmente en los gestos de las manos y otras partes del cuerpo, y tienen una gramática visual, por lo que muchas de las técnicas desarrolladas para las aplicaciones de reconocimiento de gestos y del habla se han aprovechado para el reconocimiento de la lengua de signos (RLS). Sin embargo, las expresiones faciales y los movimientos corporales, siendo cruciales para el RLS, no han recibido aún la atención merecida en este escenario.

Por lo tanto, el objetivo global del proyecto es desarrollar nuevos algoritmos, sistemas y conjuntos de datos, basados en el procesamiento del habla y del vídeo, y en técnicas de aprendizaje automático, para extraer información multimodal que permita decodificar los canales de comunicación verbal y no verbal de las lenguas orales y de signos.

Goals

This project pursuits several goals with different R/D/I scope and time horizon for their achievement:

1. Developing facial expression recognition techniques beyond the 6 basic expressions of emotion: compound emotions and communicative non-emotional facial expressions (short-term objective).
2. Design and development of hand-sign recognition algorithms on static images and dynamic video streams (short-term objective).
3. Building a progressively large dataset for Spanish SL and annotation tools to be used within the ELAN package (short-term objective).
4. Design and development of a coding tool to translate trajectories of hands, arms, troncal and head, and facial expressions into linguistically interpretable information in Spanish SL and into prosodic modifiers for ASR (long-term objective).
5. Starting from outputs of Goal 1, design and development of audiovisual techniques for enhanced detection of emotional and linguistic cues useful for a richer closed-caption: questions, laughter, sadness, sobbing, dubitation, turn-taking, etc. (long-term objective).
6. Launching an international benchmarking contest on Spanish SL using the acquired dataset (short-term objective).

Activities

The following activities were planned to achieve the short-term objectives and move towards the long-term objectives. Should you want to delve into details, follow the links.

Activity 1: Understanding and describing nonverbal communication channels.

Activity 2: Compiling a dataset of Spanish SL

Activity 3: Developing facial expression recognition tools in communicative scenarios

Activity 4: Developing a hand-based signing recognition tool

Activity 5: Multimodal fusion and interpretation of nonverbal communication channels 

Activity 6: Result Dissemination

Description

Energy delivery has evolved from closed infrastructures to high complex interconnected systems or smart grids. The interconnection of a broad range of smart devices that constantly assess the state of the grid, the availability of power, and the demand brings new opportunities to increase the efficiency of these ecosystems. However, it also requires from adequate solutions to face potential cyber threats, which usually take advantage of new vulnerabilities. This is, indeed, the main aim of the MAGOS project: to providing energy operators with affordable solutions for monitoring and detecting potential threats in smart grids.

The work will be supported by four main pillars. (1) Gathering and analyzing data from social networks, specialized blogs and sites (OSINT) and also from smart devices connected to the smart grid (such as smart meters). (2) Privacy of data and communications in Smart grids to protect and anonymize users personal data and consumption profiles (applying obfuscation techniques) to avoid user reidentification and leakage of private information. (3) Scalability of the proposed solutions to analyse large and different data sources. In MAGOS will take advantage of the combination of (i) a centered-approach, based on Big Data computing and (ii) a distributed-approach, based on Fog Computing. (4) Secure assessment in smart grids to offer the operators a service to perform vulnerability assessments from the analysis of the gathered data.

Goals

In this MAGOS global framework, the specific objectives of the UVIGO sub-project focus on two main aspects. On the one hand, and related to the first pillar, obtaining and analysing the data,:

• Data Fusion and Interoperability issues. Designing fusion mechanisms to merge data from open sources, like digital social media (social networks, chats, blogs, forums, etc.) together with data from the smart devices connected to the Smart Grid from different perspectives: energy distribution, consumption, energy market, etc.
• Irregularity detection from fusion data. The solution will detect: (i) irregularities on distributed communications in different forums that could be evidence of orchestration behaviours (suspicious communities), based on natural language processing (hate speech, information about smart grid vulnerabilities, etc.), geo-location analysis of the sources and the interaction processes in open media (process mining); and (ii) irregularities on consumption patterns, that could entail potential threats.

On the other hand, and related to the third pillar, we will face the scalability problem of the analysis from a distributed perspective:

• Fog-oriented approach at the user level that aims to move the analysis to the devices that manage the data, to reduce the scalability problems.. Thus, irregularities detection would be done at edge and even intermediate nodes to supplements analytics at upper levels in the cloud.
• Fog-oriented solution based on distributed agreement. Both edge and intermediate nodes should collaborate to perform the irregularities analysis, thus nearby devices will share and compare their results. A distributed consensus solution will be designed in order to allow the interconnected devices to make a decision (which would trigger deeper analysis at upper levels) by using a voting system with reputation metrics

Description

Las redes de comunicaciones inalámbricas avanzan hacia despliegues en los que el número de equipos por unidad de área será al menos un orden de magnitud mayor que el actual. Este proceso de densificación trae consigo un incremento sustancial de la eficiencia espectral y de la capacidad de transmisión de la red, pero también incrementa notablemente el consumo de energía de los dispositivos y de todo el sistema, por causa de la interferencia, el acceso no coordinado o el acoplamiento de los ciclos de uso en los dispositivos.

En este proyecto se están (1) ideando algoritmos y procedimientos nuevos para reducir el consumo de energía en los dispositivos y en las estaciones base de la red, manteniendo la eficiencia espectral; (2) desarrollando modelos matemáticos para estimar la eficiencia energética de los dispositivos y de las redes de comunicaciones, en especial en entornos densos (5G y 5G+) y en comunicaciones D2D, pero también en grandes redes conmutadas Ethernet , como las de los centros de datos; (3) utilizando medidas en tiempo real del estado de la red para optimizar el ciclo de uso de los dispositivos y reducir su consumo de energía, sin afectar al retardo ni a la tasa de forma sensible (networking anticipatorio), mediante técnicas de compresión/estimación estadística y ML.

Description

Secure communication is a fundamental resource for modern society. Each time we make an “online” purchase with our credit cards, conduct financial transactions using Internet banking or transmit medical or genetic data, we should be concerned about secure communication. The security of classical cryptographic solutions is often based on unproven computational assumptions. That is, these solutions are vulnerable to both algorithms and hardware advances and, therefore, they cannot provide long-term security. Indeed, the tremendous progress made recently in the development of a quantum computer seriously jeopardizes the security of all public-key classical cryptography—the most widely-used technique to protect our communications nowadays.

Quantum key distribution (QKD), on the other hand, holds the promise of achieving the Holy Grail of cryptography—absolute security in communications despite the computational capability of an adversary, which is only limited by the laws of Physics—and thus it is a perfect solution to assure long-term security. Unfortunately, however, this statement is only true in theory, and the security of QKD implementations is seriously threatened by quantum hacking. This is so because typical experimental QKD realizations do not satisfy the requirements imposed by the theory to guarantee security. For instance, real devices behave  differently from the theoretical models assumed in the security proofs. Also, security proofs usually consider the existence of a protected space devoid of any unwanted information leakage in which the legitimate parties can privately generate, process and store their classical data. That is, they assume that the QKD systems are free of covert channels and all their optical/electronic devices, as well as their classical post-processing units are honest and located in perfectly shielded labs. However, all these assumptions are very hard (if not impossible) to guarantee in practice. Indeed, the behaviour of real devices is affected by the environmental conditions and can depend on their response to external signals, unawarely triggered by a legitimate user, or maliciously injected into the QKD system by the adversary. Also, the adversary could modify the hardware of the QKD setup, or infect its software with malware, to make it fail at a crucial time, or to hide a backdoor that leaks crucial data to the channel. Note that this type of attacks is a fundamental problem even in conventional cryptographic systems.

Goals

In this ambitious project we aim to relax these unrealistic and hardly feasible assumptions and develop security proof techniques which could be used to prove the security of existing practical QKD implementations. That is, the goal is to develop a general formalism, which will include typical imperfections of real devices, to prove the security of practical QKD setups when some of its optical/electronic devices and classical post-processing units could be corrupted. Also, here we will evaluate the realistic situation where the optical devices which are honest do not lie inside private spaces inaccessible to the adversary but they could leak part of its internal information to the channel. Importantly, the results of this project would apply to conventional QKD, “device-independent” QKD and “measurement-device- independent” QKD. The achievement of these goals would represent a fundamental step forward in bridging the gap between the theory and practice of QKD.

Description

The connected vehicle is already a reality in the market, with a process of vertiginous growth that opens up a whole set of new services, among which the autonomous driving must be mentioned as the most disruptive.

In this scenario, the satellite is postulated as the ideal platform to provide Over The Air Services (SOTA), such as video broadcasting or remote updating of software for infotainment units (road conditions, news, etc.) of the vehicles. Precisely for this type of applications, the satellite is more competitive and efficient than the terrestrial networks (3G / 4G), since from the satellite a single file of update for all the vehicles is sent (broadcast transmission), whereas with the networks a file is sent for each vehicle.

In addition, the massive use of “Over The Air” software upgrade systems requires highly available and low cost networks, so satellite communications using the Ka band offer great potential.

Goals

The main objective of the Sat2Car project is to develop and validate a comprehensive system of mobile services and applications (Over Air Services) in vehicles through a communications satellite operating in the Ka band that allows managing, queuing, transmitting, validating, and deploy software updates remotely over a fleet of vehicles.

Description

In TraceThem, we will carry out research in algorithmic techniques of multimedia and multilingual search, working in real environments where current techniques fail in performance, generalisation and scalability. Apart from the traditional audiovisual contents (TV shows, news, movies, series...) new scenarios and types of contents have emerged in recent years (MOOCs, video blogs, tutorials...) where the automation of the search process for accessing the contents is a key aspect; processing these multimedia documents involves the added difficulty that, often, contents appear in different languages, representing a higher technological challenge, as tools adapted to different languages are needed, which is not always possible due to the lack of resources or tools to enable completely language independent content indexing. The information that we intend to extract is always within a communicative context (“from” someone and “for” someone), so the characterisation of the people involved in this context will play a central role. We will focus on finding information about people and their way of interacting (“who they are”, “what they say”, “how they communicate”, “how they are doing”), with a special interest in discovering people and content. The extraction of information related to people will be performed through audio processing, video processing and combined audio and video processing. To do this, we will focus on searching for technologies and new solutions for: multimedia content analysis, voice and face biometrics, audio segmentation and speaker diarization, detection of the emotional state and detection of people interacting. Content extraction will be primarily performed by processing audio using both language-dependent and language-independent search on speech. The scientific-technical impact and dissemination of project results will be favored by participation in international competitive evaluations related to the described issue, as these are important in the development of this project activity because they allow to use data sets related with tasks that constitute the current technological challenges. Moreover, in these competitions, common experimental frameworks are set up to enhance collaboration with other research groups and to allow comparisons of different algorithms, helping to discover the strengths and weaknesses of algorithms and developed systems.

These are  two video demo of our research on  "Semantic Indexing and Searching in Multimedia Contents" that are related to the TraceThem activities:

• https://www.youtube.com/watch?v=LJxQKF8Fl4E
• https://www.youtube.com/watch?v=hdJgmFxJFrM

Description

SpeechTech4All es un proyecto concedido por el Ministerio de Economía y Competitividad en la convocatoria de 2012 del Programa Nacional de Proyectos de Investigación Fundamental. El proyecto tiene una duración de tres años y en el participan las Universidades de Vigo, Politécnica de Cataluña y del País Vasco.  Carmen García Mateo, directora del Grupo de Tecnologías Multimedia del centro AtlantTIC de la Universidad de Vigo será la coordinadora del proyecto .

El proyecto está dedicado a la investigación avanzada en las principales tecnologías del habla (reconocimiento de voz, traducción automática, conversión de texto a voz) en todas las lenguas oficiales habladas en España, al reconocimiento del estado emocional del hablante, y a la construcción de marcos experimentales multimodales (voz y facial) y multilingües (castellano, gallego, catalán, y euskara) que permitan mostrar el trabajo realizado.

Como resultado del proyecto se obtendrán avances de investigación en cada una de las tecnologías mencionadas. Algunos ejemplos de estos avances son la búsqueda de la universalización del servicio de personalización de voces sintéticas, el desarrollo de técnicas de adaptación al dominio en traducción automática, o el desarrollo de sistemas de detección del estado del hablante mediante procesado conjunto de voz y cara. Se prevé que el proyecto participe en campañas de evaluación competitiva, entre ellas las organizadas por la Red Temática en Tecnologías del Habla, y por Interspeech.

 Con el fin de dar visibilidad a los avances logrados en todas las tecnologías, así como de ilustrar el marcado carácter social que se pretende dar al proyecto, se definen dos demostradores:

 1) El primero, que integra la mayor parte de las tecnologías trabajadas, consiste en el subtitulado multilingüe de material audiovisual relacionado con el campo de la educación: documentales, ponencias, seminarios...

 2) El segundo va dirigido a una de las aplicaciones genuinas de las tecnologías del habla, como es dotar de voz a personas que por diferentes motivos presentan un nivel severo de discapacidad oral, usando un sintetizador que se adapte a las características específicas de dicha persona.

Description

En este proyecto nos proponemos realizar un agente conversacional (tambien denominado chatbot por voz o simplemente chatbot) feminista orientado al diagnóstico del deterioro cognitivo leve mediante el análisis de conversaciones del chatbot con pacientes. Entre los fines estarán, por un lado, la definición de una guía de buenas prácticas en el diseño de chatbots por voz para tareas en las que la interacción se realiza con personas con deterioro cognitivo leve. Y por otro, que el diseño del chatbot se realizará de tal forma que evite los sesgos de género.