The Samsung company has presented the Tallk application, a tool for software that facilitates the communication of people with reduced mobility and absence of oral language, such as ALS patients.
It has been developed by the startup Spanish Irisbond, specialized in solutions eye tracking. The app, which uses a tablet’s camera for interpersonal communication, is available in Spanish for certain models of Samsung’s Galaxy Tab. The objective is to facilitate verbal communication for people suffering from this type of pathology and other similar ones, as well as to give them greater independence. It is under study to favor, through some type of agreement and promotion, the access of the system to sick people.
USE OF ARTIFICIAL INTELLIGENCE
The project has had a strong R&D component and with a combination of Artificial Intelligence, facial tracking mechanisms and methodologies of Deep learning Based on neural networks, it has been possible to develop an eye tracking without the need for additional devices and simply by using the integrated camera of the tablet.
The project has gone through different phases, one of algorithm development to perform the eye tracking using the cameras of the Samsung Galaxand and, on the other hand, the application has been developed Tallk, designed to be controlled with the eye. Finally, the results of both phases have been integrated by applying different usability tools, to make it possible to control the application with a glance.
With a size of minimum screen of 8& rdquor; For usability and accessibility reasons, Tallk’s eye-tracking technology uses the tablet’s camera to locate the iris using the user’s facial patterns. Tracking the iris through a standard tablet camera has been a challenge during its development, since, until now, it could only be done through additional infrared-based cameras.
Thanks to different algorithms and Artificial Intelligence, the software translates the movement of the eyes into approximate traces within the screen to access a predictive virtual keyboard and speak the text entered by the user, using the voice configured in the accessibility menu of the tablet. The Irisbond algorithms, together with the modeling and recognition of facial patterns together with applied Artificial Intelligence, capture the movement of the eyes and translate it into movements within a screen. Anyone can operate the virtual keyboard naturally, intuitively and without using their hands; thus creating a new way of relating to their environment through eye tracking technologies. Irisbond carries out a project of collaboration with the Massachusetts Institute of Technology (MIT) to investigate techniques to refine your algorithms.
The use of neural networks to create facial recognition maps that are capable of identifying characteristic points of the face and landmarks; Combined with ocular vectors, it generates models that, conveniently trained with thousands of data, allow effective gaze tracking.
The application also allows you to manage a collection with your favorite phrases, as well as a history of recently used ones, so you can access and activate them at any time. The keyboard allows the user to communicate with Bixby, Artificial Intelligence assistant, in order to use its commands to solve different requests.
The Tallk application is available in Spanish and can be downloaded as totally free in the Galaxy Store and Play Store in Spain. Once downloaded, you just have to configure the application and calibrate the cursor through the camera of the compatible tablet, a list that will gradually expand.