AI is constantly taking on new challenges, from detecting deepfakes to boosting synthetic biology experiments. But to do this, huge data sets are needed to train artificial intelligence systems.
Such a process is expensive, time consuming and provides information that AI can only do one good thing with.
For example, to train an AI to differentiate between an image of a dog and one of a cat, the system is fed thousands of images labeled with dogs and cats. A child, on the other hand, can see a dog or cat only once or twice and can remember to differentiate them. How can we make AI learn more from the process that children use?
A new way to train AI systems
A team from the University of Waterloo in Ontario tried an answer.
Waterloo team methodology it is based on changing the specificity of the data used to train artificial intelligence systems. Training an AI system to identify a new class of objects using just one example is what they call “one-shot” learning. But I take it a step further, focusing on learning “less than one-shot” or learning LO-shot, in short.
LO-shot learning consists of a system that learns to classify different categories based on a smaller number of examples than the number of categories.
What this means? Let’s say you want to teach an AI to identify dogs, cats and kangaroos. How could this be done without a few clear examples of each animal?
The key, says the Waterloo team, is in what they call soft labels. Unlike rigid tags, which label a data point as belonging to a specific class, soft tags eliminate the relationship or degree of similarity between that data point and multiple classes.
In the case of an AI trained only on dogs and cats, a third class of objects, say kangaroos, could be described as 60% as a dog and 40% as a cat – hypothetically, of course.
“Soft tags can be used to represent learning sets using fewer prototypes than there are classes, resulting in large increases in sample efficiency compared to regular hard label prototypes,” they say. the authors of the paper.
This means that an AI system will be able to recognize a kangaroo without ever seeing one, just telling it that a kangaroo is a cat fraction and a dog fraction – both known images of AI.
How does the system work?
The authors of the paper use a simple machine learning algorithm called k-nearest neighbors (kNN) to explore this idea in more depth.
The algorithm works under the assumption that similar things are most likely to exist next to each other.
To train a kNN algorithm to differentiate between categories, choose specific features to represent each category, such as size or weight, in the case of animals.
With one feature on the x-axis and the other on the y-axis, the algorithm creates a graph in which data points that are similar to each other are grouped next to each other. A line in the center divides the categories and it is quite simple for the algorithm to discern which side of the line the new data points should fall on.
The Waterloo team kept things simple and used the color plots on a 2D chart. Using their colors and locations on the graphics, the team created synthetic datasets and accompanying software labels.
When the team put the algorithm to draw the border lines of the different colors based on these soft tags, it was able to divide the chart into more colors than the number of data points given in the soft tags.
Although the results are encouraging, the team acknowledges that they are only the first step and that there are still many more explorations of this concept.
One idea the team is already working on is for other algorithms to generate soft tags for the algorithm that will be trained using LO-shot; manually designing soft labels will not always be as easy as splitting diagrams into different colors.
The potential of LO-shot to reduce the amount of training data required to produce functional AI systems is promising. In addition to reducing the costs and time required to form new models, the method could make AI more accessible to industries, companies or individuals who do not have access to large data sets – an important step in democratizing AI.