Rustling leaves, a creaking branch: To a mouse, these sensory impressions may at first seem harmless -- but not if a cat suddenly bursts out of the bush. If so, they were clues of impending life-threatening danger. Robert Gütig of the Max Planck Institute of Experimental Medicine in Göttingen has now found how the brain can link sensory perceptions to events occurring after a delay. In a computer model, he has developed a learning procedure in which the model neurons can learn to distinguish between many different stimuli by adjusting their activity to the frequency of the cues. The model even works when there is a time delay between the cue and the event or outcome. Not only is Gütig's learning procedure vital for the survival of every living creature in that it enables them to filter environmental stimuli; it also helps solve a number of technological learning difficulties. One possible application is in the development of speech recognition programs.

 

 

In the animal world, dangers are frequently preceded by warning signs: telltale sounds, movements and odours may be clues of an imminent attack. If a mouse survives an attack by a cat, its future will be brighter if it learns from the failed attempt and reads the clues early next time round. However, mice are constantly bombarded with a vast number of sensorial impressions, most of which are not associated with danger. So how do they know which sounds and odours from their environment presage a cat attack and which do not?

This poses a problem for the mouse's brain. In most cases, the crucial environmental stimuli are temporally dispersed from the actual attack, so the brain must link a clue and the resulting event (e.g. a sound and an attack) even though there is a delay between them. Previous theories have not provided satisfactory explanations as to how the brain bridges the gap between a cue and the associated outcome. Robert Gütig of the Max Planck Institute of Experimental Medicine has discovered how the brain can solve this problem. On the computer, he programmed a neural network that reacts to stimuli in the same way as a cluster of biological cells. This network can learn to filter out the cues that predict a subsequent event.

It depends on the frequency
The network learns by strengthening or weakening specific synapses between the model neurons. The foundation of the computer model is a synaptic learning rule under which individual neurons can increase or decrease their activity in response to a simple learning signal. Gütig has used this learning rule to establish a new learning procedure. "This 'aggregate-label' learning procedure is built on the concept of setting the connections between cells in such a way that the resulting neural activity over a certain period is proportional to the number of cues," explains Gütig. In this way, if a learning signal reflects the occurrence and intensity of certain events in the mouse's environment, the neurons learn to react to the stimuli that predict those events.

However, Gütig's networks can learn to react to environmental stimuli even when no learning signals are available in the environment. They do this by interpreting the average neural activity within a network as a learning signal. Individual neurons learn to react to stimuli that occur in the same numbers as those to which other neurons in the network react. This 'self-supervised' learning follows a principle different to the Hebbian theory that has frequently been applied in artificial neural networks. Hebbian networks learn by strengthening the synapses between neurons that spike at the same time or in quick succession. "In self-supervised learning, it is not necessary for the neural activity to be temporally aligned. The total number of spikes in a given period is the deciding factor for synaptic change," says Gütig. This means that such networks can link sensory clues of different types, e.g. visual, auditory and olfactory, even when there are significant delays between their respective neural representations.

Not only does Gütig's learning procedure explain biological processes; it could also pave the way for far-reaching improvements to technological applications such as automatic speech recognition. "That would facilitate considerable simplification of the training requirements for computer-based speech recognition. Instead of laboriously segmented language databases or complex segmentation algorithms, aggregate-label learning could manage with just the subtitles from newscasts, for example," says Gütig.

Machine learning systems are everywhere. They predict the weather, forecast earthquakes, provide recommendations based on the books and movies we like, and even apply the brakes on our cars when we're not paying attention.

To do this, software programs in these systems calculate predictive relationships from massive amounts of data. The systems identify these predictive relationships using advanced algorithms -- a set of rules for solving math problems -- and "training data." This data is then used to construct the models and features that enable a system to determine the latest best-seller you wish to read or to predict the likelihood of rain next week.

This intricate process means that a piece of raw data often goes through a series of computations in a system. The computations and information derived by the system from that data together form a complex propagation network called the data's "lineage." The term was coined by Yinzhi Cao, an assistant professor of computer science and engineering, and his colleague, Junfeng Yang of Columbia University, who are pioneering a novel approach to make learning systems forget.

Considering how important this concept is to increasing security and protecting privacy, Cao and Yang believe that easy adoption of forgetting systems will be increasingly in demand. The two researchers have developed a way to do it faster and more effectively than can be done using current methods.

Their concept, called "machine unlearning," is so promising that Cao and Yang have been awarded a four-year, $1.2 million National Science Foundation grant to develop the approach.

"Effective forgetting systems must be able to let users specify the data to forget with different levels of granularity," said Cao, a principal investigator on the project. "These systems must remove the data and undo its effects so that all future operations run as if the data never existed."

Increasing security and privacy protection
There are a number of reasons why an individual user or service provider might want a system to forget data and its complete lineage. Privacy is one.

After Facebook changed its privacy policy, many users deleted their accounts and the associated data. The iCloud photo hacking incident in 2014 -- in which hundreds of celebrities' private photos were accessed via Apple's cloud services suite -- led to online articles teaching users how to completely delete iOS photos including the backups. New research has revealed that machine learning models for personalized medicine dosing leak patients' genetic markers. Only a small set of statistics on genetics and diseases are enough for hackers to identify specific individuals, despite cloaking mechanisms.

Naturally, users unhappy with these newfound risks want their data, and its influence on the models and statistics, to be completely forgotten.

Security is another reason. Consider anomaly-based intrusion detection systems used to detect malicious software. In order to positively identify an attack, the system must be taught to recognize normal system activity. Therefore the security of these systems hinges on the model of normal behaviors extracted from the training data. By polluting the training data, attackers pollute the model and compromise security. Once the polluted data is identified, the system must completely forget the data and its lineage in order to regain security.

Widely used learning systems such as Google Search are, for the most part, only able to forget a user's raw data -- and not the data's lineage -- upon request. This is problematic for users who wish to ensure that any trace of unwanted data is removed completely, and it is also a challenge for service providers who have strong incentives to fulfill data removal requests and retain customer trust.

Service providers will increasingly need to be able to remove data and its lineage completely to comply with laws governing user data privacy, such as the "right to be forgotten" ruling issued in 2014 by the European Union's top court. In October 2014, Google removed more than 170,000 links to comply with the ruling, which affirmed users' right to control what appears when their names are searched. In July 2015, Google said it had received more than a quarter-million such requests.

Breaking down dependencies

Building on work that was presented at a 2015 IEEE Symposium and then published, Cao and Yang's "machine unlearning" method is based on the fact that most learning systems can be converted into a form that can be updated incrementally without costly retraining from scratch.

Their approach introduces a layer of a small number of summations between the learning algorithm and the training data to eliminate dependency on each other. So, the learning algorithms depend only on the summations and not on individual data. Using this method, unlearning a piece of data and its lineage no longer requires rebuilding the models and features that predict relationships between pieces of data. Simply recomputing a small number of summations would remove the data and its lineage completely -- and much faster than through retraining the system from scratch.

Cao believes he and Yang are the first to establish the connection between unlearning and the summation form.

And, it works. Cao and Yang tested their unlearning approach on four diverse, real-world systems: LensKit, an open-source recommendation system; Zozzle, a closed-source JavaScript malware detector; an open-source OSN spam filter; and PJScan, an open-source PDF malware detector.

The success of these initial evaluations has set the stage for the next phases of the project, which include adapting the technique to other systems and creating verifiable machine unlearning to statistically test whether unlearning has indeed repaired a system or completely wiped out unwanted data.

In their paper's introduction, Cao and Yang say that "machine unlearning" could play a key role in enhancing security and privacy and in our economic future:

"We foresee easy adoption of forgetting systems because they benefit both users and service providers. With the flexibility to request that systems forget data, users have more control over their data, so they are more willing to share data with the systems. More data also benefit the service providers, because they have more profit opportunities and fewer legal risks.

"We envision forgetting systems playing a crucial role in emerging data markets where users trade data for money, services, or other data because the mechanism of forgetting enables a user to cleanly cancel a data transaction or rent out the use rights of her data without giving up the ownership."