Blog

28. August 2019

Using Meta-Neurons to learn facts from a single training example

When people see a new animal, meet a new person or visit a new place, they don’t need to repeat that experience thousands of times to remember it — so why should computers have to?

Human learning comes in two forms, a fast and a slow one. The slow one requires a lot of repetition which seems to be necessary to conquer a new cognitive field such as learning a new language. But once a field is mastered, learning new facts within this field requires very few, possibly even only one example. It appears, that the brain regions involved in processing this field have been pre wired to the regions they depend on. So once a new fact needs to be learned, this pre wiring is used to speed up the training of the neurons involved in processing this new fact. It is important, that during this training a distinction is made between new and existing knowledge. Otherwise, all connections based on already existing knowledge would be lost due to the repeated learning of something that is already known. So the neurons involved in the processing of an already known fact must therefore be able to suppress the formation of new knowledge. ...

9. August 2019

How to efficiently propagate activations in a massive neural network

When people see a new animal, meet a new person or visit a new place, they don’t need to repeat that experience thousands of times to remember it — so why should computers have to?

In traditional neural networks using the sigmoid activation function, all neurons are more or less activated. There is no clear case of an inactive neuron here. ...

3. August 2019

On adding negative feedback synapses to a neural network

The missing link in deep neural networks

The special thing about adding negative recurrent synapses to a neural network is that they introduce inner states within the network. ...

9. December 2018

On integrating symbolic inference into deep neural networks

What makes biological neural networks so superior to their technical counterparts? Is there anything we have overlooked so far?

Deep neural networks have been a tremendous success story over the last couple of years. Many advances in the field of AI, such as recognizing real world objects, fluently translating natural language or playing GO ...

25. October 2017

Aika: Ein semantisches neuronales Netzwerk

Wenn es darum geht Informationen aus natürlichsprachigen Texten zu extrahieren, stehen einem verschiedene Möglichkeiten zur Verfügung. Eine der ältesten und wohl auch am häufigsten genutzten Möglichkeiten ist die der regulären Ausdrücke. Hier werden exakte Muster definiert und in einem Textstring gematcht. Probleme bereiten diese allerdings, wenn kompliziertere semantische Muster gefunden werden sollen oder wenn verschiedene Muster aufeinander aufbauen oder miteinander interagieren sollen. ...

20. June 2017

Machine-Learning-Bibliothek hilft dabei Texte zu verstehen

Volltextsuche ist ein Kernbestandteil des Internetzeitalters, nichtsdestotrotz lässt sie bis heute viel zu Wünschen übrig. Sie ist ganz hervorragend dazu geeignet, um exakte Worttreffer in einer großen Menge an Dokumenten zu finden. Was allerdings bis heute noch nicht zuverlässig funktioniert ist, ein Wort nur in seiner gewünschten Bedeutung zu finden. Genau hier kann die Java-Bibliothek Aika weiterhelfen. ...