Inside the Black Box: How Does a Neural Network Understand Names?

By Kfir Bar, CHIEF SCIENTIST, BASIS TECHNOLOGY

6:30 arrive, register, pizza, network
7:00 ACM intro
7:10 Presentation
Live stream at https://www.youtube.com/watch?

Abstract:
A cornerstone of customer relationship management, chatbot
analytics, and research automation systems, Named Entity Recognition (NER) is a key commercial application of Natural Language Processing (NLP). State of the art approaches to NER are purely data driven, leveraging deep neural networks to identify named entity mentions—such as people, organizations, and locations—in lakes of text data. In this talk, I will present our latest research on NER and provide real-life examples of how we are applying these cutting-edge techniques to ten different languages, including Spanish, English, Arabic, Persian, Korean, and Japanese. We’ll look at accuracy, speed, and memory footprint, while comparing some of the best known deep architectures with a basic statistical approach. I will focus on the interpretation of the network, when assigned to learn names
across many languages.

We’ll start with a detailed description of our neural architecture for NER,
which is based on a generic Long Short-Term Memory (LSTM) implementation, a specific flavor of recurrent neural network for sequence tagging. We encode word as well as letter embeddings as a single neural pipeline. Our decoder is based on Conditional Random Fields (CRF), leveraging label distributions from across the entire input text. We will then look into the internal network activation values, on different input conditions, with a special focus on highly inflected languages. Our latest findings show key neurons that get activated for different linguistic aspects.

Speaker Bio
Kfir is the Chief Scientist of Basis’ text analytics team. He has spent many years working in a wide range of natural language processing disciplines, including statistical machine translation, machine learning, and language generation. Kfir joined Intuview in 2005 as CTO, supporting national security and counter-terrorism missions by deducing authorship, sentiment, intent, and other contextual information. In 2013, he co-founded Comprendi, which transforms big data into actionable marketing insights. Kfir lectures at several academic institutes in Israel where he teaches courses in computer science, digital humanities, machine translation, algorithms, and NLP. Kfir holds a Ph.D. in computer science from Tel Aviv University for a thesis on Semantics and Machine Translation.