Why neural networks aren’t fit for natural language understanding

One of the dominant trends of artificial intelligence in the past decade has been to solve problems by creating ever-larger deep learning models. And nowhere is this trend more evident than in natural language processing, one of the most challenging areas of AI.

In recent years, researchers have shown that adding parameters to neural networks improves their performance on language tasks. However, the fundamental problem of understanding language—the iceberg lying under words and sentences—remains unsolved.

Linguistics for the Age of AI, a book by two scientists at Rensselaer Polytechnic Institute, discusses the shortcomings of current approaches to natural language understanding (NLU) and explores future pathways for developing intelligent agents that can interact with humans without causing frustration or making dumb mistakes.

Marjorie McShane and Sergei Nirenburg, the authors of Linguistics for the Age of AI, argue that AI systems must go beyond manipulating words. In their book, they make the case for NLU systems can understand the world, explain their knowledge to humans, and learn as they explore the world.

Consider the sentence, “I made her duck.” Did the subject of the sentence throw a rock and cause the other person to bend down, or did he cook duck meat for her?

Now consider this one: “Elaine poked the kid with the stick.” Did Elaine use a stick to poke the kid, or did she use her finger to poke the kid, who happened to be holding a stick?

Language is filled with ambiguities. We humans resolve these ambiguities using the context of language. We establish context using cues from the tone of the speaker, previous words and sentences, the general setting of the conversation, and basic knowledge about the world. When our intuitions and knowledge fail, we ask questions. For us, the process of determining context comes easily. But defining the same process in a computable way is easier said than done.

There are generally two ways to address this problem.

Read more: TechTalks

How Computers Parse the Ambiguity of Everyday Language

If you’re one of the 2.4 million Twitter followers of the Hamilton impresario Lin-Manuel Miranda, you’ve come to expect a delightful stream of observations, including tweets capturing conversations with his son Sebastian, now 3 years old. Earlier this month, Miranda offered one such exchange under the title, “S’MORES. A Real-Life One-Act Play.”

Me: So that’s the marshmallow but you’re going to eat it with this graham cracker and chocolate.

[My son looks at me like I am the dumbest person alive.]

Sebastian: No, I’m going to eat it with my MOUTH.

[End of play.]

A charming slice of life, to be sure. But in that brief interaction, young Sebastian Miranda also inadvertently hit upon a kind of ambiguity that reveals a great deal about how people learn and process language—and how we might teach computers to do the same.

The misinterpretation on which the s’mores story hinges is hiding in the humble preposition with. Imagine the many ways one could finish this sentence:

I’m going to eat this marshmallow with …

If you’re in the mood for s’mores, then “graham cracker and chocolate” is an appropriate object of the preposition with. But if you want to split the marshmallow with a friend, you could say you’re going to eat it “with my buddy Charlie.” If you’re only grudgingly consuming that marshmallow, you could say you’re going eat it “with great reluctance.” Or you could say “with my hands” (or “with my mouth” like young Sebastian) if you’re focused on the method of eating.

Somehow speakers of English master these many possible uses of the word with without anyone specifically spelling it out for them. At least that’s the case for native speakers—in a class for English as a foreign language, the teacher likely would tease apart these nuances. But what if you wanted to provide the same linguistic education to a machine?

As it happens, just days after Miranda sent his tweet, computational linguists presented a conference paper exploring exactly why such ambiguous language is challenging for a computer-based system to figure out. The researchers did so using an online game that serves as a handy introduction to some intriguing work currently being done in the field of natural language processing (NLP).

Read more: The Atlantic

Natural language processing will help humans and machines have more empathy

With the current breakneck pace of innovation, it may seem like technology is on an aggressive mission to solve all of humanity’s most pressing problems.

And in some respects, we’re making great progress. We’ve broken tremendous ground in such areas as renewable energy, disease prevention, and disaster recovery. But when it comes to addressing important human-centric challenges — things like workforce diversity, unconscious bias, and employee and customer satisfaction — technology has a history of coming up short.

That’s because technical problems like jet propulsion or GPS are largely math- and physics-related, which is where computers (and programmers) excel. But solving human problems like employee engagement usually requires empathy, and that’s notoriously hard to codify. Humans are emotional creatures, especially when it comes to making decisions. First we feel, then we apply logic to help justify our emotional response, and, finally, we act. Thus, any attempt to help people make better decisions that doesn’t account for emotional factors is almost always destined to fail.

However, with recent advances in artificial intelligence, and especially natural language processing (NLP), we finally have the technological tools for tapping into the power and complexity of human feelings. This approach has major implications for how we design systems, and it’s leading to solutions with more humanistic points of view.

Read more: VentureBeat

How Silicon Valley is teaching language to machines

The dream of building computers or robots that communicate like humans has been with us for many decades now. And if market trends and investment levels are any guide, it’s something we would really like to have. MarketsandMarkets says the natural language processing (NLP) industry will be worth $16.07 billion by 2021, growing at a rate of 16.1 percent, and deep learning is estimated to reach $1.7 billion by 2022, growing at a CAGR of 65.3 percent between 2016 and 2022.

Of course, if you’ve played with any chatbots, you will know that it’s a promise that is yet to be fulfilled. There’s an “uncanny valley” where, at one end, we sense we’re not talking to a real person and, at the other end, the machine just doesn’t “get” what we mean.

For example, when using a fun weather bot like Poncho I may ask, “If I go outside, what should I wear?” The bot responds, “Oops, I didn’t catch that. For things I can help you with, type ‘help’.”

Yet, when I ask, “If I go outside, should I take an umbrella?,” the bot’s almost too-clever response is “Nah, you won’t need your umbrella in Santa Clara, CA.”

Read more: Venture Beat