Google and Facebook are battling to create the ultimate virtual assistant

From flight schedules to email pile ups, the human experience is increasingly subject to data overload. Our agrarian ancestors typically communicated with just a handful of people in a given day, and it was under these largely tribal circumstances that our brains developed their social circuitry. As the raft of information we are responsible for increases exponentially, it’s no exaggeration to say we humans are quickly running up against the limits of our information processing capacity. Enter the AI virtual assistant.

Two press releases this week are indicative of the different ways two of the front runners in the field, Google and Facebook, are attempting to create the virtual assistant of the future. With relatively little fanfare, Google took the lid off a new feature of the Gmail inbox, called Smart Reply. The aim of Smart Reply is to generate intelligent automated responses to common emails we receive.

Ever since the “Hummingbird” update to Google’s search algorithm back in 2013, an update which essentially allows Google search to understand the meaning behind a user’s query rather than treating it as a blind string of words, Google has been pushing artificial intelligence in higher and higher levels of semantic understanding. No doubt that much of this progress owes to the hire of Ray Kurzweil, one of the early wizards of artificial intelligence and high priest of the singularity movement, whose pioneering work in AI centered on speech recognition.

Google’s approach to artificial intelligence, and hence its virtual assistant, has largely focused on understanding human speech and its meaning. This is only natural, since it is an area directly related to its Google business platform: internet search. Somewhere along the way Google must have realized that every email we receive can be treated as a search query. We typically write to someone because we seek some form of information from them, in essence we are querying their personal storehouse of information. As Google increasingly has access to that personal storehouse of information, it becomes capable of responding to emails just like it would to a search query.

While automated responses to emails may ease some of the mental anxiety we are exposed to on a daily basis, it raises some disturbing philosophical questions. At what point have we ceded too much of our decision making to a machine intelligence, and what happens when the automated reply starts responding to messages what were generated with automated reply? Will entire conversations take place between computers with a minimum of human input?

Facebook software has learned to accurately predict whether precariously balanced virtual blocks will fall down.

Facebook, on the other hand, has taken a different track in regards to developing its virtual assistant. While it is no less interested in natural language processing, a much larger part of its business model is dependent on imagery and visual data, which users upload in ever-increasing droves. It’s no surprise than that Facebook is devoting massive resources to making advances in the field of artificial intelligence related to image processing.

Two developments this week point to how Facebook’s virtual assistant might benefit from enhanced visual cognition. One recent publication relates to the ability of their AI to make physics-based inferences about a visual scene, for instance understanding when a stack of blocks pictured in a photograph is likely to fall over. The other advance relates to scene recognition and developing algorithms that can describe what is taking place in a photo.