The Death of Visual Analytics and the Dawn of Conversational BI

In the last several years we have seen the emergence of a new breed of business intelligence products that have made it possible to build highly interactive and visually expressive and rich dashboards and reporting experiences. Products like Tableau, Domo, and Looker to name a few are replacing established BI heavyweights with a focus on self-service and rich visualizations.

What is driving this trend? Well anyone not living under a rock for the last then years will tell you that the explosion of data on the internet coupled with the advancement in Big Data related technology have made storing and accessing data much easier than ever before. But this alone is not the whole story.

Self Service BI is Good but not Good Enough

Products like Tableau have come onto the seen to lower the barrier for connecting to internet accessible data sources and as well to traditional sources locked up in relational databases and in the billions of excel spreadsheets sitting around the enterprise world. Driven by this, Tableau, for one, has been successful for three primary reasons:

1. Provides many out of the box data source connectors with an easy to use interface - connect to just about any data source.
2. Self service analytics without some of the heavy lifting - you don't need an army of data and tech experts to model your data and meta-data.
3. Highly compelling and visually rich analytics features - the visualization you can create with Tableau are stunning - not always easy to do, but much more achievable than ever before.

So this is all great, but what does this have to do with the death of visual analytics? I seem to be saying richer BI visualization is blossoming and inspired by tools like Tableau. Well, I will argue that item number three listed above is an evolutionary dead end and that are going to see a gradual trend away from visually rich analytics.

There is such a thing as too much of a good thing. More visualization does not mean you are solving business problems more effectively, answer questions faster, finding root cause (answering why questions), or getting better predictions and trends? In fact ,too much visualization might be overwhelming users.

A Stroll Through BI History

Let's take a quick ride back in time before we look forward. Human civilization has been evolving for thousands of years and our way out of the stone age was guided by the development of human language and communication. While it is true that a picture can say a thousand words, the spoken or written word, on the other hand,  can express all of human existence in short phrase, e.g "to be or not to be" or "I love you". Human expression through words is powerful - more powerful than any picture.

My point is that human communication is the most powerful expression and exchange of information. It is a fact that visualization is a powerful tool, but it pales in the presence of the written or spoken word. You can probably guess where I am going now.

Computers and computer to human interfaces have evolved over the past sixty or so years on a twisted evolutionary path. We started with simple command line tools and interfaces (mainframe), where we issued simple grunting commands and got back simple grunted responses from our computers. We then saw this lead to the evolution of rich graphical computer windows, icons and the mouse (point-drag-click). While this helped advance our interface and interaction with the computer and with extracting data from within these artificial devices, this path of human to computer interaction is effectively an evolutionary dead end. It pales in comparison with what is coming next.

Evolving Toward AI Conversations - More Than Pictures

Products like Tableau, Looker and others will need to evolve in the coming years or be left in the dustbin of technology. While we have seen amazing advancement in rich and interactive visualizations of data, I argue this is the wrong path and effectively an evolutionary dead end. How many times have you looked at Tableau dashboards (or other BI visualization) and saw beautiful and rich colors, shapes and graphics only to be overwhelmed by the information? What does this information mean, what does it tell me, what questions and answers are buried in this beautiful and rich visualization?

Tableau: Endangered Safari Animals

What if instead of being bombarded by visualization alone, you can converse with the data - converse with the machine? Having rich visualizations can be fantastic, but I would want to ask the machine to answer questions about the visualization - make predictions or tell me "why" this is occurred - point me a the root cause. We are moving to a new dawn where machine learning and AI will help us make sense of the information around us that that is currently locked up and visualized by computers. And this requires a new way (back to the future) for humans to interact with BI.

While computers started out as simple command line beasts, our current evolution toward more and more visualization is an evolutionary dead end. We will soon be moving toward a voice and messaging first world - where visualizations will augment our experience of information and are a tool for us to engage in conversation with our AI powered BI applications and virtual assistants. Chatbot BI virtual assistants are on the horizon.

More Than Just Looking Pretty - Answering Questions

You can see the beginning of this already. Tableau recently announced they will be releasing, in 2017, a new NLP interface to their platform - competitors will follow - and this is only the beginning. We will one day be able to ask questions of your BI in natural human language. The AI powered analytics revolution is coming. Conversational interfaces are a game changer for BI. Analytics as a conversation will no longer be the stuff of movies and sci-fi.

Driven by the advancement in AI and machine learning and with the massive surge in adoption in virtual assistants, chatbots and messaging/voice applications, the future will be here sooner than we think.

Goodbye Apps and Hello Bots

The shift in the market is undeniable. Bots are beginning to challenge the established mobile app store ecosystem. There is plenty of evidence that mobile app adoption has plateaued and that the average user has lost their excitement for downloading and experimenting with new apps. There are more than 2 million apps in the Apple app store now! Ask the average mobile developer - it is almost impossible to get your app noticed or discovered in such a crowded space. Apps will always be with us, much like desktop applications and the company website, but there is a sea change.

Disruption and the New Players
It is becoming clear that jumping from one mobile app to the other is not a great experience for most users (especially for enterprise users) and this is giving messaging apps like Slack and Facebook Messenger the opportunity to become the new app/bot marketplace. GUI-less bots are more easy for users to transition from and to, and they make it seamless to switch between bots and more natural to interact with an application service using human like conversation (something people are already doing in droves on messaging apps). These bots are basically mini-apps with conversational interfaces. Slack (for the enterprise) and FB Messenger (for consumers) are both becoming the new application playground; and the promise of an AI enabled world is lurking within them to provide a user experience that traditional GUI apps are not capable of.

Microsoft is chasing Slack (using Skype) to establish itself in the enterprise team messaging market and in this new emerging bot marketplace. For Microsoft, this is obviously an opportunity to disrupt the mobile app market (where they have lost) and establish an early beachhead with bots, AI and enterprise team communication. Microsoft has clearly been leading the charge with products like Cortana, LUIS and their Bot Framework. All the other big players are in the bot and AI game as well, and the race is definitely on for who can deliver the best bot solution for developers. There is a new land grab in the making between the big tech giants, developers and startups.

How Do I Deliver My Bots?
I describe all this because to deliver conversational applications (aka bots) to end users, developers need a platform and a bot marketplace. Messaging apps will be that vehicle to supplant the traditional app store ecosystem, because building your own custom bot infused mobile app will not be the way to go for most developers in the future. Building a custom mobile app for your bot might still be possible in some situations where an app already has an established user base - like a banking app - but for the average bot developer messaging apps, like Slack, will be the delivery platform.

Messaging apps like Slack also offer a lot of out the box backend integration to help deal with single-sign-on, identity management, permissions, roles and executing custom business logic (via webhooks) for backend integration. Apps like Slack provide much of the platform plumbing for this backend integration that your bots will need, and enterprises are already adopting team messaging apps like Slack. This all lowers the barriers for connecting your bot to a companies cloud and back office systems, in order to get access to the necessary the data and enterprise systems.

Summary
I think the future model for developers will be to deliver their bots and AI conversational services through tools like Slack and possibly others popular "platform messaging apps" such as Cisco Spark, HipChat, Fowdock, FB Messenger, Skype, Kik, and others. All these messaging centric platform apps are already spreading fast through the corporate and consumer world. Developers will be leveraging these messaging platforms to deliver their AI services in the form of bots and conversational user interfaces . Mobile app stores will always be with us, but the game is changing. The new AI marketplace is happening now, get your bots ready!

Bots, AI and the Future of Augmented UX Design

We hear a lot these days about technologies such as futuristic looking VR goggles and mobile apps with augment reality that enhance our interactions with the physical world using a computer generated reality that overlays and assists in our interpretation of the world around us. As computer users, have become accustomed to rich visual interfaces as desktop, web and mobile app technologies have matured. However, the next leap forward in human-to-computer interaction will not be more visual effects, but in fact less, and we are seeing the beginnings of this shift in what we refer to today as "bots". This is only the beginning of a seismic shift in how we as end users interact with applications.

Now, what if we could have the same augmented reality type experience applied to the countless GUI applications we all deal with on a daily basis both at work and at home and from desktop to mobile? What do I mean? Computer applications are already computer generated, so why do they need an augmented reality? Yes that is true computer/mobile applications are already based in the virtual real-estate of the computer (or mobile device), but why do we need any augmented assistance while dealing with the computer application. If you reflect carefully on what is happening with bots and AI in type applications in general, we are seeing the creation of a new human to computer mode of interaction that can assist us in how we interact with the virtual world of the computer application. The dumb and boring old computer or mobile application screen is about to get a big dose of intelligence! Having an intelligent conversation with your application (not just mouse clicks and keyboard taps) will become the norm and is not just the thing of science fiction. Note, this bot/AI augmented application does not necessary have to voice converse, have a personality or hold a deep philosophical discussion with us (maybe some day), but it will be able to assist us in our current world of application beyond just the visual windows, buttons and menu options we have today.

We have come accustomed to interacting with our computers using an already mature human-to-computer model of clicking on buttons and visualizing our experience with a computer through drop down lists and dialog boxes among other widgets and interfaces. But what if we could augment our interaction to a consumer application (i.e. banking app) or an enterprise application (i.e. supply chain application) with a  an intelligent chatbot that could aid us in the interaction with said application and the many knobs, controls and actions you can invoke on the application screen? This bot assistant could remember what we have done in the app in the past and guide us through taking actions using a combination of chat/message exchanges sprinkled in with intelligently timed suggested actions. This could in fact lead us to to a situation where we do not needed the full blown array of buttons and menus that bloat the apps we have today. We could have a conversation with the application with the help of an intelligent and conversational chatbot assistant.

Well this is where the world will soon be fast moving towards. With the ubiquity in mobile communication and advancements in machine learning, AI and big data, the scene is now set for every application we have come accustomed to using to have a chatbot assistant that can aid us in our interaction with the application itself. No more stupid dark ages style help documents to sift through. Think of it as online docs help on steroids and this just the beginning.

Any enterprise or consumer application team not starting to think how to they can replace their outdated online help docs and bloated UIs with more efficient and engaging intelligent and interactive chatbot assistance will be left in the dustbin of technology history. Don't worry you have a few agile sprints before this happens :)

This transformation is not to be taken lightly although. It will be a significant investment of both engineering/technology and a big leap in thinking in how we design user experiences for end users and how we can expand the visual application metaphors we have grown accustomed to with new intelligent chat assistants that can guide us through the navigation of information and assist us in the potential actions that can be taken within an application.

This technology leap will require a big shift in thinking from product owners, UX designers, information designers and engineers. This will require everyone in the product development ecosystem working together using enhanced business and engineering processes that put this new augmented UX design philosophy in the forefront while on the engineering side leveraging fast maturing technologies in the areas of NLP, AI, and machine learning to enable captivating and predictive conversational engagements between humans, their devices and their applications.


The applications of the future, whether on your desktop or on your mobile device, will in the coming years begin to manifest augmented UX capabilities. Be prepared for this new world whether you are a developer, UX designer or end user of these applications. Conversational interfaces are coming to an application near you to augment and enrich your application user experience!

Analytics as a Conversation

The pendulum is swinging in the business intelligence and analytics world. The on going technology evolution driven in part by the adoption of Big Data, machine learning and other advancements in cloud computing have made the storing, modeling and analyzing of huge volumes and velocities of data possible. The tools and IT skills needed to turn this data into rich visual information is now more possible than ever before.

Products like Tableau, Splunk, Qlik, Birst, among others, have brought rich visualization and actionable-minded analytics (actionable analytics still not that common :) to the masses. It is now easier more than ever to build rich visualizations, reports and dashboards. Building BI solutions to tackle all that data percolating around us and across social networks, IoT and within the enterprise is available to the IT masses to build compelling visual user experiences.

But there is trouble brewing on the horizon. Is there such a thing as too much data? Too much information? Too much visualization? I have built my share of BI and I have seen many amazing and compelling visualizations and dashboards using powerful solutions like Tableau and many home grown SaaS BI platforms. But I think it is time to step out of the forest and look at how humans effectively interact with information.

While we rely heavily on our visual sense, even the most well intention and minimalistic BI dashboard (and its supporting drill-down reports) might not be the best solution all the time at getting to the information you want or need. Humans have another ability for consuming information, the conversation (question and answer).

There are many technologies now converging and making it possible for us to evolve our BI stack beyond purely visualization based analytics. Analytics-as-a-Conversation (A3C) in my mind is the next frontier for BI. It does not necessary replace today's rich visualization based BI, but augments it.

What is A3C? Well, in movie terms, it is sort of the Matrix. It is about having a conversation with your BI and getting at what you need (the what) through normal human-like conversation (think texting, hashtags, tweets and even emojis). Also, this conversational form of BI is a much more natural way of interacting with complex information and can more naturally lead to asking not just the "what" questions but the "why" questions to your BI Matrix. And this form of information interrogation lends itself to setting a more clear context to the information exchange, as the BI conversation progresses from one question-answer to the next question-answer. For example, perhaps you ask your A3C system the value of a particular KPI or which KPI is the most off its norm this quarter. And then this can naturally lead to such questions as to "why is this KPI higher this quarter?"

Obviously we are not Neo and we are not talking to the Matrix, so the system has to be taught (or programmed to learn) how to converse with a human-like grammar and has to programmed to extract what it needs from the grammar/questions using NLP and then translate that into queries against the target data and metadata system. There would have to be bounds on the grammar and enough knowledge of the system's metadata to compose the proper answers. No small engineering effort, to say the least, but from where we are today with AI, bots, machine learning, NLP and general computing stacks, the technology is there to accomplish this.

Why now? Because the technologies needed to construct the BI Matrix I am describing is largely here and the data volumes are now, in my mind, overwhelming even for the best BI visualizations. With a bit of creativity (and sweat), and with current availability and advancements in Machine Learning, AI and general computing power, it is possible today to begin to build such intelligent conversational analytics systems and user experiences. Don't forget this a about changing how the user "experiences" data.

It is not just about data volumes and technology capabilities, human interaction has itself evolved in the past decade. We have seen with the recent explosion of mobile and social communication that humans are using texting and short messages for communication more than ever and with no sign of ebbing. In fact, texting is quickly becoming the dominant form of communication and the main form of information exchange across the globe and across all demographics.

How is this better than the visualization based BI we have today? Well, I would say it is not necessarily a replacement for the BI we have today, but is instead complementary and can lead to BI answering questions of "what" and "why" that the original BI developer/modeler could not necessarily anticipate out of the box. And as artificial intelligence and machine learning systems continue to evolve and improve the potential is virtually limitless and no longer bounded by what can be rendered on a 2D display or a click of the mouse.

The revenge of the CLI (the command line interface) is upon us :) But don't underestimate the conversational CLI, it will prove to be orders of magnitude more powerful than any visualization a human can conjure up. The CLI is coming back, but it will be smarter and more interactive and have a bit of a human personality. It will not be called CLI anymore (that is for the techies), it will be called CUI/CUX (Conversational User Interface/Experience) and it will be embedded in the fabric of our mobile and desktop apps of the future.

Stay tuned....Analytics-as-a-Conversation is coming and we will all be talking about it (or talking with it).

What's Next? Conversational Enterprise Applications

There is a lot of chatter these days (excuse the pun) around AI, machine learning, and chat bots and how this technology stack can be used to engage with users, at a human like level, to exchange information and automate tasks. The elusive goal of an all intelligent AI machine that can by indistinguishable from a human to help us with day to day tasks has been with us since the Turing test and has been embedded in our psyche from countless sci-fi movies.

Today, that elusive goal is closer with the many advancements in computing and communication technology. We are starting to see the real application of such technology in tools like Slack where chat bots sit in the background ready to engage in channels/rooms to assist and respond to natural language chat communications to help solve/automate DevOps tasks or monitor and manage IoT infrastructure among many other applications. And we also see it with more casual consumer applications like Siri, Cortana and Alexa.

Where this is all headed is exciting both for consumer and enterprise applications. However a lot of the current focus for how and where conversational interfaces can be applied is still stuck in the past. In my opinion there is too much attention given, for example, to a chat bot's personality and if that chat bot is behaving with true human like mannerisms. I think this distracts from the actual transformation that is happening and the opportunities that lie ahead for where and how conversational interfaces can transform business applications. A conversational interface does not necessary need a human personality to be effective - keep that in mind when you go down the path of building this new form of user experience into your applications.

If we look back in time, we started first with "dumb" computer command line interfaces (i.e. the green screen CLI), then through the 80s, 90s, and early apart of this century we went through a steady evolution toward more visually rich human to computer UX (think desktop apps, web x.0 and mobile). Interestingly enough, this has brought us full circle and back to the command line interface (CLI). But this new CLI is now "intelligent" and has the potential to take us to the promised land of conversational human to computer interaction. I wont get into the theory of why the intelligent CLI (aka the conversational interface) has reemerged and why it will prove to be more effective than our current bloated visual UX application world. And remember that the conversational interface can also be voice driven, but voice to text is more or less an added bonus and part of the longer technology maturity in this space.

What does the future hold? I propose that this next generation intelligent CLI should augment every business application in the coming years. Every enterprise should take a hard look at their current UI applications (business and consumer facing applications at every level) and make it a high priority to embed AI chat bot like intelligent CLIs (with or without a personality :) into every user experience and business function they have and for every application persona. Whether you are building an ERP application for an HR manager or a sales executive, or an IoT monitoring platform or an analytics dashboard, every one of these applications should have an intelligent conversational interface to augment the visual interface. By 2020 any enterprise not beginning to bring to the market such investments in conversational interfaces within their applications will be left in the dark ages of the visual only UX world.