Why is there a need for Streaming Analytics?

There are three key technology approaches that are driving the need for a solution:

1. The arrival of new data architectures, as seen within the Big Data arena, now looking for a streaming architecture.
2. The Internet of Things (IoT): assets, sensors, machines pushing data into real-time analytics and operational management.
3. Cloud analytics: cloud deployments are increasingly looking at dealing with real-time ingestion and distribution.

How does streaming processing change things? The most common type of analytic processing is based on a pull model: the data is collected in a repository and queries are run against the data. The responses are based on the data at the query time. Queries have to be rerun as the data updates, and results may take seconds, minutes or even hours to produce. This is a reactive style of management; the insight in the data is only realized when someone looks for that value. The new paradigm is to have “active queries” where the answers update in real time as the data changes. This is Streaming Analytics.

One of the key drivers, IoT, embodies these concepts. IoT is connecting objects and devices: everything from cell phones, televisions, fitness trackers, as well as manufacturing equipment, automobiles and jet engines. IoT is creating real-time business events that present significant opportunities for companies to transform how they: engage with customers, operate their business and deliver their services. However, in order to do this, companies are faced with the task of analyzing large volumes of data; in motion and at rest. Their challenge is to identify patterns, assess the past, make real-time predictions and act on these insights.

Researchers forecast that there will be over 50 billion devices connected by 2020 and this clearly will continue to increase dramatically. These devices capture vast quantities of data, but it will take significant maturity levels for data and analytics leaders to turn this ocean of data into valuable insight and action.

Why use Kx for Streaming Analytics?

In his “The Five Levels of Stream Analytics — How Mature Are You?” paper, Nick Heudecker from Gartner talks about how “data and analytics leaders are increasingly targeting stream processing and streaming analytics to get faster time to insight on new or existing data sources”. He continues, “in getting to real-time, these leaders are presented with a range of proprietary commercial products, open-source projects and open core products that wrap some existing open-source framework. However, in many cases, streaming analytics capabilities are little more than commercially supported open source bundled with some other product. Creating a streaming analytics application is left as an exercise for the buyer. The challenge is that getting real value from streams of data requires more than just a point solution. Stream analytics is a cross-functional discipline integrating technology, business processes, information governance and business alignment. It’s the difficulty integrating these areas that keeps many organizations from realizing the value of their data in real-time.”

Kx has been a pioneer and trusted performance leader for in-memory computing, streaming analytics and operational intelligence for over two decades. More than 100 major corporations, including the world’s largest banks, brokers and regulators rely on our unique technology to address the challenges posed by exploding volumes of information and ever-faster data. As the Internet of Things (IoT) transforms the world, new industries are using Kx technology because of our unique experience with real-time and historical time-series data.

In independent tests, Kx technology has been shown to be many orders of magnitude faster than traditional technologies, retrieving data in microseconds rather than seconds, aggregating data in just milliseconds, not hours, and performing complex queries in seconds, compared to alternatives which can take hours or even weeks to perform comparable analyses. Designed for speed, Kx enables IoT streaming analytics, which are fuelling business innovation, and becoming essential for organizations to react in real-time to the information they are collecting and processing as part of their business functions.

Przemek Tomczak, SVP IoT & Utilities at Kx says: “As IoT evolves and becomes the ‘Internet of Everything’, exciting new opportunities are emerging. This requires data to be processed and analyzed in real time, blending streaming and historical data from a diverse set of sources. Firms will then be able to reap significant benefits, including improved business operations, better relationships with and service for customers and the ability to develop new products and services and a reduction in costs, to name but a few.”