2017 is forecast to be a banner year for blockchain, and potentially able to facilitate the development of a distributed global supercomputer, corporates like IBM are increasingly exploring the space, alongside smaller corporates such as Software AG.
In parallel to this is the projected rise of streaming analytics, amongst other trends predicted for the Big Data space in 2017.
Increasingly, more companies will seek to combine microservices and machine learning, leveraging on new services that can capitalise on immense amounts of historical data to refine how they interpret streaming data.
With organisations not only producing vast volume of data in different formats but needing to capture and store them, followed by extracting insights from their data warehouse later, real-time data analytics and streaming analytics – the analysis of large, in-motion data called event streams – are increasingly integral in business operations.
At a fundamental level, the adoption of streaming analytics will render organisations more agile and drive the growth of their top-line, in addition to finding applications in real-time marketing and customer engagement, algorithmic trading, real-time patient monitoring system, intelligence and surveillance, supply chain optimisation and procurement, as well as a range of other domains.
Impacting verticals as diverse as the telecoms sector, media & technology (TMT) sector to healthcare, manufacturing, energy and utilities, the blockchain is already impacting the banking, financial services, and insurance (BSFI) space.
In an exchange with Giles Nelson, SVP of Product Strategy & Marketing, and Jigar Bhansali, Director Technology & Solutions, Software AG Asia, discuss streaming analytics, the blockchain and developments in the IoT space.
Streaming analytics – what’s the value to these financial institutions at a strategic, operational and tactical level?
Nelson: Streamng analytics enables data analytics in real-time. Most data analytics looking at historical data but with streaming analytics, you can have real-time insights into how people behave in their spending habits, or historically reviewing transcations on a stock market.
With data warehousing and business intelligence, you have a plethora of tools for analysis of the past. But how do you examine and take actionable data on the present? For identifying fraudulent transactions, you review data on past transactions but wouldn’t it be much better to look at transactions as they occur and identify what might be fraudulent as it occurs?
You can only do that right now by looking at the tranactional data as it comes in but streaming analytics gives us the ability to work in the present and predict what comes next.
I can’t peer into it too far but you can look into the near-term and predict transactions in minutes, hours and days using the streamed data. This allow for a time continuum within the streaming analytics product. With this you can do things such as tracking and identifying retail consumption patterns and facilitating a real-time response to transactions that are occurring.
Bhansali: Performance of the analytics is crucial as the solution is embedded in memory. I’ts about using the aggregated informaiton of customers to assist the streaming platform in making sense of the patterns and the streaming story. This involves taking the aggregate historical data and making sense of these in the context of the real-time insights that are generated.
What’s the biggest challenge for institutions when implementing and integrating streaming analytics and other digital products into their work flows?
Nelson: What’s the ROI? How will it impact customer interaction? That’s the most significant thing. But its not pertinent or unique to streaming analytics. More specifically, how do you get the data you need in order to make your streaming data – transactions or how the customers operate – work to your benefit?
The integration and storage of data in conventional systems doesn’t provide the speed needed for streaming analytics. It’s about integrating real-time data with historical data in order to conduct the analysis. The key thing is in memory, and the key strength of a company is a lot of in-house memory.
We acquired a company called TerraCotta four years ago which had a memory data grid that allowed us to take existing data from a database and put that in memory. This in-memory caching technology accelerates our own data products by putting it all in memory. The attitude is about enabling products with in-memory capabilities that can scale.
Bhansali: People operating in departments are often operating in a silo. It’s often about trying to bring together all these different teams, elements and transactions together, in order to correlate all the relationships and build the insights from there.
Richard Branson has argued that the blockchain will create an economic revolution in emerging markets. Your take?
Nelson: It’s being overstated. What does the blockchain enable? It enables you to get away from a centralised ledger run by a central authority – typically a state organisation, whether it’s a land registry or central bank – and in economies where these are not entrreenched and are corrupt, then it does give the possiblity for technological solutions.
It applies in those places where it doesn’t need a normal civil society and government to be able to implement it centrally. Does it have potential? Certainly, especially in areas such as micro-banking, where you have the large unbanked populaitons that can be serviced with digital banking solutions. In such an place, telcos can provide financial services to these people and fill in for the banks.
I agree with Branson’s comment that it certianly has the potential to do so, but you need to have someone running the blockchain; there’s got to be an porgnaisations with oversight of these private blockchains. The question is who is going to run this and what is the economic model?
For things like a land register and other similar phenomena, you need to encourage people to use models that ensure longevity of the blockchain. There’s a lot of things to sort out but there is certainly a lot of potential.
For instance, in a corrupt state with a deficit of credibility, perhaps a state-run distributed ledger could fill in and make up for some of the credibility gaps it has.
You’ve done volunteer work for PharmaSecure. In the context of non-profits, what would be the value of streaming analytics for an NGO in managing their finances and being more efficient?
Nelson: With regards to Pharmasecure, their main business model involves working with pharmaceutical manufacturers to deliver a simple authentication mechanism for end-users to validate drugs. This is due to counterfeit pharmaceuticals being a significant problem in the developing world. It is often the case that someone will end up a with a counterfeit pharmaceutical that has no effect or could be dangerous.
Pharmasecures’ model is to have a technical solution that sees delivery codes that can be scanned or typed in on feature phones. These can subsequently be validated with a trusted authority such as the pharmaceutical company manufacturing them. For their manufacture or traceability and tracking issuance to consumers, the blockchain can certainly play a role.
In all of those parts of that transaction – blockchain or otherwise – you might want to look for anomalies in the number of drugs of a certian type issued by a set of chemists in a particular location. If that has risen significantly over time (i.e. drugs being issued), either an outbreak of disease or something more nefarious is going on.
It can certainly help in monitoring and preventing fraudulent transactions – looking at particular goods or tracking those dangerous ones – so in that context, streaming analytics could be applied to something like Pharmasecure’s business.
What sort of programming languages do you need to have a grounding in to participate in the software development for data analytics solutions? What are the elements required for a basic level of technical competence?
Nelson: Imperative progamming languages like Java are probably the best, but in my view data analytics companies are more interested to look at how people program or come up with predictive analytics models in the machine learning space. What we’ve done at Software AG is focus on machine learning and predictive analytics.
You have giants (i.e. IMB and SAP) and mid-tier players also competing in this space for talent. As a data scientist I’d choose my favourite tools – there’s a plethora of different tools available – and then at some point I’d want to put those data analytics tools and model into production. It looks as if PMML (Predictive Model Markup Language) is an emerging standard, so we could be looking at marketing that.
There’s a lot of tools and those data scientists might want to use a tool called R – a programming language that’s an open source standard – and you can create your model in that and then operationalise it within Software AG’s platform. It requires some programming language skills, but the real skill is in understanding data and being able to extract meaningful insights from it.
It’s about knowing how to manipulate it and what statistics one may use on that data set – garbage in,garbage out – and essentially knowing that the information you’ve extracted from that data is meaningful.
Singapore is competing with centres like Dubai, Hong Kong, Tokyo, London, New York and Shanghai as a financial centre. What are the digital elements a financial services market ecosystem needs to remain competitive?
Nelson: In essence, you need a legal framework, in terms of how organisations make use of their digital data. Firstly, how can you make more efficient an existing proces? Can you remove some people and paper from the process?
How companies can progress is to ask themselves whether they have a rich database with digital data and imprints of their customer behaviour. From there, if they’re a retail brand they can ask how customers use their retail space and leverage on that information to deliver desired services. That then has a bearing on the infrastructure within a city.
It comes back to attitudes and legal protection around personalised data. Ideally, as a company, you want to be able to use the data that you have on somebody as liberally as you can while having some respect for it in terms of privacy. But if you are in a jurisdiction with significant restructions on personal data, it’s much harder to gain an advantage from that data.
You have different cities going through this phase, so how far does a customer want to go in trading their personal information for better services in the future? That attitute towards privacy and the legal dimension behind it..there’s no right or wrong but having knowledge of that and allowing companies to negotiate with that is an important part of companies becoming truly digital companie and monetising the data they have.
It’s about creating data analytics models. Mark Andreesen argued how every enterprise is going to become a software company. I’d modify it to say thtat “Every company is going to be a data-driven startup”.
Digital marketplace companies – not all are coming to digitisation – make up a tiny fraction of the world’s GDP. What is far more interesting is to look at the SGX, LSE and NYSE, at the Top 100 companies in the pharmaceuticals, mining, telecoms, healthcare, and manufacturing sectors. None are purely digital, though all of them have a legacy of producing something digital.
It’s those companies are going to start using digital data and its about how they will become digital companies. Now, how would a mining company benefit from digitisation?
In terms of the Internet of Things (IoT) and the questions surrounding it, now…how is that going to work? It’s about how sensors make an environment more operationally efficient. It’s from a macro view where the really interesting ones are coming from. Where does a city or country need to move on this path?
It’s about enabling access to skills – whether it’s a mining company or manufacturing company – and I would be using this technology operationally to identify an issue with production.
What I would not be used to is working with the IT people to capture data and start asking what sort of cultural chance can impact that. Every company is going to have issue swith recruiting data sceince skills who can origniate the models and communiacate how the models will impact business operations
Bhansali: Singapore has a lot of initiatives aimed at helping the country grow in that direction, with the Smart Nation initiative being one example. There’s quite a few thing sthat are already happening.
But they still have to improve the experience that citizens and customers, in terms of the manual activities like the on-boarding process; thoes processes still need to be automated and are key. It’s not just about putting in sensors, collecting the data and analysing it but how it impacts operations.
Does Software AG have any plans for corporate venturing or establishing an intrapreneur unit to grow target the different segments?
Nelson: We do make investments in other businesses. It’s question of how businesses can innovate themselves. We invest in order to get access to technology that cultutally we would find more difficult to organically create. We’re an acquisitive company and we do look to buy significant companies as well as enterprises which have something original that will expand our technology proposition.
One thing we have done in the IoT area recent was to finalise a big deal with Bosch that we announced in October. We also have some other significant IoT-related deals in the pipeline. IoT is a great area for streaming analytics and one thing we have done there is to establish a global centre of excellence in Germany, with a our consulting team involved in that.
Bhansali: Over the last two years, there’s been significant investment in our partner ecosystem, in terms of developing partnerships with a few global system integrators (SIs) aimed at developing solutions. For instance, we run a marketplace that offers those solutions where we can develop joint solutions with partnered SIs.
Let’s talk about the state of the market; cognitive computing and distributive micro-analytics have been highlighted as major growth areas. What does this translate to in real terms for corporations exploring implementations of this?
Nelson: I think that every company is going to become a data-driven company, using that data as an economic resource for new,purely digital products you can monetise. In terms of business intelligence, we’re using open source technologies and things like Hadoop; when it comes to being a data-driven organisation – as most companies will eventually drive revenues through IoT – then streaming analytics will be a key tool.
If you’re not going to do that, then you will be undoutedbly be a laggard in your business. When you’re looking at machine computing/cognitive computing – IBM Watson and DeepMind are good examples – it is an area that the company needs to get into as well.
I doubt that a technological singularoty will happen, but there will be new types of jobs create that are going to require more sophisticated human decision- making. So we’ll need to be more sophisticated about how we use digital tech.