Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« September 2019 | Main | August 2020 »

May 19, 2020

Data Fabric - The Futuristic Data Management Solution

Global Research and Advisory firm Gartner has identified top data and analytics trends for 2020, which will have significant transformative potential in the next two to five years. Data Fabric is one of the most prominent trends. With the enormous growth of both structured and unstructured data from smartphones, IoT devices, and digital channels there is a need to be able to process large amounts of data, mine it, analyze it and make it accessible. Data Fabric is a method to understand large amounts of data traversed through the cloud systems.

In this blog, we will understand What is a Data Fabric, The Data Fabric Stack, and a few Use Cases.

A.      What is Data Fabric

Data Fabric is a unified architecture and data service set running on that architecture, that helps organizations manage their data across on-premise, cloud, and hybrid cloud systems. Data Fabric is a single, unified platform for data integration that simplifies and integrates data management across platforms to accelerate digital transformation.

 

§  Connects to platforms using pre-packaged functions and connections

§  Integrates and manages data from on-premise and cloud environments

§  Support for batch and real time data streams

§  Data Quality, Data Enrichment and Data Governance capabilities

§  Support for API development and integration

 

B.      The Data Fabric Stack includes following layers - 


·        Data Collection & Storage: Ingest & Integrate data, events and APIs from any source, from on premise and in the cloud

·        Data Services: Manage several services at this layer including data governance, data protection, data quality and adherence to compliance standards

·        Transformation Layer: Involves cleaning and enrichment of batch and real-time data to enable informed decisions

·        Analytics/Sharing Layer: Realize data value by making it available internally and externally via self-service capabilities, analytic portals, and APIs


C.       Successful Use Cases

«  A leading pizza company in the world, with both delivery and carry put operations, utilizes data fabric to maintain the competitive advantage. It allows ordering of pizza from a plethora of devices including TV, Smartwatch, Smartcar, etc. resulting in 25TB of data, from 100,000 data sources - structured and unstructured. Using Data Fabric, company gathered and analyzed data from their POS systems, multiple supply chain centers, and across digital channels including text messages, Twitter, Amazon Echo

«  A leading pharma company, applied AI to develop weed identification, and enabled farmers to apply the exact solution needed to kill the weed species. It developed an app which used machine learning and artificial intelligence to match photos that farmers uploaded to the app. This resulted in better choice of seed variety, better application of crop protection products, and best harvest timing

«  Leading insurance company is utilizing data fabric to store and analyze claim data - claim report, incident data, police report, claim history, claim details, counterparty details, etc. This has helped in faster settlement of claims and also to make policies more compelling and price them competitively

Conclusion

In a world where technology is changing everyday lives, digital transformation tops the strategic agenda of most organizations and their leaders. To be successful in digital transformation journey, data is lifeline, to enable new customer touch points, create innovative business propositions, and optimize operations. Data fabric enables businesses to achieve these by offering connectors for hybrid systems, advanced data integration capabilities, and analytical capabilities. Demand for data fabric will get stronger as organizations look to stay on top of emerging technologies and new trends to stay competitive, stay relevant and maintain business edge.

May 13, 2020

'By failing to prepare, you are preparing to Fail' - Data Analytics and its role in dealing with Covid-19

During the past few months, the world is fighting a battle to contain a lesser known virus with no cure or vaccine in sight and which has had far reaching impact on the world's healthcare and economic stability. Informed decisions are need of the hour and these decisions are making use of data generated from various feeds globally across sectors. COVID-19 has brought to the core the importance of Big Data Analytics. 

Analytical tools like ML (Machine Learning), AI (Artificial Intelligence) and NLP (Natural Language Processing) are being used individually or as a combination to derive information around disease profiling, nature and spread of the virus etc.

Below are my views on how Data Analytics can help respond to a disease with such far reaching implications globally:

A.      Develop the Correct Response - Data available across sources worldwide has helped develop the right measures which has proven that early interventions can disrupt the spread of the virus. Suggesting a few ways:

a.       Quantify Risks - Identify the cities where maximum movement of folks happens. This helps be ready with different responses and strategies to be adopted like home quarantine etc. and focus on local transmission in the villages/small towns. Also, plan and implement social distancing by projecting infections vs. casualties over a 2 to 3-month time frame.

b.      Predict trajectories - Identify and categorize patients who are more vulnerable to the disease and those cases which could lead to severe respiratory issues.

c.       Develop Response Strategies - Simulate a variety of best and worst case scenarios to help the government decide the combinations of measures required to be implemented to reduce or flatten the curve.

B.      Predict the Invisible - Probably the most widely used dashboard worldwide to understand the spread of the disease is the Johns Hopkins Center for Systems Science and Engineering's COVID-19 Dashboard. This idea has helped educate the public on the exponential increase in the numbers and serves as a warning. Alternatively, it has helped governments to identify the strategy, which would work best, taking into consideration the learnings from other countries by looking at the trends.

From my understanding, the technologies being leveraged to deal with the outbreak are:

1.       Statistical & Simulation modelling - Tracking spread of the virus, predicting outcomes, estimating when the demand for healthcare services would peak, demand of ventilators etc.

2.       Artificial Intelligence -

a.    Chatbots serving as virtual assistants over tele-callers/support staff and also serve as healthcare agents to answer basic COVID-19 related questions

b.       Diagnosis - Technology used to diagnose Pneumonia, a common complication from COVID-19 from CT Scans in a short duration and with high accuracy

c.       Fever detection - Cameras with AI help detect people with fever, recognizes faces and also determine if they are wearing masks; all this without any manual intervention

d.       Research - To assist in the development of antibodies and vaccines by recreating models of the proteins that are associated with the virus

Conclusion

Eventually, mankind will find a way to put an end or at least find a cure to this life threatening virus. However, this pandemic will leave plenty of opportunities for those seeking help - using technology to promote advances in healthcare, government response, business continuity plans, and more. Using AI and ML, there is an opportunity to come up with innovative solutions or products like use of drone for delivery of medicines and essentials, robots to assist patients with contagious diseases when kept in isolation etc.

Companies and governments alike will have to start making use of the reliable predictive abilities of analytics to identify bottlenecks and plan ahead to handle such disruptions effectively in the future. Analytics will help you think about various situations in difficult times and enable organizations and governments to prepare for the worst and respond better. As Benjamin Franklin rightly said - 'By failing to prepare, you are preparing to fail'.