Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« September 2018 | Main

December 7, 2018

Embrace the Future

The unfolding of Cloud Computing, Introduction of Enterprise level Integration Patterns & up folding of micro-services has not only disrupted the existing nomenclature but also makes us think the way we do Integration. When every pioneering companies are embracing the micro services to tackle their complex enterprise architectures, there's one aspect of is which is still open for exploration i.e. Data Validation & Data Warehousing.

Yes, it is true that there are many organizations who are consciously embracing the concept of data services around there data lakes for either master data management or for analytical purpose (simple data read) but very little have been thought upon using the full flavor of micro service driven architecture on areas like data integration, data quality or validation and metadata management space of work.

If we travel back a few years in Time, the idea of SoC (separation of concerns) was ignored due to the need of heavy lifting of data and availability of Integration Tools which usually were tightly coupled with each other. These tools were an Instant hit as they wrapped up complexities of managing job failures, providing reports etc. but it could not fully tackle the learning curve, complexity involved and most important - the need to adapt to frequent changes.

The basic principle of micro service is to break a complex application and decompose it in to multiple self-contented services which can connect to each other to achieve a complex functionality. Given that above use cases are always complex in nature micro services could be a great way to automate data validation & design our future data warehouses.