EIQ Products™
Almost all conventional federated data systems have to rely on adapters and/or middleware to submit queries to data sources for execution
Data remains in data sources and queries are translated from a standard data model to queries that each data source can execute. Queries are executed on, and results retrieved from, data source systems. The components that translate queries and transform result-sets are called adapters, which operate as illustrated in the following diagram CFA1:
CFA1: Conventional federated adapter-based data access solution
Advantages
Conventional federated adapters were pursued in an attempt to overcome some of the disadvantages of data warehouses by providing the following primary benefit:
-
Data remains at source
The above primary benefit overcomes many of the data warehouse challenges of:
-
Complex ETL process (time and cost) - see comment below*
-
Data ownership issues
-
Static and dated data
-
No drill-down capabilities
Disadvantages
However, there are considerable disadvantages of conventional federated adapters that generally counter data warehouse advantages due to data source constraints:
-
Dirty data “as is”
-
Limited indexes – not consistent across data sources and not flexible
-
Limited query processing
-
Query load on data source systems
-
Query performance
-
Security
-
Data source owners aware of queries (intelligence-related)
To accommodate the translation between an application or information sharing system and any particular data source, conventional federated adapters are developed, typically, over a significant period and at great cost to cover basic requirements. It typically costs 300 to 500% of the initial adapter purchase cost to customize conventional adapters to cover basic requirements, never mind advanced or future requirements.
*The only advantage conventional adapters have over the ETL process is that schema transforms are not as difficult; however, query processing and subsequent results transforms are more complex.