Thus, data should be moved from the platform in which it resides to another platform. Mandatory cross-platform: Applications may require multiple platforms because the platform where the input data resides, e.g., PostgreSQL, cannot perform the incoming task, e.g., a machine learning task. Opportunistic cross-platform: Applications might benefit performance-wise from using multiple platforms to run one single task. Platform independence: Applications run an entire task on a single platform but may require switching platforms for different input datasets or tasks usually with the goal of achieving better performance. We have identified four situations in which an application requires support for cross-platform data processing : (1): Research and industry communities have identified this need and have proposed systems to support different aspects of cross-platform data processing. Ĭross-platform data processing As a result, today’s data analytics often need to perform cross-platform data processing, i.e., running their tasks on more than one platform. Examples of such analytics include: (i) North York hospital that needs to process 50 diverse datasets that run on a dozen different platforms (ii) Airline companies that need to analyze large datasets of different data formats, produced by different departments, and residing on multiple data sources, so as to produce global reports for decision makers (iii) Oil and gas companies that need to process large amounts of diverse data spanning various platforms (iv) Data warehouse applications that require data to be moved from a MapReduce-like system into a DBMS for further analysis (v) Business intelligence applications that typically require an analytic pipeline composed of different platforms and (vi) Machine learning systems that use multiple platforms to improve performance significantly. Modern data analytics are characterized by (i) increasing query/task Footnote 1 complexity, (ii) heterogeneity of data sources, and (iii) a proliferation of data processing platforms ( platforms, for short).
0 Comments
Leave a Reply. |