The enabling technologies of IoT — connected devices and sensors with acceleration of analytics via the cloud — rely on an integrated data infrastructure. GE Digital is at the leading edge of the convergence in Industrial IoT, providing a blended data pipeline that integrates cutting-edge digital instrumentation with financial systems like pricing, billing, and subscription management. With faster access and insight across physical plant, customer, and platform partners, GE Digital provides unique leverage via advanced analytics and artificial intelligence, to achieve data-driven efficiency, productivity, and profitability.
The Challenge
GE Digital needed an agile data fabric suited both for real-time data ingestion and enrichment, in tandem with a broad spectrum of systems of record. Building a new agile data pipeline needed a more modular approach beyond conventional data lakes and traditional data warehouses. Moreover, the architecture needed to accommodate a scalable, secure approach to tenant isolation and service multi-tenancy, as often different competitors in a single industry needed the confidence that their data was private and secure. In addition to maintaining quality data streams at ingest, it needed to deliver variable capacity for continuous bursts to data flows. This adaptive approach is essential to unlocking the potential of machine learning and AI for smart, connected products. It can unleash untapped digital potential for asset-intensive industries.
The Solution
CloudGeometry provides GE Digital’s IIoT infrastructure with data pipeline logic and integration services. This blends event data into a flexible platform serving a range of downstream analytics cases, ranging from business intelligence dashboards to machine learning. The data ingestion process draws on asynchronous event payloads from both physical and digital devices. Different enrichment algorithms create extensions to even data, such as: time-and-date for calendar intervals (e.g., global time zones) of cycles (for subscription reporting); geo-tagging to add information on event locales; and the like. Data quality is continuously tracked across data cleansing, mapping, and quality processes.
Asynchronous processing of IIoT event streams is a key to cost-effective data integration in the context of true multi-tenancy. CloudGeometry built containerized event logic that runs worker processes on demand, spinning up and shutting down pipelines to optimize compute costs. The pipeline architecture also takes advantage of storage economies thru split caching of both short and long-term key-value stores, complemented by metadata stores, for configuring of both events and the range of devices.
The Benefits
The versatility of end-to-end data pipeline and data integration by CloudGeometry provides GE Digital a flexible, continuously adaptable data movement as IIoT adoption accelerates to its full potential.
Data Ingestion
Use powerful, modern data connector framework to pull data from any source, stream or batch; add new data & transformation rules in minutes, not weeks
Agile Analytics
Cut friction of transformation, aggregation, computation combining streaming (Kinesis), Data Warehouse (Redshift) and relational (Aurora) data stores
Elastic Microservices
Easily configure and run Dockerized event-driven, pipeline-related data tasks with Kubernetes
