Data components

A Validere data platform is a centralized end-to-end solution for ingesting, normalizing, processing and delivering data generated by systems, processes and infrastructure. This section defines some of the terminology of its components.

Data Product

Data Product is a generic term used to describe a product that facilitates an end goal through the use of data. More specifically, to the data platform, a data product is any sensible collection of datasets, e.g. data from a particular client site that can have many different datasets.

Data Source

Data Sources comprise connectors to various systems. These connectors include but not limited to RESTful API, email integrations, data streams, or IoT protocols like MQTT.

Data Lake

A Data Lake is a centralized, scalable data store for all data. Data first arrives at a landing zone before it is archived in raw format. Parsers are then utilized to normalize the data such that it can be utilized by various Validere consumers.

Parser

Data Parsers provide preliminary normalization and processing of data pertaining to a pre-defined data model. There is a finite numberr of normalized data models taht are being utilized by downstream systems at Validere. Providing the necessary nomrmalized flavours of incoming data alleviates pain in processing and validation for such systems.

Sink Data Sinks are downstream consumers of data that is funneled into Validere. There is a finite numbre of sink types that can be configured via pre-defined attributes. Current Validere sink types include databases, REST API's, data warehouses, and S3.