Migration of data between disparate systems remains a significant challenge for organizations as they consider onboarding new back office systems. Organizations will often procrastinate and delay the procurement of a much-needed business transformation due to their legitimate fear of the data migration headaches this may cause. While this fear is legitimate, it also means that many companies continue to maintain archaic systems and tools thereby impacting their operational efficiencies. It is evident then that simplifying the data migration challenge can go a long way towards alleviating service provider concerns.

Data Import

Data into a back office system such as a billing system can arrive from multiple sources. When onboarding a new system, businesses typically need to migrate volumes of data from other systems into the new system. This data often needs to get transformed into a format suitable for the new system. Import of data does not just happen during deployment; during the typical operational flow of a system data continuously moves between systems. For instance, product catalog information might move from a billing system to a CRM system, while GL information might move from a billing system to an accounting system.

Data Transformation

Input data from data sources typically cannot be ingested in the receiving system without undergoing some form of transformation. This transformation can be a simple mapping from fields in the input system to the receiving system. In many scenarios however, a transformation process typically needs to be set up to transform the incoming data prior to ingestion. Fields can be combined and ignored, formulas can get applied to fields, and conditional logic applied to input fields to parse the data based on input conditions. For example, consider usage data from devices such as SIMs, switches and sensors. Each device generates data in it’s own proprietary format. To interpret this data, the system that is consuming it needs to mediate and transform this data into something that can be understood. Ideally the transformation process needs to be something that can be achieved via configuration in order to minimize professional services costs and speed up time to deploy.

Data Exports

Data export is in essence the inverse of a data import operation and needs to offer the same degree of power and flexibility as data input mechanisms. Businesses may need to extract data for various reasons; these include extraction of usage data for analytics, financial data for export into accounting systems and extraction of PII data for compliance reasons. At an operational level, data might need to get exported from one system to another such as the export of new rates from a billing system into a CRM system. Managing the export of data in a seamless and performant manner is important to the seamless operation of a back-office ecosystem. Extraction of usage data in particular can be quite problematic from a scalability perspective given the volume of usage data that can collect over time.

Import, Export and Regulatory Considerations

With the advent of regulations such as GDPR, we are seeing an increasing amount of scrutiny on how data is handled within systems. As data privacy and security gain greater traction, regulatory agencies are playing an increasing role in governing how data is treated, imported and extracted by systems. Import and export mechanisms need to treat data in a secure and compliant manner to meet these new regulations.

How LogiSense Can Help

LogiSense billing has been envisioned as a platform with a powerful set of capabilities for data import, export and transformation. Data profiles can be set up to import data, set up dependencies and transform the data. Data profiles and associated transformations can be set up through configuration which offers an economical alternative to code-based customizations.  Data can be grouped in a transactional manner for imports and exports via data flows. The reporting and invoicing engine facilitate seamless export capabilities of data in CSV and PDF format.