The world of business systems is expanding, and with it, the need for integrating those systems. It used to be that there was one main system, the system of record which was the data processing or ERP system. But now business technology is expanding – there are systems of engagement, dealing with communication and collaboration; systems of insight, providing analysis; and coming soon, systems of automation, providing connection to the world of things. This means that CIOs and IT managers should start looking at data integration as a key capability, and with that understand how to handle the common issues of data integration.
Data integration Dont’s and Dos
It’s tempting to code your own integration processes, but while this may work initially, it becomes increasingly difficult as integrations increase and get more complex. An integration platform-as-a-service (iPaaS) makes workflow management and orchestration easier, and produces results quicker than your average developer can code. An iPaaS also allows an integration team to share and collaborate on integrations much more easily than hand-written code. iPaaS is becoming the tool of choice for integration because it provides a comprehensive solution for a wide variety of integration issues.
An integration capability should include the following:
- Allow the different systems to share data
- Manage how data is exchanged between systems
- Provide control and monitoring of data interfaces
- Tools that
- are scalable as systems and data expand, both on-premise and in the cloud
- are reliable and robust, able to tackle any integration situation encountered
- provide secure encrypted communication for data exchange, and ensuring data integrity
- enable trackability of the data through the integration process
Data integration issues
What are some common issues of data integration?
- How to manage batching of data
- How to efficiently update data
- Tracking item status
- How to handle ever-evolving requirements that lead to ongoing changes
The Flowgear iPaaS allows both developers and citizen integrators to deal with these common integration challenges. Here are some tips for using three nodes that represent specific tasks in data integration. With each node is a short video that shows how the nodes are used. These videos come from a longer webinar on data integration best practices.
Flowgear integration tips
Splitter
The Splitter allows you to batch data and process each batch in the same way. This is useful if a large volume of data needs processed, so batching it makes the data integration easier for an application accepting the data as well as allowing data errors to be more easily picked up. Read more here.
Watch the Splitter video
Reducer
This node removes removes data that has not changed since the last integration and is very useful for efficient updating of master data by reduced processing time and reducing bandwidth. Read more here.
Watch the Reducer video
Statistic
With the Statistic node you can monitor the status of a data item, for example a sales order. Groups allow diffferent stats to be logged against a data item. Read more here.
Watch the Statistic video
Handling requirements changes
As for the issue of ongoing changes from users’ continually changing requirements, the Splitter video shows how easy it is to identify invalid or incorrect integrations, and how quick changes can be made.
What other integration issues would you suggest are common? Have you had a look at Flowgear to see how it would handle those issues?
