What is System Integration?

What is integration? How does systems integration work? Why do integration issues cause some many problems? How can I maximise system integration benefits? How can I mitigate or eliminate IT integration risks? Why is the process of integration so complicated? Is systems integration really as complex, costly and time-consuming as some technology service providers would have me believe?

The answers to all of these questions start with a definition. Understanding what IT integration is and knowing how to talk about it in a meaningful way is the first step towards developing efficient, effective and sustainable system integration solutions.

A pragmatic systems integration definition

System integration is the act of connecting disparate IT systems together to allow data, or event triggers, to flow between them to facilitate a broader end-to-end process. This systems integration definition sounds simple when you say it like that, but it can be incredibly complex and problematic. So why is system integration so hard? And why do so many IT integrations fail? System integration may be complex but it doesn’t have to be complicated.

All system integration solutions are built using a combination of relatively straightforward use cases. The six fundamental system integration scenarios are shown below:

Six integration types and common usage scenarios

IT integration mode 1 – Non-persistent data provision

Data is transferred from a source system to a target system for informational purposes or to be used in real time analysis or processing. This integration strategy is often used to show external data within an application’s user interface in a read-only format or to provide a data point that is used within an applications business logic or workflow.

IT integration mode 2 – Orphaned data provision

Data is transferred the target system but not updated to reflect changes in the source or donor system. This integration method is often useful to augment and enrich a transactional record with relevant information from external systems to enable historical trends to be analysed using data elements beyond the immediate content of the transactional record.

IT integration mode 3 – Linked data provision / Replication with inheritance

Data is transferred and subsequently updated in the target system to reflect any changes in the original record within the source or donor system. This ensures that data integrity across multiple systems is maintained and means that data only needs to be maintained in one place for it to proliferate throughout the application stack.

IT integration mode 4 – Bi-directional data link

Data is synchronised between two or more systems with updates on either side being reflected in the other. The concept of target and source systems becomes somewhat moot in this integration scenario as any system in the connected network can act as the originator or recipient of changes. Such integration models sometimes introduce the need to define reconciliation procedures to handle scenarios where changes are being made in all network participants and a hierarchy of trust or data precedence / provenance needs to be implemented.

IT integration mode 5 – Remote data update

A change to data in the target system is triggered remotely without the passing of data. The source system sends an event trigger which is acted upon by the target system and the data within the target system is subsequently updates. The target system may optionally send a response or confirmation notification to the source system to indicate that the event has been processed. Often used to communicate status changes or propagate inventory movements where the originating system doesn’t hold the data related to the item being amended.

IT integration mode 6 – Remote action initiation

An action in the target system is triggered remotely. The receipt of the event trigger may be acknowledged and the target system may provide progress confirmations as the resultant actions are performed or completed.

Integrating systems isn’t necessarily just about passing data and facilitating the process!

Except that it is. The six integration methods outlined above can be used to implement any systems integration requirement. Seriously! That’s it. It always comes down to these simple building blocks…

Sometimes it’s desirable to connect user-facing applications to streamline the user experience and prevent human-error related issues associated with the rekeying of data etc. The desire to push data to another application and to initiate actions that help the user to be more productive is commonplace. Initiating a “pop” from one client application to another whilst seamlessly providing contextual information to get the user to where they need to be as quickly as possible has been a common integration requirement for decades.

More recently the Robotic Process Automation (RPA) tooling market has grown on the premise of simulating the human user’s actions across multiple applications of varying technological heritages. Less than elegant techniques such as key logging, screen scraping, client-side integration approaches like OLE calls and basic macro recording have been used to enable defunct legacy applications to integrate with solutions based upon newer technologies. Whilst this has enabled businesses to defer costly legacy systems migrations it does have limitations and is not always the most appropriate approach.

If integrating systems is so simple, why do people struggle with it?

While the basic system integration methods are simple. An enterprise-grade system integration solution often involves many such actions being run in parallel or series with business logic defining what, how and when subsequent actions are initiated.

Breaking system integration business requirements down into their constituent parts is essential if a robust, stable and scalable integration solution is to be developed. Knowing what integration types are applicable to any give situation and understanding which system integration methods to use is not always obvious. The process of integration requires integration architects and developers to evaluate the various integration methodologies available and to determine the relative integration benefits associated with each.

Moving from the “What” to the “How” of IT systems integration too soon

People often get hung up on how an integration is going to be implemented before they have a real grasp as to what it is that the integration is trying to achieve. This is a fundamental mistake. Overly focusing on the integration methodology to be used often leads to inefficient or overly expensive integrations. The four integration architectures we see most often include:

• File transfer / ingestion (i.e. Extract Transfer Load (ETL) or the use of intermediary database tables / XML structures)
• Point-to-Point (via API calls or SOA services etc)
• Hub and spoke models that leverage an intermediary platform i.e. Middleware, Enterprise Service Bus etc
• Using “Quick and dirty” client-side integration tooling e.g. Screen scraping and user simulation (i.e. RPA)

The Multishoring approach to IT systems integration

Integration is what we do. Our team has witnessed all of the common IT integration failure modes and understands how to mitigate the risks, avoid common pitfalls and, ensure your integration project is successful. Whether we’re in a turn-around situation helping to get an off-the-rails integration project back on track, or working on a net new integration solution, we focus on the things that truly matter and make it work within the constraints you define.

Depending on your appetite for risk, available budget, or timeline, we will work with you to define the optimal schedule of works so that you feel confident in the delivered solution and its ability to run as designed issue free for as long as you need it. We don’t over engineer systems integration projects to artificially inflate the scope of works and we don’t skimp on activities that will materially impact the integrity of the delivered integration.

Integrations may be complex, but they do not need to be complicated. We use our experience and expertise to take challenging IT integration scenarios and make them happen. We do what is needed to make your integrations stable, secure and scalable so that you can focus on other things.

Share article on: