Guide on selecting an integration platform

9 min readDec 27, 2021

When selecting an integration platform for an organization, one would have to select from a large number of vendors offering various capabilities. This article lists down common features offered by integration platforms under different categories and tries to give some guidelines on selecting an appropriate platform for the needs of your organization.

Integration capability

Protocols: Support for common protocols such as HTTP(S), FTP(S), AMQP, MQTT, Kafka, Websockets, grpc, etc.
Message formats: Support for message formats such as JSON, XML, CSV, etc.
Business systems: Integration with commonly used business systems such as Salesforce, SAP, NetSuite, MS Dynamics, Google Docs
Data sources: Support for accessing data from various data sources and using data in integration flows. Data sources can include relational databases (Oracle, MSSQL, MySQL, PosgreSQL), NoSQL databases (Mongo DB, Amazon DynamoDB, Azure Cosmos DB) and file based stores (CSV files).
Middleware integration: Integration with other middleware components used in the application environment such as messaging systems (IBM MQ, MSMQ, Amazon SQS, etc) and BPM systems (Camunda, Bonitasoft, etc).
Cloud to on-premise, on-premise to cloud and cloud to cloud integrations: Integration platforms can be deployed in cloud or on-premise (depending on the options provided by vendors). In any case, systems that need to be integrated and applications that need to access services can reside in the same cloud, in a different cloud or in on-premise data centers. Therefore, integration platform should provide mechanisms to connect with systems deployed anywhere (e.g. cloud based integration platform may provide VPN connections to on-premise data centers).
Industry specific integrations: Support for integrating with industry specific systems such as core banking systems, Electronic Medical Record (EMR) systems, etc and processing industry specific message formats (e.g. EDI documents for industry verticals, HL7/FHIR, etc.

Selection criteria: Support for common protocols and message formats is needed for almost all integration use cases. Look for a platform that supports out-of-the-box integration with systems and databases used within your organization. Even if a platform does not support built-in integrations with certain systems, it can be a considered if it provides sufficient extension capabilities to add new connectors and message formats.

Integration flow development

Integration steps: Pre-built common integration steps such as service invocations, message transformations, header manipulations, payload validations, conditional routing, parallel executions, etc.
Integration flow designing: Support for graphical flow development, preferably backed with an option for switching to code mode.
Programmability: Support for introducing complex logic into integration flows. This is usually supported by adding scripting steps (e.g. Javascript, Groovy) or by adding custom developed steps (e.g. custom integration steps developed in Java). Integration platform may natively support programming complex logic, if it is backed by a complete programming language (e.g. Choreo).
Data mapping: Support for mapping data from one format to another (e.g. one JSON schema to another JSON schema, JSON schema to an XML schema, etc). As complex data structures maybe involved, it is better if the platform provides visual tools and suggestions to support data mapping.
Testing and debugging capabilities: Capability for testing integrations within the development tool and generating clients/testing scripts. Support for stepping through integration flows and inspecting data items at each step.
Simulations: Ability to simulate API calls and backend services with various payloads/behaviors and record/visualize integration flows under the simulation.
ML/historical data based recommendation capabilities: Train ML models based on data collections (integration flow designs and integration execution data) and provide recommendations while designing integration flows.

Selection criteria: Graphical flow development, support for basic integration steps, data mapping, testing/debugging are required capabilities for any platform. In addition, programability is also useful for many complex integration flows. ML based recommendations and simulations can be useful for major integration projects where large number of flows have to be developed.

API Management

Integration platforms should provide functionality to expose developed services as managed APIs to external and internal consumers. Related features include:

GUI based API designing capability
Deploying APIs into multiple gateways based on functional, organizational or security requirements.
API portals for app developers to search, subscribe and get keys for APIs.
Enforcing approval steps in various API related operations in order to support better governance.
API authentication and fine-grained authorization (e.g. based on OAuth2/OIDC)
Rate limiting for API traffic
Exposing both synchronous and asynchronous services as APIs
Creating new APIs by grouping selected methods from existing APIs
API tagging and categorization
Support for API standards (OpenAPI, AsyncAPI)

Selection criteria: API management is required if services need to be exposed to external users/organizations or other business units in the same organization. External API management is required to enforce security and monetization of services. Internal API management is useful for large organizations where multiple business units needs to discover and consume each other’s services.

Security

Authentication: Enforce authentication for platform portals, devops tasks, service invocations, API calls, etc.
User management: Manage relevant user accounts (integration developers, API users, administrators, operations staff, etc) internally or by integrating with an IAM system.
Payload protection: Enforce message payload protection mechanisms such as SQL injection prevention, JSON/XML schema validation, JSON/XML element count/attribute count/tag length restrictions, etc.
Backend authentication: Support for authentication between the integration platform and backend systems using protocols such OAuth2, basic auth, mutual TLS, etc.
Data encryption: Ability to encrypt message data and stored data. It may also be useful to encrypt parts of message payloads.
Encrypting sensitive configurations: Ability encrypt passwords, tokens, API keys and other sensitive information used for connecting with backend systems and data sources.
Infrastructure level security: Network level security such as firewalls, restricted subnets, VPN connections, etc and ability to deploy components of the platform in relevant zones depending on security requirements.

Selection criteria: All these security capabilities are required for integration solutions. Some integration platforms provide such security features out-of-the-box, while some others provide those by utilizing third-party systems. Features such as integrations with third-party IAM systems will be required if an organization already uses an IAM system.

Scalability

Clustering: Ability to deploy load handling components of the platform as clusters of two or more instances depending on the load (i.e. horizontal scaling).
Auto scaling: Ability to automatically scale up or down cluster size of runtime components of the platform based on the load. Usually, this is achieved by utilizing autoscaling capabilities provided by infrastructure (e.g. AWS Auto Scaling Groups, Kubernetes pod autoscaling, etc).
Traffic partitioning: Support for separating traffic for different domains, departments, user categories, etc and handle such partitioned traffic using separate runtime component clusters.
Component separation: Support for separating platform components based on the function, so that load on high traffic components (e.g. API gateways, integration runtimes, etc) does not affect low traffic functions (e.g. admin operations, dashboards, etc).

Selection criteria: It is necessary verify whether the selected platform can handle the expected load of an organization. As integration solutions are long term investments, planning has to be done for several years ahead. Horizontal scalability allows the deployment to be started small and grow as load increases in future. Auto scaling capabilities are needed if the load expected to be fluctuated frequently. Traffic partitioning is useful to scale the solution beyond platform’s scalability limits.

High availability

Component fail-over: Deploying redundant/passive instances of critical components of the platform, which can become active in case of a component failure.

Data center fail-over: Maintaining redundant/passive deployments of the platform in a separate data center/cloud region, which can be activated to handle active traffic in case of primary data center failure.

Component independence: Ability for critical components of the platform to operate (at least with a limited functionality), even after failures of other components of the platform.

Selection criteria: This depends on criticality of solutions deployed on the integration platform. Most solutions will require at least component fail-over capability to allow some headroom for admins to fix failed servers, etc. However, many large organizations may require data center fail-over and component independence to ensure business continuity even after multiple failures.

Deployment options

On-prem/cloud/SaaS: Integration platforms may offer one or more of these deployment options. Here cloud means deploying the platform on hosting environments provided by third-party cloud providers (e.g. AWS, Azure).

Deployment technology: Integration platforms may support deployments on bare metal, virtual machines (VMs) or container environments (e.g. Kubernetes, OpenShift, Amazon ECS, etc).

Regional availability: If an integration platform offers a SaaS option (iPaaS), it’s deployment may be restricted to limited number of regions. Further, it may also be possible to deploy iPaaS in additional regions on-demand. In case of on-premise solutions, it may be necessary to deploy the platform in multiple regions depending on business needs.

Control over data flow and storage locations: Organizations may have restriction on sending or storing sensitive data outside organizational network, country or region (e.g. GDPR). Integration platforms may offer support for such restrictions by deploying certain components within required boundaries.

Support for multiple production and non-production environments: Any integration project use at least one production environment and one or more non-production environments (development, testing, UAT, etc). Therefore, integration platforms should support maintaining such environments and seamless transition among those.

Continuous Integration/Continuous Deployment (CI/CD): Automated CI/CD pipelines may be available for:

Deployment of the integration platform into different environments
Deployment of updates to existing deployments of the integration platform
Deployment of integration artifacts into the integration platform

Selection criteria: Some deployment options are decided by organizational policies as well (e.g. move to cloud as much as possible or keep everything on-premise). SaaS can be a better option for smaller organizations or for initial stages of projects (to start with a low investment). Larger projects can utilize container environments to simply the deployments. If an organization operates in multiple regions, regional availability and geographical restrictions on data may be required. CI/CD and multiple environment support is required for any integration project.

Observability

Service/API monitoring: Integration platforms can capture runtime details of all connected services/API such as service health, request rate, response time, users/applications of each service, services used by each user/application, traffic patterns of services, etc.
Integration flow analysis: Delays of each flow step, branching probability, errors occurred in different flow steps, etc.
Message flow analysis: Integration platforms can work with message tracing solutions (e.g. Jaeger) to monitor message flows within the organizations.
User activity (auditing): Platform can record all relevant user activities such as deployment/undeployment of integration flows, changes to artifacts, registration of applications, etc.
Message details: Details of each message can be captured, such as headers, message payload, source/target service, etc. However, some of these can introduce a considerable overhead, therefore have to be turned on only for diagnosis purposes.
Platform operations: Runtime details of the integration platform such as uptime, system errors, deployment of modules, etc.
Infrastructure monitoring: CPU usage, memory usage, network usage, load average, etc of VMs on which the integration platform is deployed. Usually these details are captured by third-party tools.
Integration with third-party monitoring tools: Ability to share monitoring data with third-party monitoring tools such as ELK, Prometheus, Splunk, etc.

Selection criteria: Almost all observability features are required for a complete integration solution. Integration platforms should support these monitoring capabilities either as OOTB functionality or by integrating with third-party tools.

Development support, services and training

Integration platform vendors may offer support, services and training on areas such as:

Integration flow/API development
Extension development (scrips, custom integration steps, connectors for unsupported systems)
Platform administration
Platform deployment and maintenance

Selection criteria: Although modern integration platforms provide graphical editors for low-code integration development, it may be required to implement complex integration flows either using low-code editors, adding scripting steps or by implementing extensions. Therefore, it may be necessary to train an internal team to perform these tasks or get external consultancy for the implementation.
In addition, If the selected platform is not a SaaS offering, deployment and maintenance support need to be considered.

Pricing

Pricing options offered by the integration platform vendor:

Price based on number of requests received by the platform
Price based on number of CPUs/cores used by the platform
Price based on number of deployed integration flows/APIs
Price based on used features
Overall price for the project or an organization (usually offered for very large deployments)

Selection criteria: This is an area where it is necessary to negotiate with the platform vendor, specifically if your integration project is a large one. For example, it may be not sustainable to go with request count based pricing, if your solution is expected to get very large number of requests.
Furthermore, for non-SaaS offerings, it is necessary to consider infrastructure and maintenance cost as well. In addition, it is possible to check with the vendor whether organization wide pricing can be negotiated if the integration platform is used for multiple projects within the organization.