Adapter and EAI Computer Engineering Research Group PowerPoint Presentation

Slide 1 -

Copyright iConexio Technologies Canada Inc. Adapters and EAI Technology, Tools and Trends Presenter: Atul Apte President and CEO, iConexio Technologies Canada Inc.

Slide 2 -

Agenda Session 1: Background on software integration Background on adapters Overview of adapter technology Session 2: Overview of trends in adapter technology J2CA, Web Services, Semantic Adapters Overview of adapter development tools E-Fusion product architecture Q&A

Slide 3 -

Background On Software Integration Software integration is the #1 IT priority today It is also the most complex and expensive problem to solve About 30% - 40% of all IT costs are integration-related Approximately 1/2 of all integration costs are adapter-related Adapters represent the highest incremental cost of software integration Each software system that needs to be integrated requires an adapter

Slide 4 -

Background On Software Integration The primary focus of software integration is information systems (or business applications) The two main areas of software integration are: Data Function (Web Services) The two main types of software integration environments are: Point to Point (Application Server based) Brokered (Integration Platform based) The extended focus of software integration is business processes Integrating information systems with business processes Automating the integrated business processes

Slide 5 -

Background On Software Integration Components of Software Integration Platform Foundation components Host (Application Server, Integration Server, others) Messaging bus (JMS, others) Object brokers (OMG) Database engines (RDBMS, ODBMS, others) Transaction engines (XA compliant, others) Value add integration components Transformer Mapping, Object aggregation, Object decomposition Broker Routing, Filters Adapters

Slide 6 -

Current Perspective on EAI EAI is still a big, complex, expensive, tedious and repeating problem faced universally by companies of all sizes Approximately 1/3 of all IT costs are EAI related Approximately 1/2 of all EAI costs are adapter-related The definition of EAI is much broader today and includes software systems within and outside the companies firewalls The term “EAI” has been changed and misunderstood with the continuing evolution of EAI technologies

Slide 7 -

Current Perspective on EAI EAI is an ongoing initiative and requires strategic thinking, strong risk management, and realistic goal oriented implementation plans Industry studies have reported that the success rates of EAI project is very low < 5% Existing methodologies, project management practices & processes are not enough to deal with the unique and specific risk management requirements of EAI

Slide 8 -

Current Perspective on EAI iConexio has a strong commitment to solve one of the toughest problems of EAI namely development of custom/customized adapters and lowering the total cost of ownership The first step is to define EAI in a clear and simple form easy to understand, comprehend, and implement

Slide 9 -

Definition of EAI EAI is the process of integrating applications and database systems into a unified software system capable of supporting: On-demand data exchange Collaboration of shared functions Consolidation & Transformation of data into information Automating data synchronization Automating business process

Slide 10 -

EAI Domains The EAI processes are grouped into a set of logical processes (“Domains”) which simplifies the definition of specific EAI processes Application To Application Domain (A2A - Domain) The process of integrating two or more business applications Application To Information Domain (A2I - Domain) The process of integrating a business application with information producers (data warehouses), information access portals, etc. Application To Process Domain (A2P - Domain) The process of integrating a business application with one or more work flows, EDI gateways, supply chain automation networks, mobile workforce networks

Slide 11 -

EAI Domain Reference Models Every EAI Domain is supported by a context sensitive reference model that can be used to define, design, deploy a customized EAI solution Each EAI domain reference model contains: A logical architecture that defines the components and relationships of the EAI process A set of integration patterns that define the different behavior and interactions of the EAI process A technology stack containing layers of technologies and standards that enable the EAI process

Slide 12 -

Business Application Business Application Application to Application (A2A) EAI Database Database Function Function Data Integration Broker Adapter Adapter Web Services Broker Adapter Adapter SOAP XML Transformer UDDI

Slide 13 -

Information Portal Business Application Application to Information (A2I) EAI Database Database Function Function Adapter EII Server Adapter ETL Data Warehouse Portal Server

Slide 14 -

Business Application Partner / Supplier Systems Business Application Application to Process (A2P) EAI Database Database Function Function Adapter Workflow Engine Adapter EDI Gateway XML Database Function Adapter XML / EDIFACT Firewall WFMC

Slide 15 -

Data Replication Pattern A2A Patterns Data Change Detector Data Extractor Data Filter Data Source Data Target Data Loader Other Patterns Data Synchronization (Bi-directional Data Replication) Composite Data Extraction Collaboration of Shared Services

Slide 16 -

Information Creation Pattern A2I Patterns Data Retriever Meta-Data Creator Info Aggregator Data Source Information Sink Info Presenter Other Patterns Extract Transform Load (ETL) Data Staging

Slide 17 -

Task Delegation Pattern A2P Patterns Task Assignment Agent Task to Function Map Workflow Manager Business Application Function Manager Other Patterns Business Process Management

Slide 18 -

Generic EAI Solution Architecture Foundation (Host, JMS, CORBA, Transaction and Database Engines) Broker Transformer Adapter Information System Adapter Information System Adapter Information System Adapter Information System Adapter Information System

Slide 19 -

Adapter Technology Status Quo Adapters represent the highest incremental cost of integration Each applications that needs to be integrated requires a minimum of one to three adapters (A2A, A2I, A2P) The average Fortune 1000 company1 has 48 applications and 14 databases for a total of 62 potential adapters There are few specific tools and platforms available for development and maintenance of adapters 1IDC

Slide 20 -

Definition of an Adapter There is no Single Definition of an Adapter But there is a common understanding of the adapters behavior and role Adapters, Connectors, Plug-ins, Web Services, Integration Components and other proprietary terms are used in different context to mean “Adapter Type Of Functionality” Adapter definitions tend to be context sensitive

Slide 21 -

Generic Definition Of Adapter An Adapter is a software component which integrates two different instances of software systems Typically the software systems integrated are business applications (information systems) In a brokered integration environment a business application is integrated with the broker by using an adapter Adapter facilitates integration between the integrated software systems Provide connectivity between the systems Manage the interactions between the systems Enforce transformation and other business rules

Slide 22 -

Background On Adapters The concept of adapters was first introduced by the Enterprise Application Integration (EAI) vendors in the later half of 1990’s Adapters have become the most important aspect of software integration Every software that needs to be integrated requires an adapter The average Fortune 1000 company has 48 different business applications and 14 different databases¹ Adapters represent the highest incremental cost of software integration ¹ IDC

Slide 23 -

Background On Adapters Until recently there were no standards for developing and supporting adapters J2EE:JCA is the first system level standard designed to support adapters on the Java Application Server Major vendors are now supporting J2EE:JCA in their Java application servers Adapters are becoming a commodity System standards for adapters will result in more focus on integration functionality Multiple providers of the same adapter will give users opportunity to compare adapter features

Slide 24 -

Background On Adapters Generally adapters are custom developed from scratch Most information systems are proprietary or heavily customized The average cost of developing and maintaining an adapter for packaged applications like SAP, PeopleSoft, etc. over a five year period is over U$ 1 Million The adapter market has few tools available for developers The biggest challenge in creating adapter tools is the expertise and experience required to design, develop, and maintain complex adapters

Slide 25 -

Adapter Interactions An adapter use case defines a specific interaction between the two software systems integrated by the adapter An adapter use case is a sequence of one or more functional steps There are two types of steps in an adapter use case Integration function (Connection, Parsing, Transformation, etc.) Support function (System Logging, Error handling, Audit Trail, etc.) Two types of use cases Simple Use Case Involves only one adapter (two software systems) Complex Use Case Involves two or more adapters (four or more software systems)

Slide 26 -

Adapter Scenarios An adapter scenario is a specific instance of an adapter use case and is bound to a specific runtime environment Atomic Scenario (Derived from simple use case) Collaborative Scenario (Derived from complex use case) It is possible for adapter scenarios to change between atomic and collaborative scenarios depending on the runtime environment

Slide 27 -

Simple Use Case (Atomic Scenario) Simple Adapter Use Case SAP Adapter SAP API Client Check customer Add new order

Slide 28 -

Complex Use Case (Collaborative Scenario) Simple Use Case Complex Adapter Use Case SAP Adapter SAP Siebel Adapter Siebel API API Client Add new order Check customer

Slide 29 -

Adapter Technical Architecture Technology Stack Client Interface Service Gateway Document Manager Transformation Manager Scenario Manager Information System Manager Host Environment Manager Responsible for providing unidirectional or bidirectional access to adapter services Responsible for parsing, filtering, and formatting data and documents Responsible for managing single/multiple adapter scenarios involving one or more information systems Responsible for defining and implementing service contracts and access control Responsible for data type conversion, data mapping, object aggregation and object decomposition Responsible for managing connectivity to the information systems Responsible for managing the lifecycle and collaborating with the operating platform

Slide 30 -

Adapter Technical Architecture Technology Stack Client Interface Service Gateway Document Manager Transformation Manager Scenario Manager Information System Manager Host Environment Manager Optional Layer Optional Layer

Slide 31 -

Adapter Technical Architecture Technology Stack Client Interface Service Gateway Document Manager Transformation Manager Scenario Manager Information System Manager Host Environment Manager SOAP/UDDI J2EE:JCA XML / XSLT

Slide 32 -

Simplifying Adapter Complexity How to manage the technical complexity of adapters? Use model based software development techniques Pre-defined software models based on integration and design patterns will provide a robust framework for developing adapters Developing multiple adapters for one application Encapsulate fewer interfaces into one adapter instance Collaborate with external transformers instead of developing transformer functionality in each adapter

Slide 33 -

Implementation Analysis, Architecture and Design Adapter Models Abstract Adapter Model Structural Patterns Functional Patterns Conceptual Patterns Integration Patterns Concrete Adapter Model

Slide 34 -

Trends In Adapter Technologies Adapters are more sophisticated in functionality Web services enable application function level collaboration J2EE Connector Architecture enables plug-and-play support for adapters inside application servers Adapters can function in a point-to-point or broker based integration environment Standard adapter models will provide a consistent architecture and design for adapters Adapter tools will enable faster, cheaper, and easier development and maintenance

Slide 35 -

Impact of Web Services The most significant impact of web services is the shift in software application architecture Service Oriented Architecture “SOA” is becoming the foundation for new internet based business applications The concept of “Composing” business applications from existing software services Web services enables development of “SOA” compliant applications Adapter have an even more critical role in fulfilling “Services”

Slide 36 -

Impact of Web Services Where is the Synergy between adapters and web services? Web services provide a standardized platform for defining, locating, and invoking business services WSDL, UDDI, SOAP Adapter provide a standardized component mechanism for fulfilling the web services by accessing external software systems J2CA Both web services and adapters are moving towards a common semantic environment enabling rapid “composition” UDEF, RDF

Slide 37 -

What is J2EE Connector Architecture? J2EE Connector Architecture (J2CA) is a sub-specification of J2EE J2CA is a system level interface for hosting and accessing connectors to non-J2EE applications J2CA connectors extend the J2EE servers ability to interface with non-J2EE applications Most J2EE servers support J2CA specification IBM WebSphere, BEA WebLogic, Jboss, Oracle AS J2CA compliant adapters for popular applications like SAP, PeopleSoft, are now available in the market place

Slide 38 -

What is J2EE Connector Architecture? J2CA Specification (Ver 1.5) defines system contracts for managing security, transactions, and connections between J2EE application components and legacy systems Support meta-data interfaces and bi-directional access between adapters and legacy systems Message driven access to J2CA connectors is now possible

Slide 39 -

Moving Towards Semantic Adapters What is a Semantic Adapter? A Semantic Adapter provides an interface to the semantics of the adapter Adapter Behavior (Services provided by the adapter) Meta-data of the adapted systems data structures Meta-data of the adapted systems service interfaces Configuration properties of the adapters runtime environment A Semantic Adapter uses an Ontology based approach to define the semantics This involves associating commonly understood meaning to the definition of adapter properties, behavior, configuration, and associated meta-data

Slide 40 -

Adapter Development Tools Fundamental Requirements Enable just in time adapter development and maintenance Customers do not purchase adapters ahead of requirement End points (information systems) change and therefore pre-built adapters need customization too Extend the model based design (MBD) paradigm to adapters Model based development significantly reduces development time and cost, and increases adapter quality Differentiate and support abstract adapter models and concrete adapter models Enables multiple implementations of a common abstract adapter model

Slide 41 -

Adapter Development Tools Fundamental Requirements Enable adapter developers to focus on the business requirements of integrating two software systems Develop and maintain platform controllers which encapsulates the operating platform system interfaces from adapter developers Eliminates the burden of learning and programming system level interfaces and standards E.g: J2EE:JCA controller which encapsulates J2EE:JCA system contracts from the adapter developer Generate adapter framework and component shell from pre-defined adapter models Ensures design consistency, reduces bugs and enhances the quality of adapters

Slide 42 -

E-Fusion Adapter Technology Objectives: Solve the root cause of high adapter development and maintenance costs Provide the best adapter development tools and runtime platform Define the core technology architecture for adapters Define new adapter standards and methodologies Support relevant industry standards applicable to adapters Develop a single adapter technology framework capable of supporting different adapter interactions, programming languages and operating platforms Reduces the total cost of ownership for adapters

Slide 43 -

Overview of E-Fusion Adapter Tool iConexio has developed E-Fusion, a break-through, patent pending technology for Adapter development and maintenance E-Fusion reduces the time & cost to develop and maintain software adapters from 41% to 75% Rapid progression towards holistic integration of information systems for improved ROI A2A A2I A2P

Slide 44 -

Benefits of E-Fusion E-Fusion extends the Model Based Development (MBD) software paradigm and applies is to adapters E-Fusion provides specific platform controllers which eliminates 90% of the server/platform coding E-Fusion’s Development and Testing methodology significantly improves adapter quality

Slide 45 -

Benefits of E-Fusion E-Fusion sets new standards for adapters First to provide developers with design and implementation standards for the business functions supported by adapters First full life cycle adapter development methodology E-Fusion provides a Adapter Platform which includes all of the common adapter infrastructure E-Fusion enables developers to concentrate on the elements unique to each adapter project, independent of platform

Slide 46 -

Positioning E-Fusion E-Fusion provides a common adapter tooling environment across application platforms IBM platforms WebSphere Application Server WebSphere Business Integration Server (Crossworlds) MQSeries (Integrator, Workflow, Messaging) Mainframe and AS400 J2EE Application Servers CORBA ORB’s .NET Platform Legacy Operating Systems (UNIX, Linux)

Slide 47 -

Positioning E-Fusion E-Fusion provides a universal adapter hub across enterprise integration environments Business Integration Process Integration Application Connectivity Enterprise Information Portals Information Integration EAI Data Integration Web Services Application Collaboration

Slide 48 -

Integrated Collaboration Environment (ICE) Adapter Development Environment (ADE) Adapter Runtime Environment (ARE) E-Fusion Product Architecture Adapter Runtime Platform Adapter Deployment Tool Adapter Development Tool Adapter Repository Java IDE Generate Adapter Component Shell From Models Assemble & Test Adapter Define Scenarios Configure Adapters Run Adapter Scenarios Adapter Models Extends Code Application Platform Extends Register Components To E-Fusion Monitor Adapter

Slide 49 -

E-Fusion Adapter Technology Architecture: Adapter architecture based on the concept of functional layers Each layer encapsulates specific adapter functionality accessible as well defined services Each layer can be implemented as an isolated/distributed component of the adapter Architecture supports three types of interactions for an adapter Synchronization Keeping distributed data models in sync with each other Collaboration Enabling software systems to collaborate functions Delegation Enabling software systems to execute tasks on behalf of other systems

Slide 50 -

E-Fusion Adapter Technology Adapter Model Is a collection of one or more adapter patterns managed by the E-Fusion framework Integrated Collaboration Environment (ICE) E-Fusion supports two types of Adapter Models Abstract adapter model Represents the architecture and design of an adapter Adapters cannot be created directly from abstract adapter models as there are no implementation details defined Concrete adapter model Represents the implementation reference model of an adapter Includes one or more run-time scenarios Adapters can be created from concrete adapter models

Slide 51 -

Adapter Development Environment Adapter Development Environment (ADE) supports developers throughout the full lifecycle of adapter development and maintenance From design to maintenance Design, Develop, Assemble, Configure, Deploy, Maintain Develop pre-built adapter models and support custom adapter models Enable higher efficiencies in adapter development without changing existing development workflow and processes Integrate E-Fusion with leading IDE tools and extend the tools capabilities to support adapter development Define new adapter engineering and development methodologies which can be applied with generic software development tools

Slide 52 -

Adapter Runtime Environment Adapter Runtime Environment (ARE) supports distributed and centralized adapter deployments across multiple platforms Support dynamic distribution of adapter components across the platforms in the adapter runtime environment Support centralized administration, monitoring, and debugging of adapters at runtime Support dynamic language (locale) translation for multi-lingual runtime environments Support dynamic component code download and side-by-side runtime environments (production and QA)

Slide 53 -

E-Fusion Adapter Runtime Platform CPU/OS CPU/OS Application Server (J2EE / CORBA / Other) E-Fusion Adapter Runtime Platform E-Fusion Adapter Runtime Platform Repository Execution Manager Execution Manager Controller Controller

Slide 54 -

Summary Adapters are complex pieces of software performing critical integration tasks The definition of adapters continues to expand and support new application architectures (“SOA”) Adapter tools are now available for enabling rapid adapter development and deployment The trend is towards developing semantically and functionally rich adapters will continue to strengthen the role of adapters

Slide 55 -

Q&A Thank You