Open Framework, Information Management Strategy & Collaborative Governance | Data & Social Methodology - MIKE2.0 Methodology
Wiki Home
Collapse Expand Close

Members
Collapse Expand Close

To join, please contact us.

Improve MIKE 2.0
Collapse Expand Close
Need somewhere to start? How about the most wanted pages; or the pages we know need more work; or even the stub that somebody else has started, but hasn't been able to finish. Or create a ticket for any issues you have found.

Enterprise Mashups - A Primer

From MIKE2.0 Methodology

Share/Save/Bookmark
Jump to: navigation, search
Author
[[Image:{{{imagename}}}|150px]]
Name: Michael Loke
Organisation: BearingPoint
Title: Consultant
Contact: michael.loke at bearingpoint dot com

Contents

Introduction

Internet/web applications have taken a fundamental shift in the way they are used since the late 90s. They’ve evolved from being mainly static, “Web 1.0”, read-only applications to dynamic read-write capable web applications. The shift has been coined as “Web 2.0” where user generated content made up a vast amount of the content on the internet today. Example of such applications would be blogs, wikis and user generated mashups that leverage external APIs to mix and match data to form an application.

Purpose

The concept of Web 2.0 and Enterprise Mashups have been taken seriously by major corporations worldwide (i.e. Westpac Bank ). The purpose of this paper to study the feasibility on leveraging the concept of lightweight, web-based, composite applications created by sourcing capabilities from established content and systems functionality, and how a company can sustain a competitive edge by allowingend-users the freedom to mix-match/screen-scrape information to build their own applications instead of relying on heavy-weight applications typically invested in heavily by IT.

Web Architecture

2-tier, 3-tier n-tier!!!

Traditional web applications have been based around the concept of client/server, two tier architectures where servers wait remotely to service HTTP requests sent from the client. This concept hasn’t changed at all today. However, it has been extended across to n-tier architecture which is briefly described in the next section. A sample example could be a simple two tiered chat server where a central server handles all the messages sent from the client and takes responsibility in routing the right message to the right client (similar to the function of a data access layer in the OSI network architecture).

In modern, complex IT systems, we have more than 2 tier. For example, following a typical MVC (Model View Controller) design pattern, we will normally have a front-end serving web application that communicates with a middle layer application server. The controller in the application server serves client requests from the views and then passes it on back to back-end server components that responds to such requests. Typically, the backend server will communicate with the DBMS (Oracle, MySQL..etc) to do the appropriate CRUD (Create, Read, Update, Delete) operations in the DBMS. In the J2EE programming framework, this is typically handled by remote objects such Enterprise Java Beans (EJBs) which ensure safe atomic transactions. The figure below represents a typical 3 tier architecture setup comprising a web front end, application servers and DBMS at the backend.

J2EE.jpg

This is an overly simplified explanation on the architecture. There are numerous resources that describes this in far more detail in other sources such as Sun’ J2EE reference guides.

Evolution of the Web

While the above model works well, the limitations are obvious. First of all, the components in the application itself are tightly bounded. CRM & billing software, such as Siebel, typically leverage such a model. Agents that need to be integrated into this class of applications require tight integration by leveraging the exposed internal APIs of application. The typical enterprise application also makes use of such architecture.

SOA as a premier to loose and easier integration

Software developers have known that decoupling multiple , dynamic systems is best practice when architecting a solution. Most functionality that needs to be decoupled can be exposed via an integration service layer that connects the two decoupled systems together. Companies such as Tibco provide the middle integration layer capability for enterprises that need to perform Enterprise Application Integration. In this case, an architecture will typically look like the one below :-

EAI.jpg

Messaging technologies such as IBM’s MQSeries or JMS (Java Messaging System) are used to send and process messages. These messages typically adhere to the SOAP protocol or their own proprietary message formats..

In SOA, systems can be loosely coupled through the use of web services. The underlying communication protocol is SOAP. An attempt to fully describe SOA is out of scope of this whitepaper, as SOA itself is an area of specialization and warrants a full library of research paper on its own. However, at a high level, these web services are used as an independent means to extract/perform manipulation of information requested by a client application. Such web services also implement business logic that utilizes information sources that are defined at backend DBMS systems. A completely SOA-enabled enterprise will have all of its functionality loosely coupled that can be accessed individually when needed. This improves clarity on the underlying pieces of functionality, and also provides ease of maintenance (instead of maintaining a whole application, only a web service is maintained and this typically represents a small, modular piece of code).

SOA Diagram

Below is a simplified diagram of a typical SOA environment. As you can see, there are 2 distributed applications that expose 5 separate web services that can be invoked over the wire by different client applications. The elements in the application are loosely coupled and this allows easier, platform independent integration as opposed to the traditional, tightly coupled, API-based, platform lock-in method explained previously.

SOA.jpg

Web 2.0 / Enterprise 2.0

With a brief understanding on how the web has evolved, this section describes how the evolution has caused a shift in fundamental web usage patterns. The term “Web 2.0” began with a conference brainstorming session between O’Reilly and MediaLive International. Web 2.0, in a nutshell, represents the concept of the web being both a platform and the collaborative elements associated with it. The following diagram is extracted from Tim O'reilly's Web 2.0 definition

Web20MEME.jpg

Enterprise 2.0, however, is a term coined for web 2.0 elements/culture residing behind the firewall. The below diagram represents a common view of the crucial elements that are deemed necessary for an Enterprise 2.0 application to work. This diagram was extracted from a prominent blogger in the Web/Enterprise 2.0 space, Dion Hinchcliffe.

SLATE.png

Vendor behemoths like IBM and Microsoft have recently taken interest in this space. For example, IBM recently released their IBM Lotus Notes 8 that promotes all the collaborative elements mentioned, while Microsoft struck a deal with Atlassian to integrate their WIKI into Microsoft’s Sharepoint.

Problems

The points below represents some of the challenges/problems that organizations face today in the context of information management.

Data

Data residing behind the firewall represents the brain of the enterprise. Information is the most valuable asset to a knowledge worker. Traditional data access methods, such as APIs, provide limited access to the functionality within the systems such as a CRM / ERP. This also requires the user application to be tightly binded with the parent applications.

The Search Problem

A quick browse to most corporate portals will show that searching for information is a big problem for an enterprise (and also the most of the consumer retail portals). It has been well recognized that knowledge workers spend over 10% of their time searching for information, and this can present a serious hurdle to the knowledge workers doing their job efficiently. In another words, most knowledge worker ended up searching for information rather than using it to add value to the business. We shall see how a good search product/mechanism, coupled with Enterprise Mashup will solve this problem.

IT Maintenance Cost

The application written/invested and typically maintained by IT department in large enterprises incur a huge costs due to various reasons. Issues such as scalability and functional enhancements will have a maximum impact on IT business systems that are tightly integrated. For example, a small CR (Change Request) involving a change in some simple interfaces could costs a lot of money, due to the impact assessment needed on other parts of the system. We shall see how Enterprise Mashups can significantly lower the cost of maintenance and help eliminate the costs of change requests, if a change request is ever needed at all when all applications are self-generated.

Information is disintegrated from main user

A corporation/enterprise normally has structured and unstructured data residing in various repositories. Each application can draw from both types of data. The needs of information from the top of the corporate ladder to the bottom are diverse and different. There are multiple ways to approach this demand.

1. The first is to deploy multiple custom applications (by IT department) that suit different needs. This approach incurs maximum costs as needs and requirements changes with the business environment. As the number of applications grows, more maintenance work will be needed.

2. The second is to build a one-size fits all application that suit the general, wider audience. This approach is typical of a enterprise portal. However, knowledge worker will not have the freedom to define which information they want as their needs are often “based on the general demand” that IT had estimated before building the application.

Regardless of which approach is followed, there is no effective way of tailoring information to suit particular users. Budgetary constraints would render this an impossibility.

Deployment Costs

Deployment costs are associated with the implementation of a new software/software updates. By having a centric approach towards application development/maintenance, the deployment costs are potentially huge. i.e. what you build now may not be relevant anymore, and in 6 months time, another build is needed and SDLC costs will add up again.

Enter Enterprise Mashups

What are Enterprise Mashups

The concept of screen scraping information from multiple sources is not a new idea. Enterprise Mashup takes the current form of screen-scraping information to a new level.. With the paradigm of a static web-site shifting to a full-blown web application that has the capability to deliver real-time dynamic data, in practice, we are able to source information from multiple locations and compose relevant and useful applications that cater to the needs of a single individual. This concept of dynamic aggregation, applied in the enterprise, is known as Enterprise Mashup.

iGoogle

Before jumping into a full discussion of enterprise Mashups, it is useful to visit how this has been done in the public space. The diagram below shows Google’ Mashup services (iGoogle) that allows users to attach “Gadgets” of any type into their own personal information portals.

IGoogle.jpg

Here we see a small amount of gadgets deemed useful and of interest aggregated on a single web page. This reduces effort and time spent to scourge for the information manually from multiple sources as it’s all contained within a single location. Google’s gadget API is based on standard CSS/HTML/Javacript and the contents are rendered by XSLT that has to conform to google’ standards. Google’s Code website contains the detail towards creating a render-able gadget.

One page, all relevant information sources, all delivered at once.

Blog Mashup

Below is a part of a page extracted from a personal blog. As demonstrated below, it contains mashups from Twitter API, Skype API and LinkedIn API.

Blog mashup.jpg

One page, all relevant information sources, all delivered at once.

The Twit updates are used to inform the reader what the author is up to at the moment. The SkypeMe Mashup is for readers who are interested for a chat with the author while reading his blog. The Linkedin profile Mashup is used to link users who are interested in the author’s professional profile.

Foundations of Mashup

This section describes the foundation Enterprise Mashups.

How do we communicate with the backend ?

In the world of mashable API, a different approach needs to be taken as opposed to the traditional object remoting method used in various Java and .NET applications. With XML becoming the data structure standard for integration, XML HTTP APIs will seemingly look like an obvious alternative to object remoting. There has been debate about object serialization and deserialization (i.e. translating an object into XML messages and vice versa) impacts on performances, but this won’t be discussed here. For lightweight web applications that are based on widgets or Mashup components, XML over HTTP is definitely a better choice.

The communication to the backend can be as simple as invoking a SOAP-based (whether RESTful or not) webservice in the enterprise application and then provide a callback method to format the return XML into presentable format. In the public web, a non-SOAP based protocol REST protocol is used. The following diagram illustrates this:-

Center

What is the data format?

The interfaces, as illustrated above, can be as simple as invoking a SOAP based Web Service, or a REST based web service. Information syndication technology such as RSS and ATOM can also be the source point of data. Normally Web 2.0 elements such as Blogs and Wiki provide these sort of syndication to allow content to be accessed ouside of the website. Access to structural data stored within databases can be accessed via various well defined APIs for security reasons. The response from the SOA services can be in the format of XML or JSON.


Below is an example on the JSON data format :-

JSON.jpg

As opposed to the XML one below :-

XML.jpg

Example is extracted from Official JSON Website. Traditional static document types such as spreadsheets, word documents, PDFs, static webpages should also be made mashable.

How should the data be displayed?

Data can be displayed as widgets that reside on a static page, handled by a layout manager that is responsible for drag-drop like functionality that places these widgets in a page. This presents a unified presentation of information that are deemed crucial by the user for their day to day operations.

What about a Mashable Platform?

A central mashable platform/server will be needed to enable users to create their reusable services dynamically for content-mashup purposes later. This is the key technological enabler to Mashup. Products such as Kapowtech Mashup Server and Microsoft’s Popfly has provided the means for users to Mashup content on the fly from different information sources. It acts as a central platform for managing mashable servcies. The diagram below depicts the high level overview of such a potential platform:-

Mash Platform.jpg

Mashup Repositories

This is where Mashup widgets or user-created mashup services are stored.

Example Business Use Cases

Below are just several use cases where the power of mashups can been seen being utilized

1) A consumer Mashup portlet (from Fire Department) containing the below composite information :- a. Google maps showing areas with fire b. Twitter announcements that are subscribed to the fire department twitter service (similar to a RSS feed, or can be one)

2) A consumer Mashup portlet from a Financial Institution that contains the following Mashup :- a. Latest stock updates b. RSS news headlines from the banks / financial world c. Widget interface for communicating directly with the relevant parties in the bank (or CSRs) d. Status of bank loan / credit application

3) A sales manager’s self-defined portal that contains the following information :- a. Latest analytics on sales b. Client relationship management information displayed via a portal c. A list of TODOs defined by the sales manager d. Calendar widget showing work schedules

4) A technical lead’s information portal showing the following information :- a. Project Schedule b. Bug tracking information


Each of this separate composition is based of the user’s perception of what’s important for them, not based on anyone else’s. This enables quick strategic use of the relevant information to improve business performance.

Existing Products in the Market

1) IBM QEDWIKI
2) Microsoft PopFLY (integration with sharepoint)
3) Dapper.net
4) KapowTech Mashup Server

Business ROI on Enterprise Mashups

ROI.jpg

Note: Diagram is just for illustrative purposes.

As with any IT implementations, the realization of ROI within a quick turnover period of time is crucial as enterprises would not want to invest huge chunks of capital into a worm hole.

While the quantification of ROI is subject to every corporations’ purpose-of-use and their individual cost cutting metrics in measuring profitability, it is clear that the model of information access through enterprise mashups gains an upper hand as compared to the old model described in earlier sections.

The actual ROI will not be discussed here, as they are controversial, and as mentioned above, depends on the organizations profitability/cost-savings measurement metrics and also the operational/strategic goal the organization tries to achieve with SOA/Mashups

Enterprise Search to play a key part (Google Search Appliance)

GSA.jpg

As with all successful use of information, the ability to search efficiently is crucial. The Google Search Appliance exposes its search functionality via OneBox modules that are defined within the system for searching enterprise wide. The search functionality is highly customizable and integration modules can be written to extend the GSA to retrieve information from different sources.

For collaborative environments (where all uses are allow to rate/vote/exchange widgets), the ability to search for reusable services and Mashup widgets are important.

Dapper.net illustrates the power of such asset retrieval.

Dapper.jpg

Enteprise Knowledge Market

Widgets, just like everything else in the mashups, are mere representation of information. The concept of voting/assigning quotes as described by Jeremy Thomas can come into good play for Enterprise Mashups.

Similarly, Mashup widgets can have the same analytics assigned to them. iGoogle and Yahoo has employed the same system (provide screenshots) that rate “the top 10 widgets”.

Yahoo.jpg

These assets can be ranked and helps provide visibility to other users of the ecosystem. It also lets users make better informed decisions about what services they should incorporate into their Mashup.

BI

BI analytics information attached to widgets can also reveal the pattern of information access/usage. Predictive analytics has been widely used in the insurance / banking / telco industry to study user patterns of information. Studying user patterns / Mashup patterns can also help organizations to be more proactive than reactive in retaining customers or working on new strategies to improve sales figures.

Mobile Enterprise

Since widgets make use the web standard technologies like CSS, HTML and Javascript, AJAX, mobile phone adoption could be made easier as well. With the power of today’s browser (IE in Windows Mobile and Safari in iPhone), the display of widgets are possible with some optimizations needed to suit specific phone (i.e. iPhone uses SafariUI Kit to enhance web applications for iPhone usage) for user-experience enhancement purposes. Otherwise, most phone browsers or MIDs (Multimedia Internet Devices) can render most webpages fine.

Other considerations

While the adoption of Enterprise Mashups sounds enticing, there are considerations that corporations need to evaluate before taking this approach. The following questions represent minimal guidelines that Enterprises should follow when deciding to adopt SOA and Enterprise Mashups :-

1) Is information access currently made difficult for knowledge workers?

2) Does the company have a long tail of unused applications/information, that when used collectively, can yield beneficial results?

3) How free should information be allowed to be mashup and access ? (security access controls come with some industry tools such as Kapowtech’s Kapow Mashup Server)

4) Is SOA inline with the long term strategic goal of organization? Does it necessarily yield a positive ROI ?

5) What is the financial capacity of the organization, considering that SOA initiatives normally span over a long period of time?


Relevance to the MIKE 2.0 Information Management Framework

MIKE 2.0 represents information management framework for organizations. The design principles of information can be leveraged in the strategy/design phase of a SOA project to prepare for Enterprise Mashups.

Summary

SOA and Enterprise Mashup itself is not a new technology/idea. The idea of distributed and loosely coupled information sources for ease of access has been floating for a while. AJAX and REST has taken the web by storm and its advantages are very obvious (starting from google maps, to the ability to create mashups leveraging the various APIs out there that exposes the functionality in terms of REST services and returns XML upon query), enterprises should consider a different approach to enterprise architecture. The key advantages are reiterated in the points below:-

1. The enablement of strategic use of important information at any time will drive business performance as employees spend less time scourging the enterprise for important information needed to make key decisions are available.

2. Enterprise Mashups gives every single the opportunity to mash up information that they deem useful to their every day operations. Targeted use of such information can yield a positive ROI in the long run if the usage of Mashups matches the strategic goal of the organization.

3. The “broken-up” enterprise will also help drive down cost in maintenance and application development for IT department. Applications are “no-longer” 100% created by IT as they can be mashed up by users.

4. The principle of Enterprise Knowledge Market, when applied in the enterprise system, will yield high quality mashup widgets which can be reused by knowledge workers within the company. The concept of “selfishness” fuels “participation” for “recognition” will also work well in the Enterprise Mashup ecosystem as mashups are simply are of the collaborative elements in Web 2.0 philosophy.

5. Enterprise Mashups do not need deployments as all applications are “designed” and “deployed” by the user at runtime

Wiki Contributors
Collapse Expand Close