Monday, December 23, 2013

Book review : Enterprise Integration with WSO2 ESB

I had the pleasure to be a technical reviewer for the book, Enterprise Integration with WSO2 ESB from Packt Publishing. This book describes few common enterprise integration patterns using WSO2 ESB. Below are some notes from a technical point of view regarding the book. 

Due to the fact the book is a self introductory text and starts with a gentle introduction to WSO2 ESB, it can be followed easily by any new comer. It doesn't assume any previous knowledge on the product. Due to the richness of configuration language of the product, readers don't need to aware of any programming language. The book starts with a guide on how to set up the product so that the readers can try out the configurations in later chapters.

The book covers the most of the common integration patterns such as(but not limited to) content based routing, dynamic routing, scatter and gather, publish/subscribe, service chaining and detour. The book also provides complete ready to use configurations for message transformations, enterprise messaging, protocol and transport switching. And also the book provide complete configurations for each scenario that it describes. These configurations can be used as the building blocks of the real world use cases. Because of each chapter is independent from each other readers can read only the required materials and start solving the business use case using WSO2 ESB quickly and without much effort. 

Finally the book also covers how to deploy WSO2 ESB in a production environment and also how to fine tune for the same environment for optimal resource utilization and high performance. 

Sunday, August 4, 2013

AMQP transport is now ready

The AMQP transport implementation for Apache Synapse and it's now ready for production use. This is an overview of the new transport here. This transport is now part of Apache Synapse and will be available with the next release. 
The transport has an interesting list of features. It has a rich set of configuration options which can be used to configure the various bits and pieces of the transport. The feature list includes; 
  • Producer/Consumer. 
  • Publish/Subscribe(fanout exchange).
  • Worker queue(fair dispatching of worker task in a round robin manner).
  • Routing(direct exchange).
  • Publish/Subscribe to a topic(topic exchange).
  • Transactions support and message acknowledgement
  • Two way messaging support.
In order to achieve better performance the transport caches AMQP objects(connections and channels). The connections and channels are cached at the producer and consumer side. The polling task has an asynchronous architecture where I/O task actually poll the message from AMQP broker and place into an instance of BlockingQueue(in the implementation an instance of LinkedBlockingQueue was used). Once the message is placed into this buffer a separate worker task will actually process the message. All these worker pools can be configured for the requirements. At the sender side it's  recommended to remove the parameter 'ClientApiNonBlocking' for high load deployments in order to avoid creating a new thread for each message that send out. 

The polling tasks are based on ScheduledExecutorService. Configuration parameters are defined to configure the scheduled executor service. The transport is also equipped with an high availability implementation. If the broker goes offline this task will schedule a heart beat task which will tries to connect to broker in an exponential back-off fashion. This high availability task will suspend the current polling tasks until a successful reconnect attempt is made. This make sure that the Apache Synapse doesn't run out of disk space because of continuous logging of failing polling tasks. 
The transport was profiled and exposed to heavy load to test the stability. I just pumped 10,000,000 messages and the system was able to process all of them smoothly. Below is the screen shots for  memory and threads usages of the system under heavy load. It can be seen that the periodic GC cycles with a stable memory(after couple of repeating patterns the memory usage tend to become stable) and thread usage. 

Tuesday, June 18, 2013

Native AMQP transport for Synapse is on the way

AMQP is on its way to become a popular, well-established messaging protocol. I had some time to put up a native AMQP transport for Apache Synapse and following features are ready as of now.

  • Producer/Consumer 
  • Publish/Subscribe(fanout exchange)
  • Worker queue(fair dispatching in a round robin manner)
  • Routing(direct exchange)
  • Publish/Subscribe to a topic(topic exchange)
The transport is based on the excellent RabbitMQ AMQP Java client library. The source code is here. In order to compile and build the binary just type 'mvn clean install'(you will need an installation of Apache Maven on your path). The manual.pdf has sample configurations which explain how to use those features. Other features such as two way communication, transaction support, durability and QoS aspects are yet to implement. Stability of the code to avoid resource(such as file descriptors, memory etc..) leaking is yet to test under heavy load. 

Friday, January 4, 2013

Installing skype 4 on Fedora 17

Skype can be installed with a single command with all dependency in Fedora 17. Download skype 4.0(skype- and use the traditional package manager to install it with all the dependencies.

yum install skype-

Wednesday, November 7, 2012

WSO2 Cloud Gateway(CG) 1.0.0 is avilable

WSO2 CG 1.0.0  is available for download. The release announcement is here, the original release notice is here. If you have feedbacks, questions post them here(you need to subscribe) or report them here.

Thursday, October 4, 2012

C++ CGAgent implementation

As you may know WSO2 Cloud Gateway(CG) and it's CGAgent pair is based on Java. Recently I had the pleasure of writing a CGAgent in C++ for one of the users of WSO2 CG. As described previously CG uses Apache Thrift for underline high performance reverse transport implementation. As part of this new C++ CGAgent I had to generate the C++ stubs for the Java Thrift server using the same IDL. CGAgent just log into the Java Thrift server via SSL and retrieves any request messages together with any transport header and hand over to the component which invokes the actual business logic. Once the response is available the C++ CGAgent will send it over to the Thrift server in CG server via the same SSL channel. 

This also demonstrate how to write Thrift based a Java SSL server and C++ SSL client and get them communicate over a SSL channel. I am planning to write a separate guide since it seems there is not much documentation around this.

Thursday, August 23, 2012

WSO2 Cloud Gateway(CG) 1.0.0 RC1 is avilable

WSO2 CG 1.0.0 rc1 is available for download. The release announcement is here, the original release notice is here. If you have feedbacks, questions post them here(you need to subscribe) or report them here.

The product was earlier known as WSO2 Cloud Services Gateway and it was renamed to WSO2 Cloud Gateway in order to expose the product's capabilities more clearly.

It is encouraged users to deploy WSO2 ESB to deploy pass through proxies for their private services and publish those proxy services into the WSO2 Cloud Gateway. In order to encourage this best practise WSO2 Cloud Gateway Agent is now shipped by default with WSO2 ESB.