MuleSoft Batch Processing Tutorial: Bulk Processing Using Batch Job With Batch Steps & Batch Aggregator

By | March 21, 2021

With various other powerful features and functionalities available in Mule 4, another popular and widely used functionality provided by Mule 4 is its Batch Processing capability which allows processing bulk data in batches in parallel in order to achieve better throughput and efficiency when dealing with large sets of data.

In this tutorial, you will learn how to work with MuleSoft Batch Processing by using Batch Job Scope. You will also learn how to use Batch Aggregator to aggregate batch data sets and how to use Accept Expressions for conditional execution of additional batch steps.

MuleSoft Batch Processing: Use Batch Job, Batch Steps, Batch Aggregator for Bulk Data Processing

When dealing with large sets of data, It becomes a performance bottleneck when such data sets are processed in a sequential manner and such approach also results in performance degradation with higher memory utilization, lower throughput and longer response times. To cater for such situations, batch processing of such large data sets is an optimal approach which can greatly help dealing with such situations and process large data sets in batches with parallel processing.

MuleSoft Batch Job Processing Phases:

Batch Jobs in Mule 4 contain multiple phases where different types of actions are performed in each step. Below are the steps of a Mule 4 Batch Job:

* Mule 4 Batch Processing Load and Dispatch Phase

For any batch job, Load and Dispatch is the first phase which gets triggered implicitly without any manual intervention or action from your side. During this phase, a Batch Job Instance is created and a persistent queue is created. Data is split into records and sent to the persistent queue in this phase.

* Mule 4 Batch Processing Process Phase

Process Phase is an important phase during which processing of the batch records takes place. During this phase, batch steps are executed for each record picked from the persistent queue.
You can have multiple batch steps in your batch job and batch steps are executed for each record which is getting processed (unless there is an accept expression which doesn’t get fulfilled for any record).

* Mule 4 Batch Processing On Complete Phase

This “On Complete” phase provides a summary of the processing and you can use this phase to generate any summary reports and send/share those summary reports with any other systems or parties as per your business case.

MuleSoft Batch Processing is an interesting topic and you may refer to official MuleSoft documentation  about batch processing to learn more about advanced technical details of Batch Processing.

Ajmal Abbasi

Ajmal Hussain Abbasi is Integration Consultant By Profession with 13+ years experience in Integration domain mainly with TIBCO products. He has extensive practical knowledge of TIBCO Business Works, TIBCO Cloud, TIBCO Flogo, TIBCO Mashery, TIBCO Spotfire, EMS and TIBCO ActiveSpaces. He has worked on a number of highly critical integration projects in various sectors by using his skills in TIBCO Flogo, TIBCO API Management (Mashery), TCI, Tibco Designer, TIBCO Business Studio, Adapters, TIBCO EMS, RV, Administrator, TIBCO BE, TIBCO ActiveSpaces etc. Ajmal Abbasi has experience with MuleSoft ESB as well. Ajmal Abbasi is also experienced in the area of API Management particularly with WSO2 API management platforms. Ajmal Abbasi is also experienced in developing solutions using Core Java and J2EE Technologies. You can contact Ajmal Abbasi for Consultancy, Technical Assistance and Technical Discussions.

More Posts - Website - Facebook - LinkedIn - YouTube

Leave a Reply

Your email address will not be published. Required fields are marked *