With various other powerful features and functionalities available in Mule 4, another popular and widely used functionality provided by Mule 4 is its Batch Processing capability which allows processing bulk data in batches in parallel in order to achieve better throughput and efficiency when dealing with large sets of data.
In this tutorial, you will learn how to work with MuleSoft Batch Processing by using Batch Job Scope. You will also learn how to use Batch Aggregator to aggregate batch data sets and how to use Accept Expressions for conditional execution of additional batch steps.
MuleSoft Batch Processing: Use Batch Job, Batch Steps, Batch Aggregator for Bulk Data Processing
When dealing with large sets of data, It becomes a performance bottleneck when such data sets are processed in a sequential manner and such approach also results in performance degradation with higher memory utilization, lower throughput and longer response times. To cater for such situations, batch processing of such large data sets is an optimal approach which can greatly help dealing with such situations and process large data sets in batches with parallel processing.
MuleSoft Batch Job Processing Phases:
Batch Jobs in Mule 4 contain multiple phases where different types of actions are performed in each step. Below are the steps of a Mule 4 Batch Job:
* Mule 4 Batch Processing Load and Dispatch Phase
For any batch job, Load and Dispatch is the first phase which gets triggered implicitly without any manual intervention or action from your side. During this phase, a Batch Job Instance is created and a persistent queue is created. Data is split into records and sent to the persistent queue in this phase.
* Mule 4 Batch Processing Process Phase
Process Phase is an important phase during which processing of the batch records takes place. During this phase, batch steps are executed for each record picked from the persistent queue.
You can have multiple batch steps in your batch job and batch steps are executed for each record which is getting processed (unless there is an accept expression which doesn’t get fulfilled for any record).
* Mule 4 Batch Processing On Complete Phase
This “On Complete” phase provides a summary of the processing and you can use this phase to generate any summary reports and send/share those summary reports with any other systems or parties as per your business case.
MuleSoft Batch Processing is an interesting topic and you may refer to official MuleSoft documentation about batch processing to learn more about advanced technical details of Batch Processing.
