Blog Post

Introduction

What is Pagination?

Pagination is the process of dividing a large set of data into smaller chunks or “pages” to make it easier to manage and display, particularly when working with databases or APIs. It’s a common technique used in web development to handle large volumes of data efficiently and improve user experience.

For example, suppose a database query returns 1,000 records. In that case, pagination can be used to display 10 records per page, allowing users to navigate between different pages (Page 1, Page 2, etc.) rather than loading all 1,000 records at once.

In this blog, we will see how to implement the pagination in MDH with the help of the repository APIs.

Prerequisites to achieve Pagination:

1. Create a Repository.
2. Create a source.
3. Create a model.
4. Attach the source with the model and add some data quality steps.
5. After configuring the model Publish and Deploy it to the Repository.

Let’s jump into the platform and follow some steps to achieve pagination in Boomi MDH.

Steps:

Step 1. Go to platform.boomi.com

Step 2. Login by giving your valid credentials (Username and Password).

Step 3. Home page will be displayed, now click on “Integration”.

Step 4. Click on the “Create New” button.

Step 5. Select “Process” and click on “Create”.

Step 6. Click on the “No Data” option and click on “OK”.

Step7. Drag and drop an HTTP client connector. Click on the “+” in connection to configure the connection.

Step 8. Give the URL, Auth type, UserName and Password.

URL:Go to Atom Management à Click on “Test” atom cloud à Click on Shared web server à Copy the Base URL à Add “/mdm” at the end.

User Name: Go to MDH à Select the Test repository à Configure Tab à Copy the Username

Password: Go to MDH à Select the Test repository à Configure Tab à Copy the Auth Token

• After giving all field values Save and Close, now it’s time to configure the operation.

Step 9. Click on the “+” button in the operation to configure the operation.

1. Request and response profile should be None
2. Content type is application/xml
3. HTTP method is POST
4. Add a Request Header “Accept” that is “application/xml”

Step 10. Configure the resource path (universes/<universeID>/records/query). Set  universeID as replacement variable.

Step 11. Now go to your MDH service. Click on ServicesàMaster Data Hub. Click on your repository. There should be some golden record present in your repository (if not then upsert  data to MDH first).

Step 12. 

Click on your model à Click on the “Sources” tab à Now copy the universeID from here.

Step 13. Add a set properties shape before the HTTP client connector and set the universeid in ddp.

Step 14. Before the set properties shape add a Message shape to generate the request body for the API.

We will provide the offsetToken from the dpp, which we will set after the API call. I kept the Limit as 2, that means in each call we will retrieve 2 records.

Step 14. Drag a set properties shape and set a DPP, the value will come from the MDH query response.

Step 15. Add a branch shape with 2 branches.

Step 16. Add a Data process shape to split the document.

Step 17. Add a “Add to Cache” shape to add the documents in a document cache.

Step 18. In the 2^nd branch add a Decision shape to check if the offsetToken is not null.

Step 19. Join the true branch back to the message shape.

Step 20. In the false branch add a Retrieve From cache shape to retrieve all the documents from the document cache.

Step 21. Add a Data Process shape to combine the documents and finally a Stop shape.

Step 22. Now save and test the process.

As we set the limit as 2, there will be 2 records and a unique offset token in each call. At the last call, the offset token value will be null, meaning no more records are left in golden records.