API Pagination

how to traverse large data sets

Cursor Based Pagination in List Endpoints

Ordergroove's APIs support cursor-based pagination, which we strongly recommend for fetching data from List endpoints. Cursor-based pagination ensures consistent response times and avoids potential issues with data integrity.

To use cursor-based pagination, include the following header in your request: X-OG-API-VERSION: 2

Sample Request Header

{
  "x-api-key": "{YOUR_KEY_HERE}",
  "X-OG-API-VERSION": "2"
}

By default, List endpoints return 10 results per response. You can adjust this using the page_size parameter, up to a maximum of 100 results.

Fetching Additional Results with Cursor-Based Pagination

When there are more results than the page_size limit, the response includes next and previous attributes, which contain URLs with a cursor parameter for fetching subsequent or previous pages.

Requesting the Next Page

To fetch the next set of results, use the URL provided in the next attribute of the response:
To fetch the previous set of results, use the URL provided in the previous attribute of the response:

Example https://restapi.ordergroove.com/subscriptions/?cursor=cD0yMDI0LTA0LTEwKzEwJTNBMjElM0EwMQ%3D%3D&updated_end=2024-06-26&updated_start=2024-06-24

Sample Response

{
"next":"https://restapi.ordergroove.com/subscriptions/?cursor=cD0yMDI0LTA0LTEwKzEwJTNBMjElM0EwMQ%3D%3D&updated_end=2024-06-26&updated_start=2024-06-24",
"previous":null,
"results":[{
  	"public_id": "f9cb2f93e1c845eb9de9eff46ddb3cbf",
        ...
        }]
}

Why Use Cursor-Based Pagination?

  • Consistent Performance: Cursor-based pagination provides stable response times, even for large datasets. In contrast, page number-based pagination slows down as the page number increases.
  • Data Integrity: It eliminates the risk of duplicate or missing records caused by data updates while the dataset is being retrieved.

Recommendation For Count

Cursor-based pagination does not provide the count attribute in the response which represents the total number of records matching the filters used on the endpoint
For user-facing interfaces (e.g., paginating orders or subscriptions), we recommend:

  • Allowing users to navigate to the next and previous pages of results.

  • Avoiding direct navigation to arbitrary page numbers, which is not supported with cursor-based pagination.

Page Based Pagination (Now Deprecated)

Page-based pagination is now deprecated but remains available for use. However, we do not recommend using it for datasets exceeding 20 pages. For larger datasets, we strongly recommend transitioning to cursor-based pagination, which offers better performance and data consistency.

When there are more results than can fit on a page, you can loop through the results using page query string parameter

Example: https://restapi.ordergroove.com/orders?page_size=100&page3

Risks of Using Page-Based Pagination

While page-based pagination may seem straightforward, it comes with significant drawbacks that make it less suitable for handling large datasets or dynamic data:

  • Performance Degradation: Response times can slow down significantly as you retrieve higher ordinal pages, especially for large datasets. This can impact the overall efficiency of your data retrieval process.
  • Data Integrity Issues: Page-based pagination is more prone to inconsistencies, such as duplicate or missing records. These issues often arise if the underlying data is updated (e.g. new records are added or existing records are modified) during the retrieval process.

For these reasons, we are deprecating Page based pagination and strongly recommend using cursor-based pagination for better performance and data consistency.


❗️

To ensure optimal performance and maintain platform stability, we now support up to 5 concurrent threads for API requests when utilizing pagination. We strongly advise against exceeding this limit, as it may lead to degraded performance or temporary throttling. If you have any questions or require assistance, please contact us at [email protected].