The biggest advantage of an API first platform is that you are able to connect that system into your other applications in a manner that meets your needs which then creates opportunities for business optimization and automation. 

That integration might mean something simple like when an order is placed in system A that you want to look up related information about that order in system B and then submit the results of the combined information to system C.

This orchestration allows multiple distinct systems to work together to solve business challenges but at the same time focus on their individual core competencies instead of trying to create a single monolithic piece of software that tries to do everything for everyone in the organization.

One of the challenges of integrating platforms together is understanding how to efficiently retrieve data. When data volumes are small it is easy to request all the data and sort through it manually in your code. However, as data volumes grow there is a desire to filter the data on the server to improve integration performance. 

Then, when dealing with extremely large data volumes, it makes sense to not only filter data on the server, but to request that data in chunks/pages so that an application can continue to request more information as needed.  This way you can try and pinpoint the data that you’re looking for.

In this article, we’re going to look at some different examples of retrieving data from LogiSense Billing v10.3.0 and how the newly introduced Search Paths can ease common integration challenges.

Typically, most API first platforms are written in a RESTful API fashion. This allows developers to use the internet to retrieve and save data in an external system using a standardized format. Typically to retrieve information developers use what are called a GET request to “get” the data about an entity/object in that remote system.  However, this can typically be pretty rudimentary, and trying to find particular information can be difficult.

To that end, Search Paths were introduced in LogiSense Billing v10.3.0 for each entity in the system and are available for all APIs that support standard RESTful GET requests. Now in v10.3.x, one Search request can replace up to four GET requests using a “deep search”, but more on that later.

Let's look at an example of each of the 3 major types of bulk data requests using the Account Package entity. This entity is used to store information about the products/packages that a specific account has assigned to them:

Scenario 1 - I want all the Account Packages and I'll filter them locally

• In v10.2.x : A GET request to /Account/Package would be used
• In v10.3.x : A POST request to /Account/Package/Search with no filters or search criteria would be provided.

Note: the response for each request is limited to 10,000 Account Packages.

Because this is a simple scenario, the requests in both versions look quite similar.  A basic request asking the system to retrieve all the products/packages that have been attached to all the accounts in the system

Scenario 2 - I want all the Account Packages for a specific account

• In v10.2.x : A GET request to /Account/Package with a search filter in the header of : {"filters": [{"name":"accountId","operator":"eq","value":"10000000"}]} would be used

• In v10.3.x : A POST request to /Account/Package/Search with a search request body of:

{

  "query": {

    "search": [

        {

            "name": "accountId",

            "operator": "eq",

            "value": "10000000"

        }

    ]

  }

}

On the surface, this may seem similar as well.  However, when you are managing companies and products around the world, handling complex search criteria and unicode characters in some of that search criteria becomes an important distinction.

Now let’s start to build on this scenario by adding in more complex requirements. Let’s say that the account that I have specified is a large customer, maybe a global IoT account that has 20,000 active SIMs/endpoints/licenses.  That means that my simple request now becomes a bit of an issue trying to retrieve and process 20,000 points of information.

Maybe in my integrated system, I want to provide the user with a way to browse these Account Packages, sort them, filter them, and other user interface functionality which is now going to be quite difficult to manage.

This is where you may now want to start and introduce pagination in the v10.3 API. This allows you to request the information in “pages” and then request different pages at a time in those results as the user navigates their way through the list.

Scenario 3 - I want the first 15 account packages for a specific account

In v10.3x you can send a POST request to /Account/Package/Search with a search request body like the following specifying that you want the first page of the results and each chunk of results should be returned 15 at a time:

{

  "query": {

    "search": [

        {

            "name": "accountId",

            "operator": "eq",

            "value": "10000000"

        }

    ],

    "pagination": {"pageNumber":1, "pageSize":15}

  }

}

To change up which page of results you would like to return you can simply change the pageNumber parameter to list the page you wish to retrieve.

This new Search API allows a developer to use a single path to retrieve all, filtered or paged data simply by changing the body of the request. Generally speaking, using the API request body is easier to work with than traditionally passing down request headers or trying to use some kind of path parameters for all the combinations required. 

This search syntax also allows you to chain multiple search operators together, so you can search based on multiple fields with different AND/OR logic on how to pinpoint the specific data you’re looking for.

Additional Search API features

The Search API also supports features such as requesting the top X records, specifying only the fields of the object/entity you wish to be returned, and OrderBy conditions. An example request that uses these features would look like this:

POST to /Account/Package/Search with a search request body of:

{

  "query": {

    "top": 10,

    "fields": [

      "identity",

      "name",

      "effective"

    ],

    "search": [

      {

        "name": "effective",

        "operator": "gtEq",

        "value": "2021-01-01T00:00:00Z"

      }

    ],

    "orderBy": [

      {

        "name": "name",

        "direction": "asc"

      }

    ]

  }

}

Deep Search Requests

Now it’s time to dig into some really exceptional enhancements!  Prior to v10.3.x, many APIs supported retrieving all details for a specific object/entity. For example, appending /Detail to a GET request looked like this: /Account/Package/12345/Detail

This /Detail API is useful in that it provides all the data for that Account Package (12345) in a single request that will return some of the deep related child entity data along with it to avoid multiple API calls, but we wanted to address 2 remaining issues: 

1. Unifying this behaviour under the single Search API 
2. Providing the developer control over which details were retrieved (how deep do you want to go in the related data, and which related data do you want)

For example, if the developer wanted to just retrieve just the Account Services created after January 1, 2021 ordered by name for a specific Account Package the request would like this:

A POST request to /Account/Package/Search with a search body of:

{

  "query": {

    "search": [

      {

        "name": "identity",

        "operator": "eq",

        "value": "12345"

      }

    ],

    "accountServices": {

      "search": [

        {

          "name": "created",

          "operator": "gtEq",

          "value": "2021-01-01T00:00:00Z"

        }

      ],

      "orderBy": [

        {

          "name": "name",

          "direction": "asc"

        }

      ]

    }

  }

}

This allows the developer to specify exactly which deep data they want to retrieve, and to specify constraints, filters, and sort logic to that deep data (accountServices) in the result.

These kinds of deep queries save multiple API calls and complex client-side orchestration to loop through results and constantly request more information layer after layer.

As of v10.3.x there are 3 entities which support Deep Search based on the most commonly asked for API integrations: /Invoice/Item, /RatePlan and /Account/Package. 

More Deep Search APIs will continue to be added in future releases as we examine the most common integration points to the billing platform and continue to improve the ease of integration for 3rd party developers.

If you have an API that you are using all the time and would like to see some Deep Search functionality added to it, then please do reach out to your customer support representative to see if we can get it scheduled as an API enhancement.