Anypoint DataGraph

This year we’ve seen MuleSoft make some exciting additions to its offering. MuleSoft Composer and Anypoint DataGraph are two new products that promise to accelerate integration in new ways that go beyond the traditional programmatic consumption of REST APIs. “Rest” assured (pun intended!), REST APIs and API-led connectivity are still the foundation that every company needs to get right in order to efficiently drive reuse of assets and unlock their data. What’s interesting is that in most scenarios the consumers of said REST APIs are likely to require data to come from several ones at a time. Attached to this comes the need for significant additional effort, in the form of:

1. obtaining access to each API
2. writing custom code to fire requests to each API
3. writing custom code to parse the responses from each API
4. sieving through all of the API’s response fields – not just the ones that you need

 

So what can be done to reduce or avoid these efforts altogether?

Traditional REST may not be enough, but pairing it with a relational spec might be the answer. Allow me to take a step back and take you through a short overview of GraphQL.

GraphQL is a specification for a query language for your APIs. It was conceived in 2012 by Facebook and then popularised by big players like GitLab, Pinterest and Shopify, after its release to the public in 2015.

With GraphQL you define a graph schema that provides a complete and understandable relational description of the data entities in your APIs. Developers can then refer to this schema to write requests in the form of relational queries that allow them to obtain with a single request data that spans multiple related data entities, giving them only what fields they ask for and nothing more. 

The best part is that all of this is achieved without requiring the consumers to develop any custom code! All they need to do is write the query.

Ok, so now you may have already worked out for yourself why GraphQL is important when talking about Anypoint DataGraph. Allow me, however, to spell it out anyways: Anypoint DataGraph implements the GraphQL specification. As such it enables enterprise architects to devise a unified schema of related data types using existing REST APIs, so that then this schema can be exposed as a service and queried using GraphQL.

The buzzword to focus on here is “relational”. You will get value out of this product when you consume data that has relationships defined in terms of “foreign keys”. Effectively, you can think of the data resources in your REST APIs as tables in a relational database – however the big difference is that your query results will not have a tabular structure but a tree structure, similar to an XML document.

 

Say now you’ve identified a bunch of REST APIs that fit the part.

How do you go about including them in your Anypoint DataGraph unified schema?

Each API you want to include must have its spec published on Anypoint Exchange and it must have an active and running instance. The interesting bit is that you can bring into your unified schema any API, even if not native to the Anypoint platform, so long as you have a working instance of it listening on an accessible URL – anywhere! Once an API is imported, the Anypoint DataGraph tool automatically picks up data types and their fields. These can be renamed as well as completely left out if necessary and relationships can be defined between the data types.

Anypoint DataGraph is a separately licenced add-on SaaS app. Beyond the obvious “having to pay for it”, this also implies that:

+ you don’t need to worry about its provisioning or maintenance. Behind the scenes it utilises an amount of vCores you want to allocate to it (yes! It eats up your vCore allowance!)

– it is not available for on-prem (however the APIs imported in the unified schema can live anywhere, so long as their spec is on Exchange)

The product also comes with a Query Builder web UI that allows you to write, test and profile GraphQL queries right away. With it you can drill down the queries’ making parts and see the time it takes to execute each.

“But how does all of this fit in with the API-led Connectivity model?” I hear you ask.

In a very simplistic way one can argue that this new service wraps both Process and Experience layers’ responsibilities in one neat package: on one hand it orchestrates for you REST API requests to fetch all your records across multiple data entities; on the other hand it gives you a tailored experience by presenting to you only the fields you are interested in – nothing more, nothing less.

 

So does this mean that if you get Anypoint DataGraph then you don’t need Process / Experience APIs anymore?

Well, not quite. You will still need your Experience APIs to present results in a different media type if your consumers can’t work with GraphQL. And indeed you will still need Process APIs to implement complex transformations beyond simple joiners on keys, and that can only be achieved via tools like DataWeave.

 

Anypoint DataGraph is not the first flavour of server-side GraphQL support that MuleSoft introduced.

Prior to this, MuleSoft had published an open source module that allowed you to program the GraphQL query resolvers as flows in your custom Mule apps, which meant a lot of coding was required to build out the GraphQL server service. Anypoint DataGraph by comparison does not require developers to write a single line of code to expose the service. It can all be achieved by clicks and no code. This alone is already a massive improvement from the previous product, but one must not forget that this is a product that is still in its infancy.

In fact, when wondering what the present and future of this product holds, a bunch of things come to mind.

Firstly the GraphQL specification includes “mutations”, a mechanism to allow manipulation of server-side data in write mode. For now Anypoint DataGraph only works in read mode and mutations are not supported. 

Another sticking point is the lack of an ability to include in your client-side queries any filters defined as complex boolean conditions (e.g. filter out orders with a total that is greater than $100). For now all you can do is filter by exact match on fields marked as identifiers (e.g. CustomerId = 123) although to be fair a standardised way to implement filtering based on parameters is missing from the GraphQL specification itself.

In terms of protecting your service from abuse, for now non-configurable hard limits restrict things like the number of concurrent calls, timeout periods, number of fields per query and query depth (which loosely speaking translates to how many levels you can nest your query to ask for related entities). These hard limits can feel quite inflexible compared to the way MuleSoft users are used to define custom policies to protect the APIs managed through Anypoint.

What would be really interesting to see is whether Salesforce plans on bringing together the Anypoint DataGraph and MuleSoft Composer for Salesforce products by releasing a GraphQL connector for Composer thus enabling automated consumption of any data entity published onto the Anypoint DataGraph schema.

 

A Parting Thought

As a final parting thought it is worth reminding you that by nature the GraphQL specification gives developers the ability to fetch, through a single endpoint, data that resides on disparate systems of record.

Because of this there is a danger that a GraphQL endpoint may enable data access in a way that was not originally intended. The GraphQL specification itself recommends that Authorisation should be handled by the business logic layer, instead of the GraphQL layer, where it can become cumbersome, hard to maintain and inconsistent with the authorisation rules set out by the business logic layer. This means that when putting together your unified schema you really need to ensure that authorisation of your clients against the DataGraph service works in tandem with that of the DataGraph against the REST APIs in your unified schema.

 

Summary

In summary, this new product introduced by MuleSoft is a powerful SaaS tool for architects, which allows them to expose a GraphQL service by means of clicks and no code. This service materialises a new way to consume APIs, accelerating developers’ access to data that sits across disparate REST sources. Whilst this means that this product cannot bring immediate value to companies without a well-established application network, it promises to accelerate even further consumption for those with a good foundation of RESTful APIs.