This document covers best practices with regard to developing, implementing and reviewing iHub recipes.
Throughout the document, the below use case will be used to give context to iHub concepts. Screenshots, examples and code snippets will all pertain to this use case. The example solution consists of:
- Calling the Messaging Interactions API every hour, to find conversations where the consumer has not sent a message in the last hour
- Then call the Web View API to tell a CB manager bot to close those conversations
The Idle Conversation Closer use case is configured on the Golden Account, 21480370, under the “Idle Conversation Closer” project.
iHub Terminology
iHub is broken up into three main items; Projects, Recipes and Ingredients. Those three areas then use the concept of Connectors, Datapills and Schemas which then allow us to get the full power out of iHub. These are all used to organise and build solutions.
Projects
It is best practice to create a project for each solution. Using the example above, the idea would be to create a verbosely named project called “Idle Conversation Closer”.
Name the project something verbose and meaningful
Recipes
Inside a project, the solution will consist of Recipes. These Recipes contain the logic of the solution. With our use case, we would:
- Create a recipe that would call the Messaging Interactions API and paginate through all the conversations.
- Create a recipe to filter through the conversations to find only the ones that have gone idle
- Create a recipe that calls the Web View API for the idle conversations
- Create a recipe that is invoked every hour and orchestrates the other recipes
It is sometimes useful to think about how you would create a solution in NodeJS, and then create a recipe for each imagined file, to separate the logic into components.
Inside a project, you can further organise your recipes using folders, if the logic is complex enough to require it.
- Be verbose with the naming of recipes
- Don’t be afraid to split your logic into multiple recipes
Ingredients
Ingredients are then the building blocks that build up the logic inside a recipe. To keep the NodeJs analogy going, if each file would be a recipe, then the ingredients would be the functions.
In the screenshot below, we can see six ingredients making up the recipe.
Just like in NodeJS, add comments to the ingredients, explaining what is going on.
Schemas
The most important but the most difficult thing to get right are schemas. If you don’t get them right, they will waste hours of your time trying to resolve and become a massive headache.
In short, schemas are a way of defining what format the data is in, as the data is being used throughout the recipe. If iHub knows the data is an array of objects that contains key/value pairs, iHub will then allow you to create a loop ingredient to iterate through the data. If the data is a string, then it won't let you loop through it, because it knows it’s the wrong data type.
That is where the headaches come from. If the schemas are not perfect, then iHub will block you from performing certain actions. Even if you know the data is in the correct format, if the schema is not exactly correct, then iHub will stop you from progressing.
Once you have created a schema, you will then be able to access the values as datapills and use them in other ingredients at a click of a button.
Datapills
If recipes are files and ingredients are functions, then you can take datapills to be variables. If an API call is made and returns data in a given format (defined by the schema), then iHub will automatically create datapills based on that schema.
You can then reference these datapills throughout the recipe, just like a variable in JS. Looking at the below request body for the Web View API, on the right-hand side, there are 4 datapills being used; BotID, Conversation ID and User ID.
The idea is that you can click on the value on the left-hand side and populate that into the ingredient.
Datapills have a small icon next to them to show where the value has been defined. Bot ID’s symbol matches the “New call for function” ingredient on the Recipe data panel. That means it is defined when the recipe is invoked. The Conversation ID datapill comes from the forEach loop.
Just like in JS, CC account numbers, domains, and any kind of id should all be referenced via datapills and should not be hardcoded.
Connectors
Connectors are OOTB ways to “connect” with other platforms and services. If you want to make requests to Google Sheets, Salesforce, or make an API call, then you will need to create a connection.
The most common connector we would use at LP is making simple REST API requests. To do that, you would create an “HTTP” ingredient and follow the configuration steps.
When you are on the connection step, there will be an option to create a new Connection. Name the connection something verbose (“Messaging Interaction API Connection” for example), fill out the rest of the details, and then hit the Connect button.
- The http connector does not support Auth1.0a authentication
- This means for most LP APIs, you should pick “none” for “authentication type” and use a “bearer token” returned from the Login API
- Only one connector of each type can be used per recipe
- If you have one connector for MSG Int API and one for the Webview API, those HTTP ingredients need to be in separate recipes.
Environment Properties
Just like in a Node app, FaaS and CB, static project values should be stored in an environment. These environment values are:
- Account-wide and cannot be specified per project or recipe
- Should be used to store Auth1.0a keys or any other sensitive static values
- Preferably, you would store credentials in the Connector itself, such as auth2.0 credentials, if supported.
Depending on the use case, you may find that the end user may need to update some value. In the Idle Conversation Closer example, the solution closes conversations that have been unresponsive for more than an hour. If you want that time value to be easily configurable by the end user, you may wish to put it into the environment.
Anything that you would put in the FaaS environment or .env file in NodeJs, you would put in an Environment Property.
Developing Recipes
Recipe Triggers
When you create a recipe, you will be asked to specify a trigger. This is how you define whether the recipe will trigger:
Trigger from an app - For example, the “Google Sheets” app lets you listen for sheet updates. You could trigger a recipe based on a new row being added.
Run on a schedule - specify how often the recipe should trigger
Build recipe function - On invocation from another recipe
Trigger from a webhook - This provides an endpoint that other platforms can call to trigger the recipe.
Build an API endpoint - This seems to just be a more powerful version of the “Trigger from a webhook” option. This is an additional and expensive feature so should only be used if the webhook option is not sufficient.
Which Ingredients to Use?
Default Ingredients
“IF condition” and “IF/ELSE condition”
Just like with an if statement in
Repeat action
This is simply a loop that can be used to iterate through an array. If your schema is wrong and an array isn’t correctly described, you will not be able to use a “Repeat action” on that datapill.
Call function
This is what you would use to call another recipe.
Handle errors
This is the equivalent of a try/catch in JS. Anything that could fail should be wrapped in a Handle errors ingredient.
For example, when paginating through many requests to the Messaging Interactions API, you would want to put the “HTTP” ingredient inside a “Handle errors” ingredient, inside a “Repeat action” ingredient. This will catch any single failed request.
Inside the error handler, you would want to log some meaningful message and then either do nothing else (move on to the next “Repeat action” iteration), or stop the job altogether.
Stop job
This is what you would use to gracefully fail a job. For example, if a request to the Domains API fails, you would want to fail the whole job. If one request out of ten to the Messaging Interaction API fails to return a batch of 100 conversations, you may be happy to simply skip it and move on to the next.
Usually, you would want to call “Stop job” in the “Handle errors” error handler, or in an “IF/ELSE condition” if some critical condition isn’t met.
Action in an app
Outside of the commonly used ingredients, there is a massive library of other ingredients that should be used.
Other Ingredients
Logger
Logger is similar to console.log() in JS. It is highly recommended to log anything that would be useful to understand, especially to assist with debugging.
There are some massive caveats to be aware of:
- If a logger is in a “Repeat action”, you will only see the last log. If you are making ten API calls and the fifth one fails, you will only see the log for the tenth request
- The logger parses all messages to a string. This is extremely annoying when trying to debug schema issues. It is highly recommended to not attempt to debug these kinds of issues with a logger as it will lie to you.
HTTP
As discussed in the Connector Section, this will be an often-used ingredient. Again, these should be wrapped in the “Handle error” ingredients.
Create Variable
So far, we have analogous concepts for the file and functions in JS. The final piece is the humble variable and conveniently, the iHub version is also called “Variables”.
The variable ingredient has several actions.
Create variable - You would use this to create a variable of type string, number, integer and booleans. You can also create a variable with the arrays or object type, however, this requires manually creating the schema as the UI doesn’t let you do this.
Update variables - is then what you would use to update the created variable.
Create list - You would use this to create an empty array.
Add items to list - This is what you would use to “push” one value into the list.
Add items to list (batch) - This lets you add many values to a list in one go. This would be analogous to the “Array.prototype.concat()” function in JS.
When paginating through the Messaging Interactions API responses, you would first create a list outside of the “Repeat action” and then add items to the list in batches, since each API request returns an array of up to 100 conversations.
Schema Deep Dive
As said previously, schemas are one of the most important aspects to get right. If they are wrong, then iHub will actively block you from performing certain actions if it doesn’t understand the format of the data.
HTTP Schema
The most common use of schemas is the payload and response body of API calls. The UI allows for two types of schema creation: “Use JSON” and “add fields manually”.
Use JSON
With the JSON approach, you can paste in an example of what the format should be, and then iHub will work out the schema. While this does save time for something complex like the Messaging Interaction API’s response, I found it tends to lead to more issues than not.
The Messaging Interaction API has many optional fields and some fields will even change type so you can end up with missed or incorrect values from the schema, depending on the sample JSON provided.
- The Use JSON option struggles most with integers. It will automatically set integers to the number type. This means an integer will get a decimal point and zeros proceeding the integer.
- 123 integer becomes 123.00 number
- Using a number with “.00” on the end will break URLs
Below is an example of creating a schema from pasted JSON.
To correct the number type, you can edit that property and change the type to integer.
With extremely complex schemas that iHub really struggles with, you may find that you need to “Edit Schema” and directly edit the JSON. This give you much more flexibility than the UI, however, it can be quite complex and should only be done as a last resort.
The good news is that you don’t need to schema the whole response, just the values that you need. For example, if you only need the Conversation Id from the “info” object of the MI API, then you only need to add that to the schema. If you later realise you need the “startTimeL” value, that can be added to the schema manually later.
Add fields manually
Let’s take a look at an example. Let’s say we want to create a schema for the Messaging Interactions API to get the Conversation ID. We know that “conversationHistoryRecords” is an array of objects and those objects contain the “info” object, which then contains the “conversationId” string.
To create this manually, click add field.
- Name: This is the property key
- Label: This is a user friendly name that you can use to make the solution more verbose.
And starting with the top of the properties, you can create the “conversationHistoryRecords” array.
Now, do the same again to create an object called “info” and create a string called “conversationId”.
You will then end up with three properties. You must then click and drag the properties so that they are in the right places.
We are left with an array of objects that contain the “info” object, which has one key/value pair for “conversationId”. Although the schema doesn’t explicitly say it, “conversationHistoryRecords” and “info” are both in their own objects, so the format is:
{ “conversationHistoryRecords”: [ { “info”: “conversationHistoryRecords”: “ABC123” } ] }
Finally, now that the schema is complete, iHub will allow you to drag and drop the response body via datapills. In the first example, because I am using a logger ingredient, the only option available to drag and drop is the Conversation Id string, and some describing values about the “conversationHistoryRecords” array.
In the second, for a repeat ingredient, I can only access the array, because that is the only thing that is iterable.
JavaScript Schema
After API calls, the next most common schema to define is that of the JavaScript ingredients.
Here in the example, there are two string inputs and one string output. In practise, this looks like:
{ “conversationId”: “abc123”, “botId”: “def456”} and { “webViewHeader”: “ghi789” }
This format means that the “main” function is being called with an object and should be handled as such. Here, the object is being deconstructed to access the two strings directly.
And finally, the output also needs to be in object form, even though we have said it is a string. Here, an object with a “webViewHeader” key is being returned, which matches the schema.
Recipe Calls and Responses
Just like with everything else that accepts inputs and outputs, recipes can be called from other recipes and therefore, require an input and output schema. In the below example, the Shift Status API recipe has been defined to require two string inputs and will return a boolean output. This is defined on the “Function Call Trigger” ingredient of the recipe.
Text vs Formula Inputs
Many ingredients require inputs in some form or another. Hovering over such a field will reveal a “Text/Formula” toggle.
When set to a field to text, you will still be able to use datapills to populate the text string. As demonstrated in the below picture for the Request URL, you are able to add string/number type datapills straight to the URL string.
Sometimes, more complicated actions are required to create a value. Performing any mathematical operations or value comparisons will likely require a formula. Formulae will allow you to perform mathematical operations and value comparisons, as well as use inbuilt Workato functions, such as parsing a number to a string or getting the current date.
A list of built in formulas from Workato are available HERE. Under the hood, Workato is using Ruby, so the syntax is very similar.
Commonly Used Formula Functions
- “now” returns the current date
- “now.to_i” returns the current date as in epoch time, as a number type
- “.to_i” can be used to turn a string number into a number number
- “.to_s” converts values to a string
- “.to_json” is sometimes required to force arrays in to the right format to use with API requests
- “.match?()”, “.includes?(...)” and “.starts_with()?” are just some of the helpful string testing functions that may be useful
When an API call is failing, it can be very difficult to debug if the issue is due to a value being the incorrect type. Luckily, it is fairly easy to change types between strings and numbers.
Testing and Debugging
To test recipes, simply hit the Test recipe button. You will then see iHub step through the recipe, highlighting any errors that are encountered. You can then click on each step to see what the inputs and outputs are.
- Remember Logs are stringified which can be extremely misleading when debugging type issues
- A log inside a loop will only show the last log
The Jobs screen can be used to view all test and production invocations on the recipe. If a recipe calls another recipe, it will give you a link to that other recipe to view the logs from there.
Versioning
Individual recipes do have the concept of versioning. They are automatically created whenever a recipe is saved.
Unfortunately, if you have a large solution that has many recipes, or if you save often, it is a very tedious process to change versions. I highly recommend that once a solution has been built, comment on the version.
Transferring Across CC Accounts
Under “Tools”, in the “Recipe lifecycle management” option, you can create a “manifest”. A manifest is effectively a snapshot of all recipes and environment values that a project uses.
Every time a new version of the project is ready, a new manifest can be created. Unfortunately, this is very much a manual process and requires stringent naming of the manifest to be useful.
If you make small updates to a project that don’t warrant a full new manifest, you can update an existing manifest.
Now that a manifest exists, you can simply export the ZIP and import it straight onto another CC account.
To import, the project folder would need to have been already created, to give the import tool something to import into.
This can also work as a way of duplicating a solution on the same CC account.
Updating Production Projects
Now that we have the concepts of versioning and import/exporting solutions, we need to then cover the most complex version of this process; updating a live, production solution.
1st Time Deployment Process
- Build your solution on a test account
- Once done, create an export manifest called “{Solution} V1” and download the ZIP
- On another account (be it production or another test account), create a project, then navigate to the import screen
- Create an import manifest that imports into the newly created project. Call the manifest “{Solution} V1”.
“N”th Time Deployment Process
- Update the solution on the test account
- Once done, create an export manifest called “{Solution} V{“N”}” and download the ZIP
- On another account, navigate to the import screen and create a new import manifest called “{Solution} V{“N”}” that imports into the existing project.
- You will get a list of differences between the current version and the newly imported version to review.
- Click import to immediately update the solution to the new version.
Rolling Back
If an issue is found and rollback is required the easiest thing to do is use the versioning feature for small roll backs of individual recipes.
If something more drastic is needed, you will need to re-import and create another manifest to import back into the project. This is not the smoothest of processes, but it shouldn’t be a problem if you have followed best practices around creating import/export manifests.
Review Process
Unlike FaaS functions and NodeJs applications, it is not possible to code review something like an iHub Recipe in gitlab. This means the process is somewhat more manual.
Below are the steps that should be followed to ensure that the solutions have had a fresh pair of eyes look over them, looking for any edge cases or possible optimisations.
- Create the solution and documentation.
- If the solution consists of several components, such as a iHub project, CB bot and FaaS function, all aspects should be uploaded to a gitlab repo
- If the solution is purely a iHub project, creating a repo for just one zip file has limited value, but can still be useful for keeping a snapshot of the solution safe
- The iHub recipe should be exported as a zip and added to the repo
- The repo readme.md should be the documentation itself, or link to the documentation.
- Schedule a meeting with another SE (and SA if they are interested). The meeting should cover:
- An end to end demo the solution
- A walk through of the iHub project. This should include reviewing it’s recipes, ingredients and any other components that may be at play, such as FaaS or CB
- During the demo and walkthrough, the reviewing SE should highlight anything that has not been considered such as:
- Any edge cases that haven’t been taken into account
- Any missing logic such as error handling, missing pagination, etc etc
- Offer any advice or better ways of handling aspects of the solution
- Highlight any security concerns such as the storage of secrets and the like
- Following the live demo, if required, the reviewing SE should then spend some time deep diving into the solution by themselves, reading the documentation and ensuring that there are no gaps.
- If the solution consists of multiple components, such as a companion FaaS function, then the FaaS function should also be code reviewed the normal way via gitlab feature branch review process.
