What is background processing in the context of SharePoint Online?
While a lot of actions and tasks in an application take place within the application user interface (data entry, manual process triggering), there is very often a need to trigger actions independently of user interface. Triggering processes on data change, running processes on scheduled time and running background totals and aggregations are just some of the background operations commonly found in business solutions.
The situation with business solutions on top of SharePoint is not really much different at all. Back in the days of full trust code solutions on the SharePoint Server, we were developing event receivers and timer jobs for those purposes, and those were perfectly valid options at that time. On the other hand, developing both timer jobs and event receivers required coding skills in C# and somewhat complex deployment processes. One of the ways around this was creating workflows, either through standard SharePoint workflows, or through third-party workflow tools available on the market. Still, workflow platforms have well-known issues, such as trackability, manageability, long-running workflows, pausing and resuming and interrupted workflows.
In essence, there are four main types of background processes and actions which can appear in a typical solution:
- Background aggregations are probably the most common case: running totals across the data sources. A typical situation in SharePoint is when in one list you need to have an aggregated value from another list. Developing a workflow for that purpose would require too many compromises, so developing an event handler was a standard procedure in such cases.
- Data inheritance is pretty similar. In SharePoint, it is most often used to inherit metadata from a list item in one list to a bunch of items in another, usually for document or picture tagging. The same issues with workflows exist here, so event handlers were the preferred way.
- Conditionally triggered processes mean running processes or blocks of code when data or states in SharePoint change. For example, when a new item is added in a list, or when permissions have changed, a certain process should be started. Depending on the process, a workflow or an event handler with a custom code would be a preferred course of action here.
- Scheduled processes are blocks of code run in regular time intervals, usually for reminders or mailings. SharePoint timer jobs were the usual course of action for implementing such processes.
In respect of Office 365 and SharePoint Online, migration to the cloud did not bring significant improvements in the development of these background processes. Event handlers can be replaced with remote event handlers (deployed as provider-hosted add-ins) or webhooks, both of which require development skills and external systems (such as those from Microsoft Azure). Microsoft Flow is gaining more and more traction for the workflow scenarios, even if it is not yet quite there with SharePoint conditions and actions, and third-party workflow platforms in the cloud are usually less powerful than those in SharePoint on the premises.
This is why we at skybow are progressively introducing more and more background processing capabilities into our flagship product: skybow Solution Studio.
Exploring background processing possibilities with skybow Solution Studio
To get to the background processing configuration, open any list from the solution Explorer in skybow Solution Studio Online, and click on the "Things in background" tab at the top of the main area.
When open, you will see four subtabs, each of them offering different capabilities.
When open, you will see four subtabs, each of them offering different capabilities.
Standard column properties
In the "Columns" tab, it is possible to add new columns to the list of libraries, delete columns, or change existing columns. By clicking on the "Settings" gear icon besides the column name, you will enter the column settings, where standard properties and also automatic aggregations are available.
When a column properties dialog box opens, the default is the "Standard" tab. There you can change the column the same way you could in SharePoint.
When you click on the "Values tab, the dialog box for setting background calculations will open.
If you take a look at the "Calculated Value" drop-down, you will see that we have multiple options available.
Total, Count, Min, Max and Average operations are for running bottom to top aggregations. For example, imagine we have a wine cellar inventory site, with two SharePoint lists – the list of wines, and the inventory list, where for each wine we have information on the number of bottles per vintage, and the price for each bottle. Of course, the inventory list has a lookup field to the wines list.
Now, in the wines list we have a number field (let’s call it “Total amount”) where we want to aggregate the numbers of bottles in the inventory list, which are stored in the amount field. By simply choosing the “Sum” aggregation, and choosing “Wine inventory” list and the “Amount field”, the aggregation will be set.
Note that the field types of the aggregated value field (“Total amount” in this case) and the field which is being aggregated (“Amount” in this case) have to be identical.
Additionally, we could set up the aggregation filter, where we can choose to aggregate only the bottles of a specific vintage (“2010” in an example on the screen below) or any other kind of conditional aggregation.
In a way, metadata inheritance is an opposite process to aggregation. It is often needed to propagate metadata from top to bottom, using the lookup values as keys. This is very much true in the search scenarios. Imagine the following case: we have the different marketing images for the wines in our wine cellar. They are stored in a picture library, with a reference to the wine. But those images from the picture library don’t have any metadata on their own; the metadata are actually at their “parent item”, which means those images are not properly indexed by the search engine, and so they are difficult to find.
Metadata inheritance in skybow Solution Studio Online enables you to propagate metadata from top to bottom. Of course, the fields of the same type have to exist in the target list or library, but it takes only a slight configuration – shown in the image below – to configure it.
List settings and list content
In the “List Settings” tab in “Things in background”, we can configure stuff like list title, description and icons.
In the “Content” tab, we can select whether the content of this list should be packaged together with the solution during the deployment and publishing scenarios. This is particularly important for the lists with so called “master data” – imagine the lists of grape varietals, countries and wine regions, which we would definitely need for our imaginary wine cellar inventory scenario – they all contain several hundred or even several thousand items. If we want to deploy the solution to the target environment, we don’t want to re-enter these values again; it would be much more practical if they could be deployed together with the solution. If you use skybow Solution Studio, with this simple checkbox, they will be deployed.
Scheduled actions - mimicking timer jobs in SharePoint Online
Timer jobs have always been one of the most used features in SharePoint. Basically, they enable developers to execute custom code in the SharePoint context in the given time intervals. There has been no "out of the box". replacement for timer jobs in SharePoint Online, though: the developers have mostly been reaching for different daemon capabilities in Azure, such as web jobs or Azure functions, to implement similar functionality.
With skybow Solution Studio Online, it is very easy to create repeatable, scheduled actions, and the journey starts with clicking on the "Scheduled Actions". tab in "Things in background", and clicking on the "Add Scheduled Action" button.
Next you need to enter the same basic information about the scheduled action, such as its title and description, but, if you want, you can also set a filter which would limit the action only to particular items on the list.
After pressing "Next", you need to define the action schedule - when do you want the action to start? You can choose between hourly, daily, weekly and monthly actions.
The last step is to define the actions which will be executed at a given time. At the moment, we support basic list item actions such as adding, deleting and editing list items, and sending emails, but in the future, more actions you already know from the skybow Action Links will find their way into Scheduled Actions.
Background processing is, as we have seen, an important part of every business solution. Those processes usually consist of different daemons such as triggered actions, background aggregations and calculations, and timer jobs. skybow Solution Studio Online is already able to offer most of the core background processing functionality in the SharePoint online context we have seen how to set bottom-up aggregations, how to implement metadata inheritance for search scenarios, how to do automated field calculations, and, last but not least, how to set up scheduled actions.
We are constantly working on this feature to bring even more actions to scheduled actions, and to implement conditional, triggered actions, so stay tuned: skybow Solution Studio Online is becoming a more and more powerful weapon for building business solutions on top of SharePoint Online as we speak.