Traditional SharePoint developers knew very well how to handle daemons in their daily work: Timer Jobs were the way to go. Microsoft was very specific in their guidance and documentation. We used (and sometimes misused!), Timer Jobs for all different sort of situations \u2013 from cleanup jobs, async data manipulation (sic!), to getting external data into the SharePoint context. Even with all the problems that timer jobs had, we learned to live with it, and to use them to our advantage.<\/p>\n

The situation got somewhat more complex with the transition to cloud, and to client-side development. First of all, there was no \u201cofficial way\u201d how to do it anymore, as in the \u201cgood ol\u2019 times\u201d. The new Microsoft is all about \u201chere is the API, you use whatever tech you want\u201d mantra. Which can be good (freedom!), and bad (bad architectural decisions!) in the same time. The trending technology was also changing over the past two or three years. Three years ago, my answer on \u201chow do you develop daemons for SharePoint Online\u201d would probably be \u201cAzure WebJobs\u201d. Now, we have even more options. In this article, I\u2019ll fully go the \u201cserverless\u201d way, and explain how to use Azure Functions for these purposes (because Functions run on hamsters, and not on servers).<\/p>\n

Why Functions? Because they will do the job just fine for the light-weight and short running pieces of code, which is very often the case with timer jobs. For anything more complex, using Azure WebJobs is still a legitimate option.<\/p>\n

Who are you, and when yes, how much?<\/h2>\n

That question from the subtitle is the first question we need to answer. It is not a new question, but it was, in the \u201cgood ol\u2019 times\u201d, answered for us. SharePoint Timer Jobs would run under whatever service account that was running OWSTIMER.EXE Windows service, and sharepoint\\system would be used to access the data. Why was it like that? It is really, easy: you don\u2019t have a \u201creal\u201d user when running a daemon, nobody has authenticated against the IIS. It is a background process that does stuff, and you shouldn\u2019t use a \u201creal\u201d user\u2019s account for that purpose. Therefore, we used the service accounts.<\/p>\n

The same is true for SharePoint Online and Azure world. For daemons, we will need to authenticate our code with a non-personal identity, just like we did before. It\u2019s called \u201capp only\u201d nowadays.<\/p>\n

So how do we do that? In SharePoint Online and Azure world, it is Azure Active Directory (AAD) what we need. If you still didn\u2019t learn it, it\u2019s time for you to do it: it is probably one of the most important pieces of today\u2019s technology for developers on Microsoft platform. We will create an AAD -secured application, to wrap our code into, and set the necessary permissions for it.<\/p>\n

The first step is to log into the Azure Portal \u2013 \u201cAzure Active Directory\u201d Management, and to \u201cregister\u201d a new web application as \u201cWeb app \/ API\u201d through that portal.<\/p>\n

<\/p>\n

Next, we need to assign the Application permissions we need on SharePoint Online and\/or Microsoft Graph APIs (or any other standard or custom API you might have available).<\/p>\n

When we choose an API, we need to give it permissions. We have two types of those: Delegated permissions, with full impersonification, where user\u2019s identity is required and passed on, and Application Permissions, where user\u2019s identity is neither required nor needed, and where we want our \u201cApp\u201d being the principal to make the changes.<\/p>\n

We should leave delegated permissions empty here, since we are developing a daemon, and therefore we don\u2019t know anything about the user. We will go to Application Permissions, and click the appropriate checkboxes, depending on what our \u201ctimer job\u201d will have to do.<\/p>\n

<\/p>\n

If you are familiar with AAD applications, you know that securing them works on a well-known public\/secret keys principle. For each application that you register with the AAD, you will generate a Client ID, which serves as a public key (you don\u2019t have to worry about losing that one), and the set of secret keys, which you need to keep \u2013 well, secret.<\/p>\n

For using our newly created application for Azure Functions, however, this will not be enough. We are not going to use the secret keys, but certificates, in order to make this application really authenticate our Azure Function against SharePoint. In production, we are probably going to purchase a certificate from a relevant certificate authority, but for this test, a self-created one is going to do.<\/p>\n

In order to do that, you will need to create a certificate on your machine (in Windows, using the Management Console or PowerShell), and to export it as a X.509 certificate (usually with .CER extension) and the private key (usually with .PFX extension). Once you have the CER and PFX files on your computer, run the following PowerShell script to extract the base64 representation, certificate key, and certificate hash from the X.509 certificate (.cer file), since we are going to need them later.<\/p>\n

\n$certPath = "X:\\\\[path]\\\\[certificatenname].cer"\n$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2\n$cert.Import($certPath)\n$rawCert = $cert.GetRawCertData()\n$base64Cert = [System.Convert]::ToBase64String($rawCert)\n$rawCertHash = $cert.GetCertHash()\n$base64CertHash = [System.Convert]::ToBase64String($rawCertHash)\n$KeyId = [System.Guid]::NewGuid().ToString()\nWrite-Output "Base 64 Certificate"\nWrite-Output $base64Cert\nWrite-Output "Base 64 Certificate Hash"\nWrite-Output $base64CertHash\nWrite-Output "Key ID"\nWrite-Output $KeyId\n<\/pre>\n
The output from this PowerShell will look something like this:<\/p>\n
<\/p>\n
You should save these values, since we are going to need them later.
\nNext, we need to go back to our AAD Application, and to enter this value in the application manifest. Click on the \u201cManifest\u201d icon in the application properties, and the manifest editor will open. Look for the \u201ckeyCredentials\u201d section in the manifest \u2013 it should be empty. Insert the following values into it, so that it looks like this:<\/p>\n
\n"keyCredentials": [\n{\n"customKeyIdentifier": "[enter certificate hash here]",\n"startDate": "[certificate start date]",\n"endDate": "[certificate end date]",\n"keyId": "[enter certificate key id here]",\n"type": "AsymmetricX509Cert",\n"usage": "Verify",\n"value": null\n}\n]\n<\/pre>\n
Please save the manifest. It should look something like this:<\/p>\n
<\/p>\n
Once we are done with this, we can move on to creating our Azure Function, which will serve as a timer job.<\/p>\n
Creating an Azure Function and connecting it to SharePoint<\/h2>\n
First, create a new, empty Azure Function from the Azure Portal. In my case, I will choose a HTTP triggered C# function, but you can choose any other language you feel comfortable with. HTTP trigger means that we are going to invoke our function manually, through HTTP operations, so we can easier test the function. This will be changed later on, in order to use timer triggers.<\/p>\n
The first step you need to do is to upload the private key (PFX) to the files section of your certificate:<\/p>\n
<\/p>\n
Once you are done with that, let\u2019s start developing the function. The first thing you will need to do, is to set the dependencies. For C#, you will need to open project.json file, and to add the references to<\/p>\n
\u2022 Microsoft.IdentityModel.Clients.ActiveDirectory
\n\u2022 Microsoft.SharePointOnline.CSOM<\/p>\n
During the background compilation of this function, Nuget packages of the above-mentioned dependencies are going to be retrieved, and the references are going to be resolved properly. I used the CSOM here, depending on APIs and libraries you want to use, you might want to reference something additional or something else.
\nIn my case, project.json looks something like this:<\/p>\n
\n{\n"frameworks": {\n"net46":{\n"dependencies": {\n"Microsoft.IdentityModel.Clients.ActiveDirectory" : "3.13.7.964",\n"Microsoft.SharePointOnline.CSOM": "16.1.5813.1200"\n}\n}\n}\n}\n<\/pre>\n
Now let\u2019s look at the function code itself (run.csx).
\nWe will first need to declare some environment variables:<\/p>\n
\nprivate static string ClientId = "AAD app client id";\nprivate static string Cert = "sharedovedev.pfx";\nprivate static string CertPassword = "pass@word1";\nprivate static string Authority = "https:\/\/login.windows.net\/[yourtenant].onmicrosoft.com\/";\nprivate static string Resource = "https:\/\/ [yourtenant].sharepoint.com\/";\nstring siteUrl = "https:\/\/[yourtenant].sharepoint.com\/sites\/[yoursite]\/";\nstring listName = "[listname]";\n<\/pre>\n
Most of these are self-explaining. Use the password that you have set during the certificate creation. Change the authority and resource to include your tenant urls.<\/p>\n
Important note here: don\u2019t store your certificate passwords in the clear text in the function, there are other ways, such as Azure Key Vault, how to do that properly.<\/p><\/blockquote>\n
In the Run method itself, we will first need to get the authentication token from the certificate:<\/p>\n
\n\/\/create AuthenticationContext\nvar authenticationContext = new AuthenticationContext(Authority, false);\n\n\/\/read the certificate private key (PFX) as byte array\nvar certPath = Path.Combine(Environment.GetEnvironmentVariable("HOME"), "site\\\\wwwroot\\\\[yourdirectory]\\\\", Cert);\nbyte[] certbytes = System.IO.File.ReadAllBytes(certPath);\n\n\/\/create X.509 certificate object out of the byte array\nvar cert = new X509Certificate2(certbytes,\nCertPassword,\nX509KeyStorageFlags.Exportable |\nX509KeyStorageFlags.MachineKeySet |\nX509KeyStorageFlags.PersistKeySet);\n\n\/\/get the auth token out of that certificate\nvar authenticationResult =\nawait authenticationContext.AcquireTokenAsync(Resource, new ClientAssertionCertificate(ClientId, cert));\n\nvar token = authenticationResult.AccessToken;\n<\/pre>\n
With this, the most important thing has been done: we have got our access token, which we can use for the calls against our APIs, in this case, against SharePoint Online.<\/p>\n
In order to do that, we need to attach this token to every call we make against SP Online.<\/p>\n
\n\/\/create a ClientContext, and attach the auth token\n\/\/to the header of each request we are going to make.\nvar context = new ClientContext(siteUrl);\ncontext.ExecutingWebRequest += (s, e) =&gt;\n{\ne.WebRequestExecutor.RequestHeaders["Authorization"] = "Bearer " + authenticationResult.AccessToken;\n};\n\n\/\/now finally we are ready to do\n\/\/some SharePoint magic with the CSOM here\ncontext.Load(context.Site);\ncontext.ExecuteQuery();\nWeb web = context.Web;\n<\/pre>\n
What did we do here? After we\u2019ve got the token, we have created a SharePoint ClientContext, and attached the auth token to each request we are going to make to the server. That way our auth token is always going to travel along.<\/p>\n
The full code of run.csx can be found here \u2013 please use it just as an example, it is neither clean, nor did I follow the coding best practices here (clear text passwords!). You will, however, get the idea how the things work from there: https:\/\/goo.gl\/M5hu22<\/a><\/p>\n
My code actually adds some items into a SharePoint list. If you open the list items which have been added by this Azure Function, you will see that the Author and Modified By fields are properly set to the principal which nicely identifies as our AAD App:<\/p>\n
<\/p>\n
Creating the timer job from Azure Function<\/h2>\n
The last step will be to change the trigger of our Azure Function from HTTP to Timer. Open the \u201cIntegrate\u201d tab of the function, and delete the existing HTTP trigger:<\/p>\n
<\/p>\n
Then click on the \u201cNew Trigger\u201d button, and select the \u201cTimer\u201d:<\/p>\n
<\/p>\n
In the last step, you will have to setup the schedule, and save the function.<\/p>\n
When setting up the schedule, the basic format of the CRON expressions in Azure is consisted of 6 positions (placeholders), representing respectively:
\n{second} {minute} {hour} {day} {month} {day of the week}<\/p>\n
For example, the following value would make timer execute every minute, on the second zero:<\/p>\n
\n0 * * * * *\n<\/pre>\n
Note: examples below are overtaken from Armin Reiter’s blog post “Azure Fuctions Time trigger cheat sheet” (https:\/\/codehollow.com\/2017\/02\/azure-functions-time-trigger-cron-cheat-sheet\/<\/a>) for the purpose of easier and compact reading. For way more<\/b> information about setting time triggers in Functions, please visit the Armin’s original blog post<\/a>.<\/p><\/blockquote>\n
The following values are allowed for the different positions (placeholders):<\/p>\n\n\n\n\n\n\n\n\n\n
Position<\/th>\n
Placeholder<\/th>\n
Allowed Values<\/th>\n
Note<\/th>\n<\/tr>\n
1<\/td>\n
{second}<\/td>\n
0-59; *<\/td>\n
{second} when the trigger will be fired<\/td>\n<\/tr>\n
2<\/td>\n
{minute}<\/td>\n
0-59; *<\/td>\n
{minute} when the trigger will be fired<\/td>\n<\/tr>\n
3<\/td>\n
{hour}<\/td>\n
1-23; *<\/td>\n
{hour} when the trigger will be fired<\/td>\n<\/tr>\n
4<\/td>\n
{day}<\/td>\n
1-31; *<\/td>\n
{day} when the trigger will be fired<\/td>\n<\/tr>\n
5<\/td>\n
{month}<\/td>\n
1-12; *<\/td>\n
{month} when the trigger will be fired<\/td>\n<\/tr>\n
6<\/td>\n
{day of the week}<\/td>\n
0-6; MON-SUN; *<\/td>\n
{day of the week} when the trigger will be fired<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
As we see, the values in the table above are flexible enough that we can define and set all the time intervals we would like. Here are just some examples:<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Expression<\/th>\n
Description<\/th>\n
runs at<\/th>\n<\/tr>\n
0 * * * * *<\/td>\n
every minute<\/td>\n
09:00:00; 09:01:00; 09:02:00; \u2026 10:00:00<\/td>\n<\/tr>\n
0 *\/5 * * * *<\/td>\n
every 5 minutes<\/td>\n
09:00:00; 09:05:00<\/td>\n<\/tr>\n
0 0 * * * *<\/td>\n
every hour (hourly)<\/td>\n
09:00:00; 10:00:00; 11:00:00<\/td>\n<\/tr>\n
0 0 *\/6 * * *<\/td>\n
every 6 hours<\/td>\n
06:00:00; 12:00:00; 18:00:00; 00:00:00<\/td>\n<\/tr>\n
0 0 8-18 * * *<\/td>\n
every hour between 8-18<\/td>\n
08:00:00; 09:00:00; \u2026 18:00:00; 08:00:00<\/td>\n<\/tr>\n
0 0 0 * * *<\/td>\n
every day (daily)<\/td>\n
Oct 16, 2017 00:00:00; Oct 17, 2017 00:00:00<\/td>\n<\/tr>\n
0 0 10 * * *<\/td>\n
every day at 10:00:00<\/td>\n
Oct 16, 2017 10:00:00; Oct 17, 2017 10:00:00<\/td>\n<\/tr>\n
0 0 * * * 1-5<\/td>\n
every hour on workdays<\/td>\n
Oct 13 (FRI), 2017 22:00:00; Oct 13 (FRI), 2017 23:00:00; Oct 16 (MON), 2017 00:00:00<\/td>\n<\/tr>\n
0 0 0 * * 0<\/td>\n
every sunday (weekly)<\/td>\n
Oct 15 (SUN), 2017 00:00:00; Oct 15 (SUN), 2017 00:00:00<\/td>\n<\/tr>\n
0 0 9 * * MON<\/td>\n
every monday at 09:00:00<\/td>\n
Oct 16 (MON), 2017 09:00:00; Oct 16 (MON), 2017 09:00:00<\/td>\n<\/tr>\n
0 0 0 1 * *<\/td>\n
every 1st of month (monthly)<\/td>\n
Oct 1, 2017 00:00:00; Nov 1, 2017 00:00:00; Dec 1, 2017 00:00:00<\/td>\n<\/tr>\n
0 0 0 1 1 *<\/td>\n
every 1st of january (yearly)<\/td>\n
Jan 1, 2018 00:00:00; Jan 1, 2018 00:00:00; Jan 1, 2019 00:00:00<\/td>\n<\/tr>\n
0 0 * * * SUN<\/td>\n
every hour on sunday<\/td>\n
Oct 15 (SUN), 2017 23:00:00; Oct 22 (SUN), 2017 00:00:00; Oct 22 (SUN), 2017 01:00:00<\/td>\n<\/tr>\n
0 0 0 * * SAT,SUN<\/td>\n
every saturday and sunday<\/td>\n
Oct 15 (SUN), 2017 00:00:00; Oct 21 (SAT) 00:00:00; Oct 22 (SUN), 2017 00:00:00<\/td>\n<\/tr>\n
0 0 0 * * 6,0<\/td>\n
every saturday and sunday<\/td>\n
Oct 15 (SUN), 2017 00:00:00; Oct 21 (SAT) 00:00:00; Oct 22 (SUN), 2017 00:00:00<\/td>\n<\/tr>\n
0 0 0 1-7 * SUN<\/td>\n
every first sunday of the month at 00:00:00<\/td>\n
Oct 1 (SUN), 2017 00:00:00; Nov 5 (SUN), 2017 00:00:00<\/td>\n<\/tr>\n
11 5 23 * * *<\/td>\n
daily at 23:05:11<\/td>\n
Oct 16, 2017 23:05:11; Oct 17, 2017 23:05:11<\/td>\n<\/tr>\n
30 5 \/6 * * *<\/td>\n
every 6 hours at 5 minutes and 30 seconds<\/td>\n
06:05:30; 12:05:30; 18:05:30; 00:05:30<\/td>\n<\/tr>\n
*\/15 * * * * *<\/td>\n
every 15 seconds<\/td>\n
09:00:15; 09:00:30; \u2026 09:03:30; 09:03:45; 09:04:00<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
For more info about the Azure Function intervals, please take a look at Armin Reiter’s blog post: https:\/\/codehollow.com\/2017\/02\/azure-functions-time-trigger-cron-cheat-sheet\/<\/a>.<\/p>\n
Conclusion<\/h2>\n
Executing daemon again SharePoint data, and performing actions in time intervals, was requirement in almost all medium and large SharePoint projects. With SharePoint server (on premises), we have SharePoint timer jobs, which do this task just fine. In SharePoint Online, however, we need to delegate this logic to some external system. Azure App Service, in all different flavors, is actually a legitimate way to do this, and Azure Functions (which are based on the Azure App Service) offer a nice gateway to do this. Especially if the code which needs to be run is a fairly simple and short, there is really no good reason against using the functions. One might argue that the development story around functions is not yet very mature, with all the issues with debugging, lifecycle etc, that long running operations are not really supported on the dynamic pricing plan (5 minutes timeout), and all those arguments stand. However, for simple operations, for simple code blocks, Azure Functions will do just fine as timer jobs replacement.<\/p>\n
\n\t\t\t  
\n\t\t\t  <\/div><\/div>\n\t\t