Continuous Deployment of SharePoint Apps (a.k.a Add-Ins) is tricky. The app package is intended to be created for every environment because it must contain a client id and secret. There is also no simple way to automatically trust the app. I show you in this post how you can create a release pipeline with VSTS Release Management and some PowerShell scripts to fully automate the deployment of SharePoint Apps. The example deploys to SharePoint Online – but the same scripts can be used to deploy to your on premise SharePoint.
Overview: Continuous Deployment with Release Management
The idea of continuous deployment is, that each push to your repository automatically triggers a build that is deployed to an environment. Using branches and different builds you can deploy different versions of your code to different stages. A simple release pipeline could look like this.
The developers work in their own Azure Subscriptions (MSDN) and test their code there. If they push a commit to the develop branch a continuous integration build is triggered that runs all the unit tests and packages the app. The app is then deployed to a resource group in an integration subscription and can there be tested automatically and/or manually.
If the code is ready for a release to production it is merged to the master branch using a pull request. The merge also triggers a build that deploys to different subscriptions in sequence. You can add automated or manual approval steps.
Package your app and web site
To create a package that we can deploy to multiple environments we have to create packages with tokens that we can replace during deployment – for the app and for the app web.
In the publish dialog of the app (right click and select Publish…) you can add tokens for the app (Client Id and Client Secret).
Here you also can create a new publish profile for the app web. Use “Web deploy package” and add a token for the site name.
To replace the Client ID, Client Secret and optional other configurations (like Connection Strings or Endpoints) you can use a Parameter.xml file. The file is used by WebDeploy and contains XPath expressions for the values that should be replaced. Just add the file to the root of your project and set the build action to none.
Here is a sample parameter file:
<?xml version="1.0" encoding="utf-8" ?> <parameters> <parameter name="ClientId" description="Value for ClientId here." defaultvalue="__ClientId__" tags="applicationSettings"> <parameterentry kind="XmlFile" scope="web.config$" match="/configuration/appSettings/add[@key='ClientId']/@value" /> </parameter> <parameter name="ClientSecret" description="Value for ClientSecret here." defaultvalue="__ClientSecret__" tags="applicationSettings"> <parameterentry kind="XmlFile" scope="web.config$" match="/configuration/appSettings/add[@key='ClientSecret']/@value" /> </parameter> <parameter name="DBEndPointUrl" description="" defaultvalue="__DBEndPointUrl__" tags="applicationSettings"> <parameterentry kind="XmlFile" scope="web.config$" match="/configuration/applicationSettings/CalculatorApp.DocumentDbStores.Properties.Settings/setting[@name='EndPointUrl']/value" /> </parameter> <parameter name="DBAuthorizationKey" description="" defaultvalue="__DBAuthorizationKey__" tags="applicationSettings"> <parameterentry kind="XmlFile" scope="web.config$" match="/configuration/applicationSettings/CalculatorApp.DocumentDbStores.Properties.Settings/setting[@name='AuthorizationKey']/value" /> </parameter> </parameters>
Build
To create the app during the server build we have to specify /P:IsPackaging=True as the MSBuild Argument for the Visual Studio Build task. To create the package for the app web from the previous step you also have to specify /p:DeployOnBuild=true /p:PublishProfile=Package. This will result in two packages getting created. I’ve already written about that. To make sure the correct package is published you can force it to go to the staging directory with the following argument: /p:PackageLocation=”$(build.stagingDirectory)”.
To publish the deployment scripts and the app package you can also copy them to the staging directory using a the copy file task.
Then you create the build output by publishing the entire staging directory.
The result is a neat little package that contains the app, the deployment script, and the web deploy package with the tokens that are replaced before deployment to each environment.
Release
In the release definition you have to add all the tokens as variables for your environment.
The release then calls the script Replace-Tokens.ps1 that replaces all the tokens in the parameter file.
The script is originally from my friend Colin Dembovski. He also created a build task that can directly used.
My own implementation of the script can also be found on github. The trick is that the script pulls the variables from the environment over the environment provider from PowerShell. So there is no need to specify individual tokens. Just add new Tokens to you parameter file and to the environment in release Management.
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true, ValueFromPipeline=$true, Position=0)]
[ValidateNotNullOrEmpty()]
[string]$RootFolder,
[Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true, ValueFromPipeline=$true, Position=1)]
[ValidateNotNullOrEmpty()]
[string]$FileName
)
function Replace-Tokens
{
[CmdletBinding()]
Param
(
# Hilfebeschreibung zu Param1
[Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true, ValueFromPipeline=$true, Position=0)]
[ValidateNotNullOrEmpty()]
[ValidateScript({ Test-Path $_ })]
[string]$FileFullName
)
Write-Verbose "Replace tokens in '$FileFullName'..."
# get the environment variables
$vars = Get-ChildItem -Path env:*
# read in the setParameters file
$contents = Get-Content -Path $FileFullName
# perform a regex replacement
$newContents = ""
$contents | ForEach-Object {
$line = $_
if ($_ -match "__(\w+)__") {
$setting = $vars | Where-Object { $_.Name -eq $Matches[1] }
if ($setting) {
Write-Verbose "Replacing key '$($setting.Name)' with value '$($setting.Value)' from environment"
$line = $_ -replace "__(\w+)__", $setting.Value
}
}
$newContents += $line + [Environment]::NewLine
}
Write-Verbose -Verbose "Save content to '$FileFullName'."
Set-Content $FileFullName -Value $newContents
Write-Verbose "Done"
}
Write-Verbose "Look for file '$FileName' in '$RootFolder'..."
$files = Get-ChildItem -Path $RootFolder -Recurse -Filter $FileName
Write-Verbose "Found $($files.Count) files."
$files | ForEach-Object { Replace-Tokens -FileFullName $_.FullName }
Write-Verbose "All files processed."
The deploy the website we have to execute the batch file from the deployment package. As Parameters we have to specify the msdeploy.axd from the azure website (/M:https://$(WebDeploySiteName).scm.azurewebsites.net:443/msdeploy.axd) and the user name and passwort from the environment variables (/u:$(AzureUserName) /p:$(AzurePassword)). We also have to specify /y to do a quite deployment and set the authentication to basic.
The most tricky part is to deploy the app package. There is no out of the box solution to automate the trust of apps. Apps in the App Catalogue cannot be deployed automatically at all. So the only way I know is to use side loading to directly deploy an app to a site and use IE automation to “click” the trust button.
You will need your own build agent to run the script and you will have to install office. You also could install some assemblies to the GAC – but I found it easier to install office.
So let’s step thru the Deploy-SPApp.ps1 script and go into details what each step does.
For each environment we use a separate client id for security purpose. So the first part is to open the app (a zip file) and read the app manifest.
Write-Verbose "Open zip file '$Path'..."
$zip = [System.IO.Compression.ZipFile]::Open($Path, "Update")
$fileToEdit = "AppManifest.xml"
$file = $zip.Entries.Where({$_.name -eq $fileToEdit})
Write-Verbose "Read app manifest from '$file'."
$desiredFile = [System.IO.StreamReader]($file).Open()
[xml]$xml = $desiredFile.ReadToEnd()
$desiredFile.Close()
We then replace the client id with the value from the environment variable.
if ($env:ClientId){
Write-Verbose "Found ClientId '$env:ClientId' in environment. Replace it in app."
$xml.App.AppPrincipal.RemoteWebApplication.ClientId = $env:ClientId
}
There also might be more values that have to be replaced (in case you use remote event receivers). Then we update the app manifest and save the file.
# Save file Write-Verbose "Save manifest to '$file'." $desiredFile = [System.IO.Stream]($file).Open() $desiredFile.SetLength(0) $xml.Save($desiredFile) $desiredFile.Flush() $desiredFile.Close() $desiredFile.Dispose() # //... $zip.Dispose()
In the next part open a connection the SharePoint site using CSOM.
Write-Host "Connect to '$WebUrl' as '$DeployUserName'..." $clientContext = New-Object Microsoft.SharePoint.Client.ClientContext($webUrl) $clientContext.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($DeployUserName, (ConvertTo-SecureString $DeployPassword -AsPlainText -Force)) $web = $clientContext.Web $clientContext.Load($web) $clientContext.ExecuteQuery(); Write-Host "Successfully connected to '$WebUrl'..."
The next step is to install the app using side loading and wait for SharePoint to complete the installation.
function Install-App($clientContext, $appPackage, $productId) {
$appName = [System.IO.Path]::GetFileNameWithoutExtension($appPackage)
$web = $clientContext.Web
Write-Verbose "Start to install app $appName..."
# Try to uninstall any existing app instances first.
Uninstall-App $clientContext $productId
Write-Verbose "Installing app $appName..."
$appInstance = $web.LoadAndInstallAppInSpecifiedLocale(([System.IO.FileInfo]$appPackage).OpenRead(), $web.Language)
$clientContext.Load($appInstance)
$clientContext.ExecuteQuery()
$appInstance = WaitForAppOperationComplete $clientContext $appInstance.Id
if (!$appInstance -Or $appInstance.Status -ne [Microsoft.SharePoint.Client.AppInstanceStatus]::Installed)
{
if ($appInstance -And $appInstance.Id)
{
Write-Error "App installation failed. To check app details, go to '$($web.Url.TrimEnd('/'))/_layouts/15/AppMonitoringDetails.aspx?AppInstanceId=$($appInstance.Id)'."
}
throw "App installation failed."
}
return $appInstance.Id
}
Then we start with the magic. We create an internet explorer instance and navigate to the authorize page to trust the app.
$authorizeURL = "$($WebUrl.TrimEnd('/'))/_layouts/15/appinv.aspx?AppInstanceId={$AppInstanceId}"
$ie = New-Object -com internetexplorer.application
$ie.Visible = $false
$ie.Navigate2($authorizeURL)
On prem you can skip the next part – but in the cloud we have sign in to Office 365. We enter username and password in the corresponding input fields and submit the page using JavaScript.
if ($ie.Document.Title -match "Sign in to Office 365.*") {
Write-Verbose "Authenticate $UserName to O365..."
# Authorize against O365
$useAnotherLink = $ie.Document.getElementById("use_another_account_link")
if ($useAnotherLink) {
WaitFor-IEReady $ie
$useAnotherLink.Click()
WaitFor-IEReady $ie
}
$credUseridInputtext = $ie.Document.getElementById("cred_userid_inputtext")
$credUseridInputtext.value = $UserName
$credPasswordInputtext = $ie.Document.getElementById("cred_password_inputtext")
$credPasswordInputtext.value = $Password
WaitFor-IEReady $ie
# make a jQuery call
$result = Invoke-JavaScript -IE $ie -Command "`nPost.IsSubmitReady();`nsetTimeout(function() {`nPost.SubmitCreds();`n}, 1000);"
WaitFor-IEReady $ie -initialWaitInSeconds 5
}
In the authorization form we just have to click the correct button. And that’s it. The app is now deployed and trusted you you can run your automated (or manual) tests against it.
if ($ie.Document.Title -match "Do you trust.*") {
sleep -seconds 5
$button = $ie.Document.getElementById("ctl00_PlaceHolderMain_BtnAllow")
if ($button -eq $null) {
$button = $ie.Document.getElementById("ctl00_PlaceHolderMain_LnkRetrust")
}
if ($button -eq $null) {
throw "Could not find button to press"
}else{
$button.click()
WaitFor-IEReady $ie
#if the button press was successful, we should now be on the Site Settings page..
if ($ie.Document.title -like "*trust*") {
throw "Error: $($ie.Document.body.getElementsByClassName("ms-error").item().InnerText)"
}else{
Write-Verbose "App was trusted successfully!"
}
}
}else{
throw "Unexpected page '$($ie.LocationName)' was loaded. Please check your url."
}
Conclusion
Building an end to end release pipeline for SharePoint online apps is very complex because it involves a lot of steps. Apps are not really designed for automatic deployment. I tried to pick out the most interesting parts here – but if your missing a piece of the puzzle then drop me a comment in the blog or contact me via twitter and I will try to help. The good thing is that the tooling with new new build and release management system in TFS / VSTS makes it a lot easier to build and deploy and to focus on the scripts and tasks for the app.
writeabout.net 
In my build definition I have added 2 steps :
1- Visual Studio Build Definition
2- Publish artifacts
After the build definition is executed successfully , I am not able to see Artifacts Explorer.
Can you please let me know.. how can I check the artifacts generated?
Set the “path to publish” of the “publish build artifacts task” to $(build.artifactstagingdirectory). Use a cop file task to copy your apps in there. Add the package location switch to your visual studio task like this: https://mkaufmannblog.files.wordpress.com/2016/02/image7.png
Hi Mike, can you pls tell me, what product is use for your deployment diagram here
PowerPoint 🙂 The images are taken from the office and cloud Visio stencils.
Very nice! Thanks for showing this and stating out what’s not possible esspecially 😉
Hi Mike,
Can you please provide me step to install wsp file (Farm solution)?
You can i.e. use SharePOint DSC https://github.com/PowerShell/SharePointDsc
See https://writeabout.net/2016/05/31/powershell-dsc-sharepoint-xsharepoint/ for more infos.
Hi mike,
I followed the steps but I have the error “The type or namespace name ‘SharePoint’ does not exist ‘ (are you missing an assembly reference?)” in the build process.
Can you tell me if you have ever had the same problem and how you solved it?