Building a Continuous Integration Environment for Sitecore Part 8 – Using Unicorn for Sitecore Item Syncronization

This is part 8 of the series of post that I am doing about building a Continuous Integration Environment using Sitecore, TeamCity, Octopus Deploy, and Unicorn.

Part 1 – Setting Up TeamCity
Part 2 – Setting up OctopusDeploy
Part 3 – Setting up SQL Server for Sitecore
Part 4 – IIS
Part 5 – Octopus Environment and Deployment Configuration
Part 6 – TeamCity Project
Part 7 – OctopusDeploy Project
Part 8 – Sitecore Item Synchronization using Unicorn
Part 9 – Displaying Build number on Sitecore Home Page & Tagging Git
Part 10 – config transformations using Octopus Deploy
Part 11 – Deploying to a Sitecore Multi-Instance Configuration

Thoughts so far

I deliberately put off including anything to do with Sitecore item synchronization until I had my build and deployment process. I wanted to get that working first so that I can resolve any issues before throwing Sitecore into the mix. If I had a failure in either TeamCity or Octopus I wanted to narrow it down quickly and not worry about Sitecore.

Why Unicorn

If you have done any development work with Sitecore, you know that one of the biggest issues is synchronizing items. Templates, layouts, etc

To get around it,

  • All your dev’s are working on a single shared Sitecore instance.
  • Every time a dev does a code checkin they have to also generate a Sitecore Package of all the changes they made
  • Use a tool

I am not a fan of people working on a shared database, even when not using Sitecore so that option is out.

Requiring people to create a package, you are going to end up with 100’s of package files, tracking them becomes a task in itself

Which leaves the last option, use a tool. The big daddy of the market is Hedgehog’s TDS. I have used TDS in the past. I know I never used the full capabilities of the product, but I never felt 100% happy using the product. I feel that it suffers from some of the issues with having dev’s create Sitecore packages, in that it requires a human to remember to create the .item files and then remember to sync. There is a lot of upfront project creation, a project for master, a project for core. Want to take a guess at how many times I forgot to create an item, or sync after doing a merge and then caused problems. Too many times to count.

And if your merge tool resolves a conflict in an .item file that TDS controls, and you and several team members have then sync’ed that file, you will have fun reversing out whatever you got left with. Once again this happened to me, spent way to much time resolving a template problem because of an item being merged “incorrectly.”

I never liked having to use Sitecore Courier either to deploy changes as that basically needs you to create a Sitecore Package which includes changes that need to be made. Trying to work out changes made on a multi-team project, with multiple check ins on git, and even with tools it was not fun trying to work out why a change was not present. Once again not having fun resolving issues because of missing/corrupt templates within Sitecore because the courier package was missing something.

I was told about Unicorn and I have been using it for several months and it does appear to be resolving some of the issues that I have experienced in the past.

  • Want to have all your templates sync’ed to disk when you maintain them in Sitecore – Yes
  • After doing a merge, automatically sync changes made – Yes using Transparent Sync
  • Not having to worry about creating packages to deploy via Sitecore Courier – Yes

Unicorn ticked these and many more boxes for me.

I am sure that anybody who has worked on Sitecore for more than 5 minutes has their pain points in regards to item synchronization

Getting Started

I am not going to go into too much detail because the readme on the home page details a lot, I am just going to detail what I did to get it working. I am also going to assume you have already set up your solution within Visual Studio.

From your web project, start the package manager console and type in the following


Install-Package Unicorn

Or just do it from the NuGet Package manager dialog. At the time of this post, I am using Unicorn 3.2.

Unicorn uses Yaml for item sync what is soo much easier for resolving merge conflicts than resolving Sitecore .item file conflicts ever where.

Unicorn comes with a Unicorn.configs.default.example. As with everything Sitecore, remove the .example and its ready to use. I initially started using this file to get my feet wet, but I feel that the default config file supplied wants to put too much under Source Control. Convention that I work by is that only items that are created for the solution should be put under source control. Actual content should never be placed under source control. Items included with the default Sitecore database should never be put under source control. Items created via packages should never be put under source control

This is the sample supplied to put items under the layout under Unicorn’s control.


<!-- Master Layout items (and renderings, sublayouts, etc) -->
<include database="master" path="/sitecore/layout">
	<!-- NOTE: You cannot use excludes with Transparent Sync. See https://github.com/kamsar/Unicorn/wiki/The-Transparent-Sync-Guide -->
	<exclude path="/sitecore/layout/Simulators" />
	<exclude path="/sitecore/layout/Layouts/System" />
	<exclude path="/sitecore/layout/Renderings/App Center Sync" />
	<exclude path="/sitecore/layout/Renderings/System" />
	<exclude path="/sitecore/layout/Renderings/Social MVC" />
	<exclude path="/sitecore/layout/Sublayouts/App Center Sync" />
	<exclude path="/sitecore/layout/Sublayouts/Social" />
	<exclude path="/sitecore/layout/Placeholder Settings/App Center Sync" />
</include>

I wanted to use Transparent Sync for development, the comment says you cannot use excludes’ with Transparent Sync

Even if I was not going to use transparent sync, I feel that you have to include too many exceptions using an exclude node. As most of development is done using MVC, all the renderings should be placed within a single folder within the Renderings folder, and then that single folder is put under Source Control

To include this within Unicorn all that is needed is


<include name="Layout.Renderings.ProjectName" database="master" path="/sitecore/layout/Renderings/ProjectName" />

The hardest part I discovered was creating meaningful unique names for each item you want to put under source control.

If you follow the convention of creating a custom folder, and then adding in the relevant items into the custom folder, there are not many folders that actually need to be put under Unicorn’s control if you follow the convention of creating a sub folder.

This is my full unicorn config file


<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <unicorn>
      <configurations>
        <configuration name="Project" description="Custom configuration for CompanyName">
          <predicate type="Unicorn.Predicates.SerializationPresetPredicate, Unicorn" singleInstance="true">
            <include name="Home" database="master" path="/sitecore/layout/Controllers/Home" />
            <include name="Layout.Layouts" database="master" path="/sitecore/layout/Layouts/Project" />
            <include name="Layout.Models" database="master" path="/sitecore/layout/Models/Project" />
            <include name="Layout.PlaceholderSettings" database="master" path="/sitecore/layout/Placeholder Settings/Project" />
            <include name="Layout.Renderings.Project" database="master" path="/sitecore/layout/Renderings/Project" />
            <include name="System.Dictionary.Project" database="master" path="/sitecore/system/Dictionary/Project" />
            <include name="System.Languages" database="master" path="/sitecore/system/Languages" />
            <include name="System.Workflows" database="master" path="/sitecore/system/Workflows" />
            <include name="Templates.Project" database="master" path="/sitecore/templates/Project" />
            <include name="System.HtmlEditorProfiles" database="core" path="/sitecore/system/Settings/Html Editor Profiles/Project" />
          </predicate>
        </configuration>
      </configurations>
    </unicorn>
  </sitecore>
</configuration>

With the exception of Languages and Workflows, all items are created by default in their own custom folder. I am using Project here but this can be replace with any relevant name.

As you can see, Workflow and Language exceptions aside, none of the items supplied by Sitecore are placed under source control.

There were a few more changes that I needed to make. All done through my own patch file so that I don’t have any problems during an upgrade.


    <unicorn>
      <defaults>
        <targetDataStore type="Rainbow.Storage.SerializationFileSystemDataStore, Rainbow" useDataCache="false">
          <patch:attribute name="physicalRootPath">~/Unicorn/$(configurationName)</patch:attribute>
        </targetDataStore>
        <dataProviderConfiguration type="Unicorn.Data.DataProvider.DefaultUnicornDataProviderConfiguration, Unicorn">
          <patch:attribute name="enableTransparentSync">true</patch:attribute>
        </dataProviderConfiguration>
      </defaults>
    </unicorn>
    <settings>
      <setting name="Rainbow.SFS.SerializationFolderPathMaxLength">
        <patch:attribute name="value">110</patch:attribute>
      </setting>
    </settings>

Rainbow.Storage.SerializationFileSystemDataStore Is to set the folder where all the yaml files are to be created. My gitignore file will not include anything within the data folder, so I had to specify another location than the one supplied by default so that it would be automatically picked up as I didn’t have to put in ignore rule exception in my .gitignore file. As you can guess not a big fan of exceptions

Unicorn.Data.DataProvider.DefaultUnicornDataProviderConfiguration is for enabling Transparent Sync.

Another change that was necessary for me was modify the setting Rainbow.SFS.SerializationFolderPathMaxLength. I set this value to 110. This was done purely because of my development environment and you may not need to alter this from the default value. This setting was changed via my own .config file and not by directly modifying rainbox.config

First Run

After reading the documentation, all you need to do is run /unicorn.aspx. This will display the Unicorn Control Panel.

Click Sync to get your first initial files. You could have already switched on Transparent Sync but I initially had it disabled. The xml above is from my final config file, and not what I used initially.

Once that has completed, go to the config files and enable Transparent Sync.

If you had enabled transparent sync before going to the Unicorn Control Panel, you will get a warning about no files and that you will need to do an initial sync. Quickest way to do this after sync. Delete the unicorn folder.

Testing

I wanted to make sure everything was working, so I created a template, then I did the following

  • Rename the .yaml file. Change automatically applied
  • Copied the yaml file, and modified the ID. New template created automatically
  • Deleted the yaml file. Template automatically deleted.

I performed more tests than the ones I have listed, and no matter what I did, Unicorn kept my Sitecore system in site.

Now that this is finished, its time to modify my build and deployment process.

TeamCity

The only change I had to make was to update my .nuspec file to include the Unicorn folder. No other changes where required to TeamCity.


  <files>
    <file src="App_Config\**\*.config" target="App_Config" />
    <file src="Assets\**\*.*" target="Assets" />
    <file src="bin\*.dll" target="bin" />
    <file src="Views\**\*" target="Views" />
    <file src="default.css" target="" />
    <file src="default.htm.sitedown" target="" />
    <file src="default.js" target="" />
    <file src="Global.asax" target="" />
    <file src="Web.*.config" target="" />
    <file src="Web.config" target="" />
    <file src="webedit.css" target="" />
    <file src="emails\**\*.*" target="emails" />
    <file src="*.ps1" target="" />
    <file src="Unicorn\**\*.*" target="Unicorn" />
  </files>
</package>

This is my updated .NuSpec file. I only had to add in the line


<file src="Unicorn\**\*.*" target="Unicorn" />

Once the Unicorn files where included in the TeamCity artefact everything else will be performed by Octopus.

Octopus Deploy

I needed to do a little bit more work with my Octopus Deploy project to deal with Unicorn, than with what I had to change with TeamCity. I performed these changes over multiple steps, and I broke the build because I wanted to do a small change and verify changes made manually rather than doing one massive big bang approach and then spending time working out what broke.

Moving Unicorn Files out of Webroot

As you can see from above, based on the Octopus Deploy project created in Step 7 Unicorn’s yaml files are going to be written to D:\WebSites\ProjectName.CI\Website\Unicorn when the Octopus step “DeploySite” is executed. I did not want to have the sync files within the web root. So the first task I set myself was to move the files out of the web root.

When I deploy to CI this is the folder structure being used

D:\WebSites\ProjectName.CI\data
D:\WebSites\ProjectName.CI\Website

The Unicorn yaml files where stored in D:\WebSites\ProjectName.CI\Website\Unicorn.

I just decided that I would move the Unicorn folder from D:\WebSites\ProjectName.CI\Website\ to D:\WebSites\ProjectName.CI\Data\

Step Template – Move Unicorn files out of webroot

This first thing I needed to do was to create a new custom step template that will

  • Copy the files to the new location
  • Delete the files from the old location

I just decided on doing a copy command instead of a move. Feel free to change if required.

This is the script to be imported into Octopus


{
  "Id": "ActionTemplates-62",
  "Name": "Move Unicorn files out of webroot",
  "Description": null,
  "ActionType": "Octopus.Script",
  "Version": 9,
  "Properties": {
    "Octopus.Action.Package.NuGetFeedId": "feeds-builtin",
    "Octopus.Action.Script.Syntax": "PowerShell",
    "Octopus.Action.Script.ScriptSource": "Inline",
    "Octopus.Action.RunOnServer": "false",
    "Octopus.Action.Script.ScriptBody": "$dataFolder = $OctopusParameters['data.folder']\r\n$websiteFolder  = $OctopusParameters['website.folder']\r\n\r\nWrite-Output \"dataFolder: \" $dataFolder\r\nWrite-Output \"websiteFolder: \" $websiteFolder\r\n\r\nif((Test-Path -Path $dataFolder\\Unicorn )){\r\n    Write-Output \"[io.directory]::delete(\"\"$dataFolder\\Unicorn\"\")\"\r\n    Get-ChildItem -Path \"$dataFolder\\Unicorn\\*\" -Recurse | Remove-Item -Force -Recurse\r\n    remove-item -path $dataFolder\\Unicorn -Recurse -Force\r\n}    \r\nWrite-Output \"copy-item -path $websiteFolder\\Unicorn -destination $dataFolder -recurse -force\"\r\ncopy-item -path $websiteFolder\\Unicorn -destination $dataFolder -recurse -force\r\n\r\nWrite-Output \"[io.directory]::delete(\"\"$websiteFolder\\Unicorn\"\")\"\r\n#[io.directory]::delete(\"$websiteFolder\\Unicorn\", true)\r\n    Get-ChildItem -Path \"$websiteFolder\\Unicorn\\*\" -Recurse | Remove-Item -Force -Recurse\r\n    remove-item -path $websiteFolder\\Unicorn -Recurse -Force\r\n\r\n"
  },
  "Parameters": [],
  "$Meta": {
    "OctopusVersion": "3.3.1",
    "Type": "ActionTemplate"
  }
}

No parameters are needed as it will be using variables already defined

Update Deployment Process

Within Octopus Deploy web portal, load the project, and select Process. Click Add Step

oct_prog43

Scroll to select “Move Unicorn files out of webroot”

Ensure that the roles CD and CD are selected. No other changes are required. Save the process

The newly created step needs to be moved. Within the process, click Reorder steps

oct_proj50

Move the step “Move Unicorn files out of webroot” from being the last step to being the step immediatelty after Deploy Site.

At this point I saved everything and triggered a build from TeamCity. Everything worked as I expected. Remember that I am not finished so even if the Deployment Process was broken I expected this to happen.

I needed to let Unicorn know where its yaml files where located.

In Visual Studio I needed to update my patch file


<unicorn>
  <defaults>
    <targetDataStore type="Rainbow.Storage.SerializationFileSystemDataStore, Rainbow" useDataCache="false">
      <patch:attribute name="physicalRootPath" xdt:Transform="Replace">#{data.folder}\Unicorn\$(configurationName)</patch:attribute>
    </targetDataStore>
    <dataProviderConfiguration type="Unicorn.Data.DataProvider.DefaultUnicornDataProviderConfiguration, Unicorn" xdt:Transform="Replace">
      <patch:attribute name="enableTransparentSync">false</patch:attribute>
    </dataProviderConfiguration>
  </defaults>
</unicorn>

I disabled transparent sync as reading the web this was shown to be a good practice on non development environments. I will be detailing how I got unicorn sync’ing working shortly

#{data.folder} is an OctopusDeploy variable, and when the site is being deployed, it will be replaced with the variable’s value

Pushed to git and triggered a new Build / Deployment processes. Once it had finished I checked everything and was satisfied with progress. The files where in the new location, and Unicorn knew where the files where. Time to get the Deployment process to perform the synchronization.

Sync’ing from Octopus

As I had disabled transparent sync. I needed to use the Automated Tool Api. The documentation on the site is very comprehensive so I don’t see the need to duplicate it all here.

On the Unicorn / doc / Powershell Remote Scripting page, you need to download MicroCHAP.dll. Place this within the Tools folder on the IIS Server. I eventually placed this file in folder D:\Tools\UnicornSync. Make a note of where you saved this file. This file is needed to authenticate with Unicorn

Environment – Variable Sets

Within the Octopus Deploy Web Portal, select Library | Variable Sets | External Software Paths

Just add in a new line that will specify the location of MicroCHAP.dll. I added in a variable named MicroChap.path, and a value of D:\Tools\UnicornSync\MicroCHAP.dll. This is important as this variable is used in new Step Template that will be created

Don’t forget to save the page

Step Template – Unicorn Sync

This step is pretty much a copy of the sample.ps1 file that is in the same folder on GitHub as MicroCHAP.dll

This step will call Unicorn’s API and tell it to perform a sync. In short you call the Unicorn Control Panel and tell it to perform a sync.

There are a few variables that need to be passed to perform the sync successfully.

Pass in the SharedSecret so that Unicorn can authenticate itself
The list of Unicorn configurations to sync
The name of the website

This is the step template to import


{
  "Id": "ActionTemplates-61",
  "Name": "Unicorn Sync",
  "Description": "Calls Unicorn sync",
  "ActionType": "Octopus.Script",
  "Version": 8,
  "Properties": {
    "Octopus.Action.Package.NuGetFeedId": "feeds-builtin",
    "Octopus.Action.Script.Syntax": "PowerShell",
    "Octopus.Action.Script.ScriptSource": "Inline",
    "Octopus.Action.RunOnServer": "false",
    "Octopus.Action.Script.ScriptBody": "$ErrorActionPreference = 'Stop'\r\n\r\n$microchap = $OctopusParameters['MicroChap.path']\r\n\r\nAdd-Type -Path $microchap\r\n\r\nFunction Sync-Unicorn {\r\n    Param(\r\n        [Parameter(Mandatory=$True)]\r\n        [string]$ControlPanelUrl,\r\n\r\n        [Parameter(Mandatory=$True)]\r\n        [string]$SharedSecret,\r\n\r\n        [Parameter(Mandatory=$True)]\r\n        [string[]]$Configurations,\r\n\r\n        [string]$Verb = 'Sync'\r\n    )\r\n\r\n    # PARSE THE URL TO REQUEST\r\n    $parsedConfigurations = ($Configurations) -join \"^\"\r\n\r\n    $url = \"{0}?verb={1}&configuration={2}\" -f $ControlPanelUrl, $Verb, $parsedConfigurations\r\n\r\n    Write-Host \"Sync-Unicorn: Preparing authorization for $url\"\r\n\r\n    # GET AN AUTH CHALLENGE\r\n    $challenge = Get-Challenge -ControlPanelUrl $ControlPanelUrl\r\n\r\n    Write-Host \"Sync-Unicorn: Received challenge: $challenge\"\r\n\r\n    # CREATE A SIGNATURE WITH THE SHARED SECRET AND CHALLENGE\r\n    $signatureService = New-Object MicroCHAP.SignatureService -ArgumentList $SharedSecret\r\n\r\n    $signature = $signatureService.CreateSignature($challenge, $url, $null)\r\n\r\n    Write-Host \"Sync-Unicorn: Created signature $signature, executing $Verb...\"\r\n\r\n    # USING THE SIGNATURE, EXECUTE UNICORN\r\n    $result = Invoke-WebRequest -Uri $url -Headers @{ \"X-MC-MAC\" = $signature; \"X-MC-Nonce\" = $challenge } -TimeoutSec 10800 -UseBasicParsing\r\n\r\n    $result.Content\r\n}\r\n\r\nFunction Get-Challenge {\r\n    Param(\r\n        [Parameter(Mandatory=$True)]\r\n        [string]$ControlPanelUrl\r\n    )\r\n\r\n    $url = \"$($ControlPanelUrl)?verb=Challenge\"\r\n\r\n    $result = Invoke-WebRequest -Uri $url -TimeoutSec 360 -UseBasicParsing\r\n\r\n    $result.Content\r\n\r\n}\r\n\r\n$websiteName = $OctopusParameters['website.name']\r\n$SharedSecret = $OctopusParameters['SharedSecret']\r\n$Configurations = $OctopusParameters['Sync.Configurations']\r\n\r\nWrite-Output \"websiteName\" $websiteName\r\nif ( $SharedSecret ) {\r\n    Write-Output \"Shared secret supplied\"\r\n} \r\nelse {\r\n    Write-Output \"No Shared Secret Supplied\"\r\n}\r\nWrite-Output \"Configurations\" $Configurations\r\n\r\nif ( $Configurations ) {\r\n    $configs = $Configurations.split(\"`n\")\r\n}\r\nelse {\r\n    $configs = \" \"\r\n}\r\n\r\nSync-Unicorn -ControlPanelUrl \"http://$websiteName/unicorn.aspx\" -SharedSecret $SharedSecret -Configurations $configs\r\n"
  },
  "Parameters": [
    {
      "Name": "SharedSecret",
      "Label": "Shared Secret",
      "HelpText": null,
      "DefaultValue": null,
      "DisplaySettings": {
        "Octopus.ControlType": "Sensitive"
      }
    },
    {
      "Name": "Sync.Configurations",
      "Label": "List of Configurations to Sync",
      "HelpText": null,
      "DefaultValue": null,
      "DisplaySettings": {
        "Octopus.ControlType": "MultiLineText"
      }
    },
    {
      "Name": "website.name",
      "Label": "Website Name",
      "HelpText": null,
      "DefaultValue": null,
      "DisplaySettings": {
        "Octopus.ControlType": "SingleLineText"
      }
    }
  ],
  "$Meta": {
    "OctopusVersion": "3.3.1",
    "Type": "ActionTemplate"
  }
}

Update Deployment Process

Within Octopus Deploy web portal, load the project, and select Process. Click Add Step

oct_prog43

Scroll to select “Unicorn sync”

oct_proj42

Make a note of what you use for the shared secret, as we will need to store this in a patch file. To generate the shared secret I used https://www.passwordgenerator.com and generated a 128 digit password to be used as the shared secret.

If you only have a single configuration file, you do not need to supply the List of Configurations to Sync. It can be left blank.

Save the step. There is no need to change the step order in the deployment process. I am happy having this as the last step run in the process.

In Visual Studio I needed to update my patch file


    <unicorn>
      <authenticationProvider type="Unicorn.ControlPanel.Security.ChapAuthenticationProvider, Unicorn">
        <SharedSecret>MySharedSecretGeneratedFromPasswordsGeneratorDotCom</SharedSecret>
      </authenticationProvider>
    </unicorn>

Save the changes and push to git.

After watching the whole process ( it takes a full 10 minutes ) I logged into Sitecore and yes, my custom template that I had created on my development environment was there. As with my local machine I did a few tests. Make sure that it would rename a Sitecore Item template, would delete a Sitecore Item template. Everything I did correctly updated the Sitecore instance on my CI environment.

Thoughts

Now after 8 steps I now have a system that will make sure that any change that is made, either to Sitecore or my custom code, is deployed to my CI environment.

So what’s the first thing I did. I broke it.

How?

I added a new template, and forgot to run “git add” before running “git commit” I know how ironic especially after what I said at the start of this post about relying on developers to do things manually.

Upgrading Unicorn

I initially was using Unicorn 3.1 and while building this process, 3.2 was released. Because I did not modify any of the original files, NuGet had no problem upgrading to the newest version of Unicorn.

Next Steps

First a break. I initially did all this over 4 days. I has taken me nearly two weeks to blog about this.

Right now I am looking to create a deploy process to satisfy the following:

Deployment Process

Your environment may be different from mine, either in name or number.

Next I need to extend Octopus to deploy to the next environment, which is my instance is QA.

Advertisements

My musing about anything and everything

Tagged with: , , , , , ,
Posted in C#, CI, Continuous Integration, Octopus, Octopus Deploy, Sitecore, TeamCity, Unicorn, Visual Studio
21 comments on “Building a Continuous Integration Environment for Sitecore Part 8 – Using Unicorn for Sitecore Item Syncronization
  1. Nice series of posts Darren. Very well written. It’s great to see how different teams have setup CI.
    Just wanted to chime in about some of the little comments on TDS you mentioned, as I feel some of TDS’s features have been missed, which may have been why you had these troubles in the first place.

    “I feel that it suffers from some of the issues with having dev’s create Sitecore packages, in that it requires a human to remember to create the .item files and then remember to sync.”
    – on this, if you’re using Sitecore Rocks with TDS, items are automatically synced through the Rocks – TDS connector, so you won’t need to remember them. Also in TDS v5.5, the new Auto-Sync feature also brings in browser support for this. So now items are auto synced, even if you’re using a web browser for Sitecore.

    “There is a lot of upfront project creation, a project for master, a project for core.”
    – The very basic TDS Project wizard only requires two settings, Web URL and Website Folder. A TdsGlobal.config also allows a one-time setup for these for ALL projects… so you don’t have to set it up multiple times. Therefore I don’t feel like there’s a lot needed for project setup. However the more advanced you get, the more settings you can turn on if you want…. but for basic setup it’s super easy.

    “And if your merge tool resolves a conflict in an .item file that TDS controls, and you and several team members have then sync’ed that file, you will have fun reversing out whatever you got left with.”
    – TDS has a Sitecore Item Merge tool that prevents this. It sounds like you were still merging the .item file through a regular merge tool (like TortoiseMerge, or kdiff)… in which case, this problem can occur due to Sitecore’s .item format.
    Setup of the Sitecore Item Merge tool is easy though. You can check it out here. https://www.youtube.com/watch?v=t-NeswJ9Vw8&list=PLb9QmtmxCbhl1uzG_XJ37Sc9fhsJNLfaM&index=22

    Again, great posts, and I understand the focus is on Unicorn in this CI setup….but I just wanted to clarify some things. As always, if you have any feedback on TDS, I’d love to hear it.

    Cheers.

    • Hi Sean

      As I was trying to say in the post, I know I was never using the full functionality of TDS. It has been, at time of writing this, more than 6 months since I last used TDS so I cannot remember all the screens, terms so bear with me I will probably use the wrong terms, but I hope that what I say will make sense.

      When I was first exposed to TDS, it was only being used to perform item synchronization. Everything else was being performed outside of the TDS environment. nant for deployments, and courier to generate packages to deploy the solution through the deployment process. All file merges where being performed when doing git fetch, pull etc. So if git reports a merge conflict, it tried to automatically resolve the issue, or you were prompted to manually merge the file to resolve the conflict. As mentioned in my post it was at this step that things when wrong. I never know that TDS has an Item Merge, and I do not know if anybody else knew about that feature. To get around this, we had to put in place a rule that says if git reports a merge conflict on an .item file, we had to always accept the remote file, sync with TDS, and push the merged file. This was the process in place, and as you said TDS has tools to avoid this pain, but no-one knew about it. This process was just cumbersome, especially on a mutli-dev team project.

      I was the demo at the Sitecore User Group in London in March 2016 where Nikola said that the new version of TDS to be released the next day will have Auto Sync.

      I never liked Sitecore Rocks right from when I used it the first time during my Sitecore Dev training. There will be people who love using Rocks, but I always had trouble with it. It used to regularly crash on my machine. I suspect that it didn’t play well with Resharper, and between Rocks and Resharper, Resharper wins every time. In addition, Rocks always seemed to be behind the latest versions of Sitecore, and could not do a lot if the functionality that you could do directly through the Content Editor. The last time I tried to use it was with the newly release Sitecore 8. Just would not work.

      Therefore again I never know about the functionality with Rocks and TDS.

      For setting up a new solution to be used with Sitecore, I never found a way to automate the TDS project creation, like I can with Unicorn. I can easily create a visual studio item template to create the config file. I did not find a way to automate creation of projects for TDS. I know that in the grand scheme of things the time taken to set up is negligible but it was still an annoyance.

      Thanks for reading.

      Darren

  2. Hi Darren, thanks for the reply.

    I completely understand. Some of the features of TDS, people aren’t aware of. And that’s all my comment was trying to do…alert any readers to the fact that these features exist, and they overcome a lot of the developer pain points that you mentioned.

    As for the project creation…. essentially a TDS project is just an xml file named scproj. I’m sure a Visual Studio project template could be created in a similar way your file template was created above. I haven’t tried this though, as I usually just use the TDS project wizard instead. It’s a basic wizard, and it even auto-fills the settings if you already have another existing TDS project in the solution, so I find it’s all I need….but each to their own.

    Again, great posts, and I don’t want my comments to take away from any of it. Was just hoping to let yourself and other readers know about the TDS features that overcome some of the pain points you mentioned.

    Cheers,
    Sean

  3. Sen says:

    One of the best written CI/CD article for Sitecore.
    Loved it!

    Thanks for the hard work!

  4. M J says:

    Hi Darren,

    Thanks again for these detailed steps. greatly appreciate your effort.

    Can you please help me understand the exact reason why you are moving Unicorn files out of the web root post deployment? is that good practice. is there a risk of those being over written?

    Just trying to finalise the deployment process and I am wondering if adding this step will actually add more time to my process thus evaluating the actual significance of this step.

    • Darren Guy says:

      I treat the unicorn files like anything in the data folder, which as you know on production systems is to not have in the root of the website. As long as you supply the full path to the root location of your unicorn files everything within Sitecore will run just fine.

      I have just deployed to production, and deploying the unicorn files on each server took 4 seconds. Remember you should not be putting content under source control, only templates, layouts. Because of this the amount of space required for unicorn is extremely low. 12 seconds out of 12 minutes it takes to update this environment is insignificant amount of time.

      • M J says:

        Hi again Darren,

        Thanks for your quick response. I have eventually set up this step as you advised in the guide. moving files from web root into data folder.

        However The bit with MicroChap.dll, i noticed you are manually dropping that onto your IIS server (prod/test) is there a way we can automate this step? as I am looking to get maximum automation out of this set up.

        Many thanks
        M J

  5. M J says:

    Hi Darren,

    Sorry Another Question, while running the unicorn sync step I keep getting the below error:

    Add-Type : Cannot bind argument to parameter ‘Path’ because it is null.

    At C:\Octopus\Work\20161006183001-228\Script.ps1:5 char:16

    + Add-Type -Path $microchap

    + ~~~~~~~~~~

    + CategoryInfo : InvalidData: (:) [Add-Type], ParentContainsErrorRecordException

    + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddTypeComm

    I have microchap.path defined as a global variable in the library of Octopus, I have also tried creating a variable specific to the project with no luck. the DLL file is verified located on the IIS server where tentacle is installed with the location matching that defined in the variable.

    What am I missing? I am not very familiar with debugging PSE script so any pointers on this front would be greatly appreciated.

    Many thanks

    • M J says:

      Hi Darren,

      The last issue has been resolved. The problem was I hadn’t bound the variable set under which the Microchap path was defined to the project in Octopus.

    • Darren Guy says:

      I thought about trying to automate everything, but I decided that there where somethings that where just not worth it. e.g. the MicroChap dll. However if you really do want to automate everything, then create a new NuGet package that contains all the tools / utilities and then create a process in Octopus to deploy that NuGet file. Just remember to update all variable paths where relevant

  6. Phil says:

    Hi Darren,
    Great series. A little out of scope but an octopus/sitecore related Q if I may.

    I can manually run a Unicorn sync via the unicorn.aspx page in a browser, however on executing with a PS script in Octopus (using Octopus Sync from the Octopus library) I receive errors as it seems to be expecting my site to be located under wwwroot (which it isnt). I see nothing in the PS script to cause this, so from the sitecore/config side of things, do you know of any config that would be causing it to look in that location?

    Thank You

    • Darren Guy says:

      I would take a step back and check all your settings in Octopus first. I know that for some of the out of the box processes if you do not specify a path to use, it will default to the wwwroot folder. Check the task log in raw or verbose mode to verify everything

    • jawadsabra says:

      Hi Phil, To add to Darren’s comment, The ps step uses octopus variables to find the site and run the sync on it’s host-name. so make sure you provide the correct location to your site in Octopus variables.

  7. Phil says:

    Thanks Darren…it turns out it was using sit from a .debug.config file , despite it being a release build and having a .config in the same folder already 🙂

  8. Thanks Darren, great info.

    I have the same thoughts about TDS. Great features and like the Rocks integration, but seems to always cause blocks and time wasting in the team. Every time I see team members moving back and forth all of a sudden. Most of the time it’s a TDS syncing problem. So we’re in the process of moving to unicorn now. Hopefully it stops the time wasting, but I’m sure there are issues with Unicorn that will come out in the wash.

    • Darren Guy says:

      Hi Anthony

      Oh yes, the biggest thing I would like to see is for Unicorn to have the ability to have excluded folders when transparent sync is enabled, but so far I have found ways around it.

    • Jawad says:

      As long as you implement Unicorn properly from step one and spread out the configurations in the best way that suits your project keeping in mind not to have overlapping configs (as in common content includes) you shouldn’t have any issues. Personally I haven’t found a compelling need to enable transparent sync however Darren raises a good point as this feature can be very useful if it supports excludes.

  9. peninsulavoice says:

    Great article! There’s something I’m wrestling with though. Maybe I’m misunderstanding, but what if you just want to sync Sitecore items? Doesn’t putting the Unicorn files in the build process mean that you have to build the entire project every time you want to sync items? Is there any way to separate item syncing from the build?

    • Darren Guy says:

      Since I wrote this article, I have been updating and refining my entire process.

      One of the first things I did was to create another nuspec file. this one to only contain the Unicorn yaml files. I then introduced a new build step into TeamCity to generate the nuget file and push to Octopus. Minimal changes where required to the process that deploys to the CM site to update Siecore from the new nuget file.

      Because it is now separate, if required, then can manually deploy the Unicorn files.

  10. Chris Benton says:

    TDS have finally(!) released Lightning Sync

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 12 other followers

%d bloggers like this: