Karine Bosch’s Blog

On SharePoint

Repair content type retention policies


Recently I was at a customer who implemented retention policies on content types. They have about 20 content types and a site collection with hundreds of sub sites with thousands of documents.

Becky Bertram wrote a nice detailed article on how to define retention policies for SharePoint 2010.

Problem description

When retention policies are applied on content types, there are 2 timer jobs that run (by default during the weekend):

  • Information management policy timer job: by default, runs on friday 11 PM. The job goes through libraries that have policies applied. It calculates the expiration date for every item.
  • Expiration policy timer job: by default, runs on saturday 11 PM. This job executes the action part of the retention policy. For example, if the action is to move expired documents to the recycle bin, expired documents will be deleted; if the action is set to move the documents to a send-to location, the expired documents will be moved.

My customer explained that when the  timer jobs ran for the first time, they ran for several hours and then just stopped running. As of then the jobs ran weekly, but with a lot of similar errors in the ULS logs:

"Error processing expiration of content in list <list name> in site <url to sp site>. 
 Error: Invalid field name. {b0227f1a-b179-4d45-855b-a18f03706bcb}".
"Error processing expiration of content in list <list name> in site <url to sp site>. 
 Error: Invalid field name. {acd16fdf-052f-40f7-bb7e-564c269c9fbc}".

From this post you can see that these guids refer to out of the box SharePoint fields:

Exempt from Policy b0227f1a-b179-4d45-855b-a18f03706bcb _dlc_Exempt
Expiration Date acd16fdf-052f-40f7-bb7e-564c269c9fbc _dlc_ExpireDate

When I tried to take a look at the Compliance Details of a document, I got the following error message: “column  ‘_dlc_exempt’ does not exist. It may have been deleted by another user”.

The Compliance Details menu option is only available on the context menu when retention policies are active for the specific document:

Compliance Details

After investigation I found out that part of the sites had the retention policies correctly applied, and part of the sites had not.

In the rest of the article I will use the term “active retention policy”. It means that retention policies are defined on content types, and that these content types are in use on a document library.

Solution

I found a very interesting article on retention policies on the net to get me started, so kudos to Mike Berryman. I started my investigation to repair the site collection based on this article.

When a SharePoint web has active retention policies, it should have the following properties on its property bag:

  • allowslistpolicy
  • dlc_sitehaspolicy
  • dlc_sitehasexpirationpolicy
  • dlc_webhasexpirationpolicy

As some SharePoint sites were working correctly and some not, I corrected each SPWeb as follows:

$web.AllowUnsafeUpdates = $true
if (!($web.Properties.ContainsKey("allowslistpolicy")))
{
    $web.Properties.Add("allowslistpolicy", $true)
}
if (!($web.Properties.ContainsKey("dlc_sitehasexpirationpolicy")))
{
    $web.Properties.Add("dlc_sitehasexpirationpolicy", $true)
}
if (!($web.Properties.ContainsKey("dlc_sitehaspolicy")))
{
    $web.Properties.Add("dlc_sitehaspolicy", $true)
}
if (!($web.Properties.ContainsKey("dlc_webhasexpirationpolicy")))
{
    $web.Properties.Add("dlc_webhasexpirationpolicy", $true)
}
$web.Properties.Update()
$web.Update()

A library with active retention policies should have the following hidden fields:

  • _dlc_Exempt
  • _dlc_ExpireDateSaved
  • _dlc_ExpireDate

I used the XML definition of these fields in order to create the missing fields:

$displayName_exempt = "Exempt from Policy"
$schemaXml_exempt = "<Field ID='{B0227F1A-B179-4D45-855B-A18F03706BCB}' 
    Name='_dlc_Exempt' StaticName='_dlc_Exempt' DisplayName='_dlc_Exempt' 
    SourceID='http://schemas.microsoft.com/sharepoint/v3' Group='Document and Record Management Columns' 
    Type='ExemptField' Indexed='FALSE' Hidden='TRUE' CanToggleHidden='TRUE' 
    ShowInNewForm='FALSE' ShowInEditForm='FALSE' ShowInFileDlg='FALSE' ShowInDisplayForm='FALSE' Required='FALSE' 
    Sealed='TRUE' ReadOnly='TRUE' OverwriteInChildScopes='TRUE'/>"
 
$displayName_expireDateSaved = "Original Expiration Date"
$schemaXml_expireDateSaved = "<Field ID='{74E6AE8A-0E3E-4DCB-BBFF-B5A016D74D64}' 
    Name='_dlc_ExpireDateSaved' StaticName='_dlc_ExpireDateSaved' DisplayName='_dlc_ExpireDateSaved' 
    SourceID='http://schemas.microsoft.com/sharepoint/v3' Group='Document and Record Management Columns' 
    Type='DateTime' Indexed='FALSE' Hidden='TRUE' CanToggleHidden='TRUE' ShowInNewForm='FALSE' 
    ShowInEditForm='FALSE' ShowInFileDlg='FALSE' ShowInDisplayForm='FALSE' Required='FALSE' 
    Sealed='TRUE' ReadOnly='TRUE' OverwriteInChildScopes='TRUE' />"

$displayName_expireDate = "Expiration Date"
$schemaXml_expireDate = "<Field ID='{ACD16FDF-052F-40F7-BB7E-564C269C9FBC}' 
   Name='_dlc_ExpireDate' StaticName='_dlc_ExpireDate' DisplayName='_dlc_ExpireDate' 
   SourceID='http://schemas.microsoft.com/sharepoint/v3' Group='Document and Record Management Columns' 
   Type='DateTime' Indexed='TRUE' Hidden='TRUE' CanToggleHidden='TRUE' 
   ShowInNewForm='FALSE' ShowInEditForm='FALSE' ShowInFileDlg='FALSE' ShowInDisplayForm='FALSE' Required='FALSE' 
   Sealed='TRUE' ReadOnly='TRUE' OverwriteInChildScopes='TRUE' />"

As some SharePoint libraries were working correctly and some not, I corrected each SPList as follows:

function RepairField($lib, $fieldInternalName $fieldDisplayName, 
       $schemaXml) { 
   $field = $null
   try
   {
      $field = $lib.Fields.GetFieldByInternalName($fieldInternalName)
   }
   catch {}
            
   if ($field -eq $null)
   {
       $lib.Fields.AddFieldAsXml($schemaXml)
       $field = $lib.Fields[$fieldInternalName]
       if ($field -ne $null)
       {
            $field.Title = $fieldDisplayName
            $field.Update()
       }
   }
}       
       
RepairField($lib, "_dlc_exempt", 
    $displayName_exempt, 
    $schemaXml_exempt)
RepairField($lib, "_dlc_ExpireDateSaved", 
    $displayName_expireDateSaved, 
    $schemaXml_expireDateSaved)
RepairField($lib, "_dlc_ExpireDate", 
    $displayName_expireDate, 
    $schemaXml_expireDate)
$lib.Update()

Each document on which a retention policy applies, has the following properties in its property bag:

  • ItemRetentionFormula
  • _dlc_ItemStageId
  • _dlc_ItemScheduleId

I corrected the items as follows:

$files = $lib.RootFolder.Files
foreach ($file in $files)
{
    $item = $file.Item
            
    # add the property ItemRetentionFormula to the property bag
       write-host ("set the retention properties  for item ID " + $item["ID"])
    $item["Modified"] = $item["Created"]

    if ($item.Properties.ContainsKey("ItemRetentionFormula") -eq $false)
    {
        $item.Properties.Add("ItemRetentionFormula", $true)            
    }
    if ($item.Properties.ContainsKey("_dlc_ItemStageId") -eq $false)
    {
        $item.Properties.Add("_dlc_ItemStageId", $true)
    }
    if ($item.Properties.ContainsKey("_dlc_ItemScheduleId") -eq $false)
    {
        $item.Properties.Add("_dlc_ItemScheduleId", $true)
    }
                    
    # clear the item stage id
    $item.Properties["_dlc_ItemStageId"] = ""
    # set the schedule type
    $item.Properties["_dlc_ItemScheduleId"] = $null
    # set the property to contain the formula
    $item.Properties["ItemRetentionFormula"] = $null
    $file.Update()  
    $item.SystemUpdate()
}

For completeness, at the end I disposed the SPWeb object:

$web.AllowUnsafeUpdates = $false
$web.Dispose()

Remarks:

  • I used the server-side object model.
  • As I had to work on the production environment, I had to write the code in PowerShell, but you can also do it in C#.

 

 

 

 

January 4, 2016 - Posted by | SharePoint 2010

6 Comments »

  1. Hi Karine

    Thank you so much for the post!

    In updating the library, I’m getting an error: Method invocation failed because [Microsoft.SharePoint.SPField] does not contain a method named ‘AddFieldAsXml’.

    Thoughts?

    Thank you for your time!

    Jeanne

    Comment by Jeanne | May 13, 2016 | Reply

    • Hi Jeanne,
      That’s not normal, that method is there both in server side object model as in CSOM. Can you post your code?
      Karine

      Comment by Karine Bosch | May 14, 2016 | Reply

  2. $web = get-spweb “https://mywebapplication/sites/sitecollection/subsite

    $web.AllowUnsafeUpdates = $true

    if (!($web.Properties.ContainsKey(“allowslistpolicy”))) {$web.Properties.Add(“allowslistpolicy”, $true)}
    if (!($web.Properties.ContainsKey(“dlc_sitehasexpirationpolicy”))){$web.Properties.Add(“dlc_sitehasexpirationpolicy”, $true)}
    if (!($web.Properties.ContainsKey(“dlc_sitehaspolicy”))){$web.Properties.Add(“dlc_sitehaspolicy”, $true)}
    if (!($web.Properties.ContainsKey(“dlc_webhasexpirationpolicy”))){$web.Properties.Add(“dlc_webhasexpirationpolicy”, $true)}

    $web.Properties.Update()

    $web.Update()

    # _dlc_Exempt was present in the hidden fields of the particular library so I excluded it from the xml and used:

    $list = $web.lists[“Document Library Name”]

    $displayName_expireDateSaved = “Original Expiration Date”
    $schemaXml_expireDateSaved = “”

    $displayName_expireDate = “Expiration Date”
    $schemaXml_expireDate = “”

    function RepairField($lib, $fieldInternalName $fieldDisplayName,
    $schemaXml) {
    $field = $null
    try
    {
    $field = $lib.Fields.GetFieldByInternalName($fieldInternalName)
    }
    catch {}

    if ($field -eq $null)
    {
    $lib.Fields.AddFieldAsXml($schemaXml)
    $field = $lib.Fields[$fieldInternalName]
    if ($field -ne $null)
    {
    $field.Title = $fieldDisplayName
    $field.Update()
    }
    }
    }

    RepairField($list, “_dlc_ExpireDateSaved”, $displayName_expireDateSaved, $schemaXml_expireDateSaved)

    ———————————————————–
    thank you again for your time!

    Comment by Jeanne | May 16, 2016 | Reply

    • Hi Jeanne, I bet there is something wrong with your $lib variable. Can you check if this is really a list or library? Because I’m 100% sure the method AddFieldXml exists on SPFieldCollection, while your exception says that it does not exist on SPField.
      Kind regards,
      Karine Bosch

      Comment by Karine Bosch | May 19, 2016 | Reply

  3. After transfering our content db from our old 2010 to our new 2013 server, i find that i can not get information management retention policies turned on for anything. what’s odd is that on our test server, which should have been copied and built the same way we did the production servers… i can create info management policies.
    could there be some root level switch to throw to get this stuff working again? Or am I stuck trying to figure out how to write a powershell script like yours to at least fix the lists and document libraries where we know we need this working?

    Comment by Jay Christianson | October 24, 2016 | Reply

    • Hi Jay,
      The retention policy timer jobs should run every week (on Friday and Saturday). Can you check in Central Admin that they are running? If so, you should at least see an indication of what’s wrong in the ULS logs.
      Kind regards,
      Karine Bosch

      Comment by Karine Bosch | November 28, 2016 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: