Code snippet – Reestablishing the backup log chain with PowerShell

We have a certain customer that has an incredible fondness for undermining the restorability of their databases. Almost all of their databases are in the full recovery model and all-too-frequently the customer will start backing up transaction logs with the dreaded NO_LOG option, which destroys the transaction log backup chain:

Msg 4214, Level 16, State 1, Line 1
BACKUP LOG cannot be performed because there is no current database backup.
Msg 3013, Level 16, State 1, Line 1
BACKUP LOG is terminating abnormally.

In a future blog post, I will go into why users resort to this bad practice and what they should be doing instead.

This solution is based on a great blog post from Kalen Delaney (blog, Twitter), that showed how the [sys].[database_recovery_status] catalog view can reveal databases suffering from this problem, using a query like so:

SELECT DB_Name([database_id]) AS [DatabaseName]
FROM [sys].[database_recovery_status]
WHERE (
    ([last_log_backup_lsn] IS NULL)
    AND
    (Cast(DatabasePropertyEx(DB_Name([database_id]), 'Recovery') AS NVarChar(16)) = N'FULL')
    AND
    (Cast(DatabasePropertyEx(DB_Name([database_id]), 'Status') AS NVarChar(16)) = N'ONLINE')
)
ORDER BY [DatabaseName] ASC;

The following PowerShell function uses the query to retrieve a list of problematic databases and then automatically backs up each one:

Function Repair-LogBackupChain (
    [String]$ServerName = $(Throw (New-Object -TypeName 'System.ArgumentNullException' -ArgumentList '$ServerName')),
    [String]$BackupDirectoryPath = $(Throw (New-Object -TypeName 'System.ArgumentNullException' -ArgumentList '$BackupDirectoryPath')),
    [String]$OutputDirectoryPath = $(Throw (New-Object -TypeName 'System.ArgumentNullException' -ArgumentList '$OutputDirectoryPath'))
) {
    [Int]$BackedUpDatabaseCount = 0;
    [String]$BackupFilePath = '';
    [String]$DatabaseName = '';
    [String]$ErrorMessage = '';
    [String]$OutputFilePath = '';
    [String]$Query = "SET NOCOUNT ON; SELECT DB_Name([database_id]) AS [DatabaseName] FROM [sys].[database_recovery_status] WHERE (([last_log_backup_lsn] IS NULL) AND (Cast(DatabasePropertyEx(DB_Name([database_id]), 'Recovery') AS NVarChar(16)) = N'FULL') AND (Cast(DatabasePropertyEx(DB_Name([database_id]), 'Status') AS NVarChar(16)) = N'ONLINE')) ORDER BY [DatabaseName] ASC;";
    [String]$TSQLCode = '';

    #   Create the necessary directories, if they don't already exist.
    $BackupDirectoryPath = Add-Directory -Path $BackupDirectoryPath;
    $OutputDirectoryPath = Add-Directory -Path $OutputDirectoryPath;

    #   Lookup the names of the databases, in the full recovery model, that
    #   lack an intact transaction log backup chain.
    [String[]]$DatabaseNames = @(& 'SQLCmd.exe' -E -S $ServerName -d 'tempdb' -Q $Query -b -h-1 -W);
    $DatabaseNames | Write-Debug;

    #   Verify that the list of databases was successfully retrieved.
    If ((Get-Item -Path Variable:'LastExitCode').Value -eq 0) {
        #   Iterate through the databases that need backups.
        $DatabaseNames | ForEach-Object {
            $DatabaseName = $_;
            Write-Host "Backing up the [$($DatabaseName)] database..." -ForegroundColor 'Cyan';

            $OutputFilePath = "$($OutputDirectoryPath)$(Get-EscapedFileNameComponent -FileNameComponent $ServerName),Backup database,$(Get-EscapedFileNameComponent -FileNameComponent $DatabaseName),$([DateTime]::Now.ToString('yyyyMMddTHHmmssfff')).out";
            Write-Debug "`$OutputFilePath = `"$($OutputFilePath)`"";

            #   Construct the name of the backup file.
            $BackupFilePath = [String]::Format(
                '{0}{1}_{2}_{3}.bak',
                $BackupDirectoryPath,
                (Get-EscapedFileNameComponent -FileNameComponent $ServerName),
                (Get-EscapedFileNameComponent -FileNameComponent $DatabaseName),
                ([DateTime]::Now.ToString('yyyyMMddTHHmmssfff'))
            );

            #   Generate the T-SQL for backing up the database.
            $TSQLCode = [String]::Format(
                "BACKUP DATABASE [{0}] TO DISK = N'{1}';",
                $DatabaseName,
                $BackupFilePath
            );
            Write-Debug "`$TSQLCode = `"$($TSQLCode)`"";

            #   Backup the database.
            & 'SQLCmd.exe' -E -S $ServerName -d 'tempdb' -Q $TSQLCode -o $OutputFilePath -b;

            If (Test-Path -LiteralPath $BackupFilePath -PathType 'Leaf') {
                $BackedUpDatabaseCount++;
            }
            Else {
                $ErrorMessage = "There was a problem backing up the [$($DatabaseName)] database.";
                Throw (New-Object -TypeName 'System.InvalidOperationException' -ArgumentList $ErrorMessage);
            }
        }
    }
    Else {
        $ErrorMessage = [String]::Format(
            "There was a problem querying the [sys].[database_recovery_status] catalog view:`r`n{0}",
            (
                [String]::Join(
                    "`r`n",
                    $DatabaseNames
                )
            )
        );
        Throw (New-Object -TypeName 'System.InvalidOperationException' -ArgumentList $ErrorMessage);
    }

    Write-Host "A total of $($BackedUpDatabaseCount) database(s) was/were backed up." -ForegroundColor 'Green';

    Return;
}

Here’s an example of using the function:

Repair-LogBackupChain -ServerName 'DatabaseServer01\InstanceA' -BackupDirectoryPath 'C:\Temp\' -OutputDirectoryPath 'C:\Temp\';

Note: The code snippet is dependent on two functions that are included in the download for my PowerShell cookbook presentation, Add-Directory and Get-EscapedFileNameComponent.

Maybe by the time that I write the follow-up blog post, I will have received permission to break the customer of this nasty habit (using a tip from Paul Randal (blog, Twitter)).

Hopefully this PowerShell snippet will save you as much time as it has for me!

Posted in PowerShell, SQL Server | Tagged , , , , , , , , , | Leave a comment

SQL Trace presentation materials posted

I just uploaded a new set of deliverables from my SQL Trace presentation (which can be found here). Let me know if you have any questions.

Posted in SQL Server | Tagged , , , , | Leave a comment

Script – Extracting XML deadlock graphs from a trace file

Today is my fourth day of SQLskills Immersion Events training, taught by none other than Paul Randal (blog, Twitter) and Kimberly Tripp (blog, Twitter). The material and delivery have been amazing and I hope to have a chance to blog about it this weekend. A fellow student, Eric Humphrey (blog, Twitter), asked about automatically extracting XML deadlock graphs from a SQL Trace file.

I’m a big proponent of using server-side traces for troubleshooting deadlocks. During my deadlocks presentation, I show how to manually extract XML deadlock graphs using SQL Server Profiler:

That works, but it’s a manual process, which is anathema to us automation adherents.

When processing XML deadlock graphs, I tend to:

  1. query the trace data, via the [sys].[fn_trace_gettable] system function,
  2. cast the [TextData] value to the XML data type, then
  3. analyze the deadlocks with XQuery.

I can see the benefit of having a few loose *.XDL files, so I spent a little time writing a PowerShell script that will automatically extract the XML deadlock graphs and save them as files. This zip archive contains the script and sample trace files.

Here is an example of using the script (assuming that you extracted the files to the “C:\Temp\” directory):

[String]$ScriptUtilityPath = 'C:\Temp\Export-XDLFromTraceFile.ps1';
[String]$OutputDirectoryPath = 'C:\Temp\';
[String]$SourceServerName = 'T-SQL001';
[String]$TraceFilePath = 'C:\Temp\DeadlockTrace_20100522T113514.trc';
& $ScriptUtilityPath -TraceFilePath $TraceFilePath -OutputDirectoryPath $OutputDirectoryPath -SourceServerName $SourceServerName -FormatXML | Out-Null;

Here’s another example that leverages PowerShell’s composable nature:

[String]$ScriptUtilityPath = 'C:\Temp\Export-XDLFromTraceFile.ps1';
[String]$OutputDirectoryPath = 'C:\Temp\';
[System.IO.FileInfo[]]$OutputFiles = @();
[String]$SourceDirectoryPath = 'C:\Temp\';
[String]$SourceServerName = 'T-SQL001';
[String]$TraceFilePath = '';
Get-ChildItem -LiteralPath $SourceDirectoryPath -Filter '*.trc' | ForEach-Object {
    $TraceFilePath = $_.FullName;
    $OutputFiles += @(& $ScriptUtilityPath -TraceFilePath $TraceFilePath -OutputDirectoryPath $OutputDirectoryPath -SourceServerName $SourceServerName -FormatXML);
}
$OutputFiles | Select-Object -Property Name, @{Name='Size_bytes'; Expression={'{0:N0}' -f $_.Length}} | Format-Table -AutoSize;

Let me know if this is useful to you, or if you have any suggestions to make it better.

NOTE: I haven’t had a chance to test the script with the SQL Server 2005 SQL Server Management Objects (SMO) assemblies. I suspect one or both of the following issues:

Posted in PowerShell, SQL Server | Tagged , , , , , , , , , , | Leave a comment

Updated presentation materials posted

I just uploaded a new set of deliverables from my Deadlocks presentation (which can be found here). The new archive just contains tweaks that were made last week at the Summit. I'm working on a few new scripts that illustrate topics raised during the presentation. I'll let everyone know when they're ready.

Posted in Uncategorized | Tagged | Leave a comment

Emerging from self-imposed exile

While I was never prolific with blogging or tweeting, I have definitely been out-of-touch for over a year. Here's why…

At last year's North American PASS Summit, I was still developing the content for my presentation at the last minute, nearly pulling an all-nighter. The code being demonstrated was barely sufficient for the talk and not nearly robust enough for release, especially to people just introduced to the technology. After delivering my presentation, I was so exhausted and sleep deprived that I decided to enjoy the conference and deferred posting my code until returning home. Unfortunately, life never slowed down and I had trouble finishing and uploading the scripts. My progress was also hindered by feature/scope creep as well as my tendency toward perfectionism.

Since getting into technical speaking, one of my aspirations has been to provide comprehensive and valuable presentation materials (slides, handouts, files, and code). I want my audience members to be able to immediately put their new knowledge to use. In a classic case of perverse unintended consequences, I was achieving the opposite. Instead of being known for providing great presentation materials, I've probably earned notoriety for empty promises to post the materials.

With the start of the new year, I resolved to focus on the project. I withdrew from many activities with the following rationale:

  • It seemed like I should be using any free time to finish working on my presentation materials
  • I didn't want to face the people that I was letting down
  • The pressure that I was placing on myself was leading to burnout

Here are some of the activities that I've consciously limited, or opportunities that I've postponed:

  • I started missing most of the local user group meetings
  • I dropped off of Twitter… the last time that I recall opening a Twitter client was on 2010-01-14
  • I've been invited to write several articles
  • I was asked to present to PASS Virtual Chapter meetings
  • I stopped penning blog posts as well as keeping current on other people's blogs

The really sad thing is that keeping my nose to the grindstone didn't help me to produce the deliverables—it just left me isolated and demoralized!

I realized that my problems were due to lack of effective process. I will definitely write more on this topic, but I adopted new tools and methodologies that include:

Perhaps most importantly, I finally implemented Getting Things Done. I will provide more details in future blog posts.

Deliverables for my presentations were posted for the first time on October 27th. Yesterday's presentation represented a huge milestone for me: it was the first time that materials were uploaded when I got up to speak. I have some modifications already, but that's the point of the incremental servicing model. Waiting until everything was ready led to nothing ever getting released.

It's time to rejoin the SQL Server community and I can think of no better place to do it than the PASS Summit. If I have failed to respond to you in the past, I apologize. I'm working through the backlog of blog comments and email messages. However, feel free to contact me again. Going forward, I am going to make sure to be responsive.

By the way, Michael Swart has implicitly forgiven me, so I feel better already!

Posted in SQL Server | Tagged , , , | 5 Comments

Today’s presentation materials have been posted

The materials for today’s presentation, Deadlock Detection, Troubleshooting, and Prevention, at the 2010 North American PASS Summit, have been posted. See the Deadlock Detection, Troubleshooting, and Prevention page for more information.

The download contains the slide deck, a handout full of links to resources, demonstration code, and a few sample files. I plan to upload some revisions as soon as I get home from the Summit (this weekend).

See you in an hour!

Posted in SQL Server | Tagged , , , | 1 Comment

Temporary post – Presentation materials

I will post everything in their permanent homes later today… but for now, here are the materials from my last few presentations:

Posted in Uncategorized | Leave a comment