Have you ever been in a server room that had you scratching your head, and wondering, “What were they thinking?”  The images certainly don’t give you any warm fuzzies, do they?

Several years ago, I was part of a group that was invited to Houston to be the entertainment at a Christmas party.  I was put up for the night at a local hotel, and in the morning arose early enough to visit the breakfast bar before being picked back up.  I loaded a plate of food, and set it at an empty table in the small area, then decided I wanted a bowl of oatmeal.  I returned to the breakfast bar and prepared the oatmeal.  When I turned back to my seat, there was a gentleman seated at my table helping himself to my plate of food.  I think I paused for a moment, dumbfounded at the sight, and then was wondering if I should approach him and ask if there was anything else I could get him, some more eggs, another biscuit, maybe a cup of coffee.  At that moment, a woman stopped and said to him, “Harold, that’s not your food, our table is over here!” as she carried his plate of food to their table.  She turned to me and apologized and told me she would get me another plate of food.  The point is, in IT, be prepared for the unexpected.

If you are drinking the Microsoft Kool-Aid, and are using the built-in feature, Windows Server Backup in Server 2008 and up, you might be vulnerable to experiencing the unexpected.  For some unknown reason, Gates and company neglected to include a reporting and alerting feature in the backup utility.  Therefore, you can set your backup schedule flawlessly, and have backups running successfully night after night.  But when the unexpected happens, and your backup fails for one reason or another, you won’t be alerted.  Oh sure, the program writes an event to its own location in the Event Log, but you won’t get any notification.  You have to manually check the logs.  Fine, if you have a server or two, but what if you have a large number of servers to keep track of?  You configured the backup, it has been running OK, so ignorance is bliss, what you don’t know can’t hurt you, right?  The servers are humming along just fine.  It’s an optical illusion, Jack!  Einstein said, “Our separation from each other is an optical illusion of consciousness”.  He was speaking about how we, as human beings, relate to each other.  As long as we remain separated from each other even in the smallest of forms, we remain unaware of how it affects our whole environment.  But the more conscious and self-aware we are, the less illusion there is.  However, it’s also a good metaphor for our backup situation.

Recently from Rattan - Subscribe to get insight direct to your inbox.

 No Time to Handle Backups? We Can Help!

Let our professional security and infrastructure team help with Backups, a Disaster Recovery Plan, a Business Continuity Plan, and/or a Process Management Plan.  Your team isn’t pulled away from their daily tasks, you avoid the frustration of not being prepared when an event strikes, and you potentially save your business money.  Start the conversation by completing the form below or give us a call at 405.810.8005.

    This is the way Microsoft programmed it, we don’t have to concern ourselves with the details, it will just work, huh?  Just because backups are failing, doesn’t mean the servers won’t necessarily continue to function.  They’re kinda like clogged arteries; you don’t realize anything is wrong on the surface, but one day you wake up and bam!  Actually, if you survive, it will have been quite a wake-up call.  We can remain separated or ignorant of the truth, and it makes us feel protected and secure.  We get a false sense of security and drift off to la la land.  At a critical moment, when we realize we don’t have a good backup because it failed, and we had the illusion everything was running fine, the blood pressure will start to rise – and I’m not talking about just our own BP here.

    We need to consistently know the backup’s status.  We need Einstein’s openness, even if it makes us feel exposed, vulnerable, and uncomfortable.  Because when we are aware, we can examine the situation and do something to change it.  That being said, there is a way to get alerts emailed to us reporting the backup status.  We know Windows Server Backup does not alert on the status of backups.  However, it does create log entries in the Event Log at Logs > Applications and Services Logs > Microsoft > Windows > Backup > Operational.  We can create a script to report on the status and send an email using Powershell’s Send-Mail Message cmdlet.  Then we create a scheduled task to trigger on an event, which will run our script.  Here are the details:

    Note:  These instructions are for Server 2008 SP1 and up that have met the pre-requisites listed at the top of this document.  Server 2003 does not support Windows Management Framework 3.0 and Powershell 3.0.  There may be alternate scripts available using legacy emailing cmdlets for Server 2003.

    1. Check prerequisites on the server to be backed up:
      1. Make sure the Windows Server Backup feature is installed.
      2. Verify the server has at least Windows Management Framework 3.0.
      3. Verify the server has at least Powershell version 3.0.
    2. Create backup job(s) in Windows Server Backup and verify functionality.
    3. On the server being backed up, create a Powershell script to send an email alert using the Send-MailMessage command. Alternatively, you can copy a good, working script to your server from blog.jocha.se, Windows Backup Mail Report Script, and edit it to create and send your reports.  The script contains instruction for editing and for creating the task to trigger the email.  The screenshot below contains the portion of the WBJobReport.ps1 script that you will need to edit:      
    4. Remove the comments from lines 25 & 26.
    5. On line 29, replace My Company with your company’s name.
    6. On line 32, replace it-admin@company.com with your recipient email address.
    7. On line 33, insert your no-reply address if you have one. I did not, so I used the email address for the account we have set up for performing backups.
    8. On line 36, replace MyUser with the name of the user for the account sending the email alerts.
    9. On line 37, replace MyPassword with the password for the sender’s account.
    10. On line 40, replace smtpserver.company.com with your email server. I used the local server name as we are sending the emails locally on the Exchange server.  Use your public url if you are sending remotely.
    11. On line 43, replace the port number if necessary for the port your mail server uses.
    12. On line 46, change true to false if you don’t use SSL.
    13. Remove the comment from line 207.

    Now that the script is ready, create the task to trigger the script to run, and create and send the report as instructed within the script.

    1. Open the task scheduler (run as administrator) on the server, and click on Create Task in the Action pane. 
    2. On the General tab, configure the following (Make sure the user account assigned to run the task has the permissions to create and run a task, and run an admin Powershell session):
    3. On the Triggers tab, select New, and configure as in the 2 screenshots blow, selecting the New Event Filter button to create a new filter, click OK twice to create the filter: 
    4. On the Actions tab, select New, and configure as in the screenshot below, then click OK: 
    5. On the Settings tab, verify ‘Allow task to be run on demand’ is selected, and click OK.  
    6. You will need to enter the password for the admin user who is assigned to run the task. This user will need ‘Log on as a batch job’ rights.
    7. On a DC, the script failed to run because New-PSSession could not connect to the destination specified in the request.
      1. Running winrm get winrm/config on the server and on a working server revealed the AllowRemoteAccess value as false on the DC, but true on the working server.
      2. Running the command to enable it only produced an output revealing a different command to run.
      3. The following command sets the value to true: winrm invoke EnableRemoting http://schemas.microsoft.com/wbem/wsman/1/config/service
    8. To enable the server to email through the Exchange server the self-signed certificate must be added to the trusted store.

    Microsoft left a glaring omission in the Windows Server Backup feature, with no built-in way to configure alerting on backup status.  I have a scene from a movie running through my mind of an old codger giving directions to get back to the main highway.  He traces a few routes in the air, then stops, shakes his head, and says, “Nope, you can’t get there from here.”  Well, with Windows Server Backup, now you can get there from here.  We can get the reporting we need to ensure we have good backups, and take action when they are not.  If you don’t set up proper reporting, and are left with a crashed server, corrupted data, and no good backup, pick a safe way to take out your frustration.  Word of advice, avoid self-inflicted injury, don’t forget there is a big difference between drywall and a stud!

    About The Author

    Donny Hilbern is a network and systems consultant specializing in analyzing, designing, and implementing network and enterprise systems.  Donny has been working in the IT field for over 25 years, with nearly 20 years of that time invested in network and system administration and infrastructure technology.  He has experienced a number of undocumented or lightly documented issues during that time.  His desire is to leverage that experience in sharing about some of those issues and how they were resolved to make IT work for his clients.