Wednesday, September 27, 2017

SCOM Day Sweden 2017

Next week I 'll be on the road again and heading over to Gothenburg in Sweden to present at the awesome SCOM Day event.

Organised by the team at Approved Consulting, I presented at this event last year and really enjoyed the networking and talking to attendees about all things SCOM. This year, I'll be talking about what's new with SCOM (including some of my favourite community management packs) and I'll also be discussing some new features and changes that are coming to SCOM 2016 over the next few months.

I'll be looking forward to a presentation on the day from Microsoft's Kevin Holman (aka SCOM Ninja/Guru/Legend). Kevin is one of the most prolific SCOM bloggers around and there's always something new to learn from his blog posts and presentations.

If you're based in Scandinavia and want to attend the event (it's kicking off on Wednesday 4th October), then you can register using the link below:

Hope to see some of you there!

Tuesday, August 29, 2017

Available Now: Inside the Microsoft Operations Management Suite (V2)

These past few months have been pretty hectic for me and as a result, this blog had to take a back seat for a while. Along with the day job, presenting at conferences and some family vacation time, I've been busy working with the team on an updated release of the awesome 'Inside the Microsoft Operations Management Suite' book.

The original release of this book was an essential 'must-have' for anybody working with Microsoft's Operations Management Suite (OMS) and it was a constant reference for me over the last year and a half while I tried to get my head around this new technology. As is always the case with cloud technologies however, things change at a drastic pace and an updated release of the original book was duly needed.

Meet the Team

With so many new features added to OMS, this update was never going to be a quick and easy task. Thankfully, the full original authoring team of Stan, Tao, Pete and Anders signed up again for the new release (bio's below):

Along with the original authors, a new team of technical editors (myself included) were brought on-board to help get this over the line:

What's it about?

This updated release provides readers with an end-to-end deep dive into the full range of Microsoft Operations Management Suite (OMS) features and functionality, complete with downloadable sample scripts.

Here's a rundown of the chapters to give you a good idea of the type of content you'll learn about:

Chapter 1: Introduction and Onboarding 
Chapter 2: Searching and Presenting OMS Data 
Chapter 3: Process Automation 
Chapter 4: Configuration Management 
Chapter 5: Change & Update Management 
Chapter 6: Extending OMS Using Log Search 
Chapter 7: Alert Management 
Chapter 8: Log Management & Performance Data 
Chapter 9: Azure & Office 365 Solutions 
Chapter 10: Service Map & Wire Data 
Chapter 11: Network Performance Monitor 
Chapter 12: Other OMS Solutions  
Chapter 13: Assessment Solutions 
Chapter 14: Security & Compliance 
Chapter 15: Protection & Recovery 
Chapter 16: ITSM Integration 
Chapter 17: Custom OMS Solutions

How much will it cost?

Keeping with the community spirit behind the first book, this release will be COMPLETELY FREE for download in E-Book format with the Kindle and Paperback versions coming to Amazon a couple months later. Keep in mind that the Kindle and Paperback versions will carry a nominal price to help cover production costs.

Download Info

You can download the e-book version right now from the link below and the Kindle and Paperback versions will be available on Amazon over the coming months:

Hopefully you enjoy reading this book and working through all the demos as much as we did and if you've any comments/requests, please ping them to the email address here.

Monday, August 21, 2017

Experts Live Europe 2017

After a few weeks off work on vacation, it's back to full swing this week as I prepare to travel over to Berlin tomorrow for the awesome Experts Live Europe conference.

Formerly known as System Center Universe Europe, this is one of my favourite conferences to present, network and learn at. If it's anything like last years event, then the week ahead is going to be a blast.

I'll be presenting a session on Wednesday afternoon titled 'Monitoring...The Next Generation' where I'll be talking about some of the best new monitoring features available in the Microsoft space.

Later on Wednesday, I'll be back on stage with my buddy Stefan Roth for an 'Experts Discussion Panel' titled 'Monitoring, Insights and Analytics'. For this panel, we're encouraging attendees to bring their questions and engage in conversations - which will hopefully give people some real technical value to bring back to their workplace.

With close to 100 different sessions to choose from spanning topics such as Microsoft Azure, OMS, EMS, Azure Stack, System Center and Windows Server, there's something for everyone. Check out the session schedule here.

Looking forward to seeing everyone over there and make sure to drop in to one of my sessions and say hello!

Wednesday, May 24, 2017

SCOM 2016 Update Rollup 3 (UR3) Now Available

Yesterday, Microsoft released a new (and widely anticipated) Update Rollup (UR3) for SCOM 2016.

This update contains fifteen documented fixes with one in particular (APM crashing IIS agent) being the most important and a top priority for me and my customers due to the agent IIS crash issue I blogged about a while back.

***Update 6th June 2017: Microsoft has posted more information about this issue remaining after deploying UR3 and have mentioned a hotfix is still in the works. Check out their latest post on this issue here.***

Here's some of the highlights of fixes that are covered in this update:

  • The Application Performance Monitoring (APM) feature in System Center 2016 Operations Manager Agent causes a crash for the IIS Application Pool that's running under the .NET Framework 2.0 runtime. Microsoft Monitoring Agent should be updated on all servers that use .NET 2.0 application pools for APM binaries update to take effect. A restart of the server might be required if APM libraries were being used at the time of the update.
  • When overriding multiple properties on rules that are created by the Azure Management Pack, duplicate override names are created. This causes overrides to be lost.
  • When the heartbeat failure monitor is triggered, a "Computer Not Reachable" message is displayed even when the computer is not down.
  • The Get-SCOMOverrideResult PowerShell cmdlet doesn't return the correct list of effective overrides.
  • When creating a management pack (MP) on a client that contains a Service Level (SLA) dashboard and Service Level Objects (SLO), the localized names of objects aren't displayed properly if the client's CurrentCulture settings don't match the CurrentUICulture settings. In cases where the localized settings are English English, ENG, or Australian English, ENA, there's an issue when the objects are renamed.
  • The Event ID: 26373 error, which may cause high memory consumption and affect server performance, has been changed from a “Critical” message to an “Informational” message.
  • The UseMIAPI registry subkey prevents collection of processor performance data for RedHat Linux system. Also, custom performance collection rules are also impacted by the UseMIAPI setting.
  • Organizational Unit (OU) properties for Active Directory systems are not being discovered or populated.
  • The Microsoft.SystemCenter.Agent.RestartHealthService.HealthServicePerfCounterThreshold recovery task fails to restart the agent.
  • An execution policy has been added as unrestricted to PowerShell scripts in Inbox management packs.
  • SQL Agent jobs for maintenance schedule use the default database. If the database name is not the default, the job fails.

You can see the full list of fixes from the official UR3 knowledge base article here.

To get access to this update, you can choose to either manually download it from the Microsoft Update Catalog here or you can use Windows Update to pull down the update automatically to your SCOM 2016 environment.

**Note: I've yet to test this update rollup on the existing SCOM 2016 agents that I've previously applied the NOAPM=1 workaround to (mentioned in my post here) and I suspect that a push install of this UR from the console to those agents will fail as the APM binaries are no longer installed. I'll create a new post on updating those agents when I've tested the process fully.**

Whatever method you choose to deploy this update, make sure to read through the full installation instructions as there are some manual tasks to carry out once the update has been applied to each SCOM role and if you're not confident, I'd always recommend waiting for Microsoft's Kevin Holman to add his walk-through post for this UR to his blog here.

Finally, this update is one part of a larger UR3 release for covering other products in the System Center 2016 suite. If you've deployed additional components of the suite alongside SCOM, then you might be interested to check out the updates now available for DPM 2016, SCSM 2016 and SCVMM 2016.

Full details of all the fixes in the main System Center 2016 UR3 downloads can be viewed at:

Monday, April 24, 2017

Monitoring Commvault with SCOM

A common request I get from customers is how to best monitor Commvault backups using SCOM. Commvault are one of the market leaders in enterprise backup technologies and I come across their products in customer sites on a regular basis.

As I don’t have a spare Commvault server to play around with in my demo environment and I’ve never really had the time to document the whole process during an actual customer deployment, a blog post on this topic has remained elusive until now.

A few weeks back I was working on a customer site who needed Commvault monitored and over the course of a lunch break one day, I managed to put some screenshots together to help document the process.

Overall, it’s pretty straight-forward to get up and running and unlike some other enterprise backup vendors, Commvault have made an effort to integrate their product with SCOM. The integration is made possible by initiating the integration from the Commvault CommCell Browser console – which then imports an unsealed management pack into SCOM for monitoring.

The management pack provided by Commvault is basic enough though and you’ll probably want to add some custom monitors and views to it as you see fit.

Management Pack Overview

The unsealed management pack provided contains a discovery rule which targets the Windows Computer class. This discovery rule (shown in the exported Excel sheet below) looks for the presence of the 'Commvault Server Event Manager Service' (the actual service name is GxEvMgrS).

When this service is detected, a new class named 'Commvault CommServer' is then created by the management pack. The class information in the management pack is shown in the image below.

There are three rules in the management pack that can generate Critical, Warning or Informational alerts in SCOM.

These rules target a CSV file named 'GalaxySCOM.csv' as their data source. This CSV file is created automatically by the Commvault application and is stored in the '\Program Files\Commvault\ContentStore\SCOM' directory on the Commvault server.

Getting Started

The first thing I'd recommend you do before deploying the Commvault management pack is to make a full list of all the Windows Services relating to Commvault that you wish to monitor. The reason for this is that the Commvault management pack will only monitor whether or not the 'Commvault Server Event Manager Service' (GxEvMgrS) service is up and running. This may be the only Commvault service you're interested in or most likely, you'll have a few more of them that are important to you.

Use the following line of PowerShell to export a list of all Windows Services on your Commvault server to a CSV file:

Get-Service | Sort-Object -Property DisplayName | Export-CSV -path C:\winserviceexport.csv

Once you've identified the service names you need for Commvault, check out my recent blog post here for a quick and easy way to monitor custom lists of Windows Services in SCOM.

The image below shows an example of the Commvault-specific services a customer recently requested to be monitored on all their Commvault servers:

Deploying the Management Pack

When you have all the Commvault services monitored, launch the CommCell Browser using an account with the required administrative permissions and you should be presented with a view similar to the one in the image below. From there, click the Control Panel button from the navigation bar at the top.

When the Control Panel area opens, you need to click the SCOM option from the Monitoring section as shown here....

This opens up the SCOM dialog box (shown below) and here, you need to input your SCOM server name along with a user account and password that has been assigned SCOM Administrator permissions.

When you've added your credentials, hit the Apply button to confirm and then click Test Configuration to validate communication between Commvault and your SCOM server is working as expected.

When you receive confirmation that the test was successful, hit the Import Management Pack button to being the import of the unsealed management pack into SCOM.

When the process is complete, you should see a status message similar to the one in the image below that confirms the Commvault management pack has been configured...

A quick check of the Installed Management Packs view in the SCOM console confirms the management pack has been imported and is ready to go...

You should now see the four simple alert views under the CommVault Operations Manager folder in the Monitoring workspace as shown here....

If you want to confirm the new class has been created and discovered, scope your Discovered Inventory view to CommVault CommServer and you should then see all monitored Commvault servers that SCOM knows about.

Opening a Health Explorer view from the newly discovered CommVault CommServer class object shows how basic this management pack actually is with just the one Service Running State monitor in place to let you know the health state of the Commvault Windows Service.

A quick jump over to the Authoring workspace and we can see the three new Commvault alert rules that have been imported (these rules all target the new Commvault CommServer class).

A check of the Data Source properties for each of the rules gives us the location and CSV file name that will be used to collect alert information from the Commvault server...

Each rule's Data Source has been configured with a wildcard Expression value relevant to the type of alert that will fire (e.g. *Critical*, *Warning* or *Informational*).

If you want to change the name or alert description format of the alert response, you can do that from the Alert properties as shown here...

Configuring the Integration

Once the management pack has been imported and your Commvault servers have been discovered, launch the CommCell Browser again, click Alert from the navigation bar and click the Configure Alert option as shown in the following image...

When the Alerts window opens, you'll be presented with a list of all enabled and disabled alerts in Commvault. We'll click the Add button here to begin the process of creating an alert for SCOM.

From the Add Alert Wizard, type a name for the SCOM alert then choose a category and type. In our example we'll create an alert called Failed Backups and we'll choose the Job Management category with a type of Data Protection.

When you're ready, click Next to move on.

At the Entities Selection window, choose the client groups and/or clients that this alert will be scoped to then hit Next to continue.

From the Threshold and Notification Criteria Selection window, use the Alert Criteria section to scope the alert to the criteria that you need. In our example, we're only interested in Job Failed, Job Skipped and Job Succeeded with Errors alerts. Ignore the other options outside the Alert Criteria section and click Next to move on when you've made your criteria selections.

At the Notification Type(s) Selection window, click the SCOM tab then enable the Select [SCOM] for notification check box as shown in the following image...

Hit Next to continue.

At the Token Criteria Selection window you can optionally add rules to the alert that will dictate if the alerts are sent or not. You can get a full list and description of the alert tokens from here.

We won't specify any rules in our example and when you're ready, click Next to move on.

From the Security window, use the Add button to specify the user accounts and groups that you wish to grant permissions for the alert to (we'll configure an admin account with the Alert Owner role for this alert).

Click Next to move on and at the Summary window (shown in the image below), confirm your settings and hit Finish to end the wizard.

Back in the Alerts view of the CommCell Browser, you can check that the new alert has been created and is enabled as shown below...

That's all you should need to do to configure the integration between Commvault and SCOM and the next time an alert condition has been met, you should see the alert dropping into the Monitoring workspace of the SCOM console similar to this one...

If you've create a new distributed application model in SCOM for Commvault and you use either the Windows Computer or CommVault CommServer class in your component groups, these alerts will rollup to change the health of the model as expected.


Using the walk-through in this post should help people get up and running when monitoring Commvault with SCOM and with some additional distributed application service modeling, SLA planning and dashboard design, you can get some really nice visibility of your backup environments all from a single console.

Monday, April 10, 2017

SCOM - New Community MP for Monitoring Windows Services

A couple of months ago I came across a blog post from SCOM community contributor Andy Leibundgut describing a new management pack he'd authored to help monitor Windows Services using SCOM and I was keen to give it a test drive to see exactly what it could do.

Built-in Windows Service Monitoring Option

You might be thinking to yourself that this capability has to be available already out of the box with SCOM and you'd be correct. The Windows Service Monitoring Template (accessible from the Authoring workspace in the SCOM console and shown in the image below) will walk you through a wizard to help configure a custom monitor for a Windows Service that might not be automatically monitored with a vendor management pack.

The Problem

This template wizard certainly meets the requirements of bringing Windows services into SCOM but as Andy points out in his post, there's a lot of monitoring bloat that comes with each service monitoring configuration you create using the template - each service has its own class, its own discovery and comes with 3 monitors and 8 overrides!

Also, if you have a large number of custom Windows services to monitor (a common request from my customers), then using the built-in template in the console to create a monitor for each one can be fairly time-consuming.

The Solution

With these points in mind, Andy looked into a better method of spinning up monitors in SCOM for Windows services without having to deal with the extra bloat and cumbersome on-boarding process for multiple services. The solution he came up with comprises a new management pack and a PowerShell-based Service MP Editor complete with an easy-to-follow user interface (shown below).

Along with the Service MP Editor, the new management pack contains the following features for monitoring Windows Services:

  • Uses the same data source for Windows Service monitoring that SCOM uses.
  • Date and time filtering so you can exclude certain days/times from monitoring on a per-service or service object basis.
  • Handy console tasks for starting, stopping and checking the status of the Windows Service.
  • Automatic service recovery (disabled by default). Works on a 3 strikes and you’re out format (overrideable setting where after 3 failures in a 24 hour period it will stop trying to restart the service). 
  • Timer reset monitor (closes itself after 24 hours and enabled by default) to watch for and alert on the 3 strike out situation.
  • Monitor all service startup types with the exclusion of disabled services from alerting.
  • Custom discovery which discovers and adds all the service objects to one class rather than scattering them about like the templates do.

Taking the MP for a Test Drive

I've been running this MP solution in my demo and semi-production SCOM environments (both 2012 R2 and 2016) for the last couple of months with no issues and I felt it was about time to spread the word on how much effort you can save when you use it.

The first thing you'll need to do is to download the latest version of this management pack and you can get it from the TechNet Gallery here.

Next up, you'll want to get yourself a coffee (or beer, if that's how you roll) and take a read through Andy's original blog post using the link below:

Note: Everything you need to know is available in Andy's blog post and for clarity, I'll just blog my own experience on getting the MP up and running here.

When you download the zip file containing the MP, extract it to a location on a computer that runs the SCOM console and you should see the following three files...

The Readme.txt contains a note from the author highlighting the fact that this MP is still in it's early days of development and that you should always test it in a non-production environment first.

The WindowsServiceMonitor.xml file is the unsealed MP used for monitoring your Windows Services and the ServiceMPEditor.ps1 file is a clever PowerShell script that launches the UI-based editor to help you customize the MP for your own (and customer) environments.

Before we go any further, we need to import the WindowsServiceMonitor.xml file into SCOM using the Import Management Packs option from the Administration\Management Packs area in the console as shown here....

Next up, we'll create a temporary folder on the same computer that you've just used the console to import the management pack with. We'll create a folder in C:\Temp\SCOM (shown in the image below) but you can use whatever path you wish for this.

Once the folder has been created, launch a PowerShell window with administrative permissions and run the ServiceMPEditor.ps1 script to open the Service MP Editor similar to the following image...

Now, there's a specific order of steps that you need to follow when entering information into the Service Editor and for this part, I've borrowed the original numbered step-by-step image from Andy's blog post that should make things easy to understand...

If you want to add monitors for a small number of Windows Services, then follow these steps in order (we'll cover importing a larger list of services later):
  1. Management Server – type the name of one of your SCOM management servers in this field.
  2. Management Pack Location – type the location of the temporary folder that you created earlier (we'll use C:\Temp\SCOM) into this field.
  3. Get MP Config – clicking this button will export a copy of the original WindowsServiceMonitor.xml management pack from your SCOM environment into the temporary folder location specified in the previous step.
  4. New Service – a click of this button will ready the Service Name field to allow you start a new service configuration.
  5. Service Name – it's imperative that you type the exact 'Service name' of the Windows Service in here and not the 'Display Name' (refer to the example in the previous image taken from Andy's blog where he has highlighted the Service name for the Print Spooler service - which is simply named Spooler).
  6. Confirm Service Edit – clicking this after you've specified the Service name and are finished choosing all your service monitoring options.
  7. Save MP Config – click this button when you're finished editing and ready to commit your changes to the management pack.
Here's a screenshot of what the editor looks like in my demo environment where I've added three services (Windows Firewall, Windows Time and Print Spooler).

After you've clicked the Save MP Config button, you can close the editor and check that the C:\Temp\SCOM\WindowsServiceMonitor.xml management pack contains the newly added services...

Now re-import the management pack from your temporary location back into SCOM using the Import Management Packs from the console. You'll get a notification stating that the management pack is already installed and you can just ignore this and hit the Install button to re-import it again as shown here...

Once the management pack completes it's discovery, you should be able to see the newly monitored services light up in the Discovered Inventory view from the Monitoring workspace of the console (make sure to change the target for this view to WindowsService as shown below).

If you're impatient like me and don't want to wait for the discovery to kick in automatically (by default it's configured to run once a day), you can either reduce the discovery time with an override or simply bounce the Microsoft Monitoring Agent service on the server(s) running the service that you want monitored and you should then see the Discovered Inventory view populating like this...

To test the management pack, stop one of your newly monitored services and after a minute or so, you should see the service roll up as a critical state to the Windows Computer object that's hosting it as shown here in this diagram view...

The nice thing with this management pack is that it comes with some custom tasks to help you manage your monitored services and clicking the Start Service task from the pane on the left (shown below), will then restart the problematic service for you without the need to logon directly to the computer!

Note: You could also enable the automatic WindowsService.ServiceStart.Recovery task option from the Diagnostic and Recovery tab of the monitor to get SCOM to restart the service automatically itself in the event of it stopping unexpectedly. This recovery task will restart the service automatically up to 3 times before giving up and alerting you to the fact that the service is constantly being stopped and started.

Importing a Custom List of Services

If you have a large list of Windows Services that you want to monitor and don't fancy having to manually enter each one into the Service Editor, then there's a handy Import Services option that allows you to import a CSV file with the list of custom services that you want monitored.

You need to understand the format the CSV file needs to be in first however as if you get it wrong, you'll end up having to either edit the XML file directly or just start the whole process again.

Here's the steps I went through in the editor to get this bulk import option working:
  1. Management Server – type the name of one of your SCOM management servers in this field.
  2. Management Pack Location – type the location of the temporary folder that you created earlier (we'll use C:\Temp\SCOM) into this field.
  3. Get MP Config – clicking this button will export a copy of the original WindowsServiceMonitor.xml management pack from your SCOM environment into the temporary folder location specified in the previous step.
  4. Import Services –  this button is used to select a CSV file that contains a list of Windows Service names that you want to monitor. The CSV file must be named WindowsServiceMonitor.csv and needs to be located in the temporary folder location specified in the Management Pack Location field.
Here's a screenshot of the steps you need to take in the editor when you want to import a list from CSV....

When you click the Import Services button, you'll be presented with the warning below stating the name and location that your CSV file needs to have...

Assuming you've named your CSV file correctly and copied it to the temporary location specified in the warning dialog, click Yes to continue and you'll be presented with the custom list of services to be monitored as specified in the CSV you previously created.

CSV Creation Tip #1 - The  CSV file needs to be formatted with four column names (Service, Start, End, DaysofWeekMask) and you need to specify values similar to the image below...

CSV Creation Tip #2 - An easy way to quickly grab a CSV file in the correct format is to just manually add one or two Windows Services to the Service Editor and then use the Export Services button to export a template CSV file that you can edit as you need.

CSV Creation Tip #3 - You can export a full list of Windows Service names to a CSV file by using the following PowerShell command (this exported CSV file won't be in the correct format for the Service Editor so you'll need to then copy/paste the names from here into the previously created WindowsServiceMonitor.csv file):

Get-Service | Sort-Object -Property DisplayName | Export-CSV -path C:\winserviceexport.csv

When you're satisfied with the imported bulk list of Windows Services, the final step you need to complete in the editor is to hit the Save MP Config button to commit the changes to the MP.

Now all that's left to do is to close the editor and re-import the newly updated MP from your temporary location back into SCOM.

After the discovery process completes, you can see the full list of monitored services from the Discovered Inventory view similar to my demo environment here...

All credit goes to Andy Leibundgut for his contribution of this management pack to the SCOM community - and make sure to leave any comments you have on bugs or suggested improvements directly on his original blog post here.