Categories
Business Intelligence Policy Based Management Power BI Syndication TSQL Tuesday

T-SQL Tuesday #75: Jump into Power BI

20121003200545.jpgSo this month I’ve decided to bring every participant in this blog series kicking and screaming into the BI world. Power BI, if you haven’t heard or seen it yet, is Microsoft’s analytics and visualization solution. This month I challenged everyone to create their own Power BI report and share it with the world via our new Publish to Web feature. So far I’ve seen some great contributions from fellow bloggers, and those will shared in the upcoming round-up post. You can also check out the winners of the recent Power BI Report Contest (Congrats Jason and James!).

For my contribution to this contest I’ve decided to share with you a work in progress. If you know me, I’m a huge lover of Policy-Based Management. In fact, I’m actually part of the Enterprise Policy Management Framework (EPMF) project on Codeplex. T-SQL Tuesday event is normally a DBA-centric event so I figured I’d help the DBA crowd wrap their heads around how a BI solution can help them in their day to day.

What I did to kick start this effort was to create this Power BI report that allows you to explore the database repository that contains the EPMF policy evaluation results. The current EPMF project uses Reporting Services to deliver its reports. This won’t change. If anything I’ll be exploring new capabilities with SQL Server 2016 and R-integration. Here’s a screenshot of what the SSRS dashboard report looks like:

DashboardScreenshotv4_cropped.jpg

 

While this report is nice, it can be a little cumbersome doing interactive analytics against this data set. The reports are somewhat “interactive” in that you can click on a visualization to drill down in to a detail level report behind it, but it’s not really interactive. Enter Power BI! Below is my first swag (read also: none of this is set in stone for final EPMF release, any/all feedback is appreciated) at creating a Power BI version of this dashboard. And yes, I’ll be doing away with pie chart. Dataviz people rejoice!!!

Direct link to report here

I’ve included a lot of the existing elements from existing report including seeing detail level information of policy results breakdown by server and policies. You also get interactive elements such as choosing policy results, server names, month. On the trending and details page (second tab), I’ve used some custom visuals from Power BI’s Custom Visuals Gallery such as the tadpole spark grid plus to show trending over time.

The best part of using Power BI to visualize on your operational data? You can access your reports and insights anywhere, anytime through our mobile apps! We’ve got apps on all major platforms (iOS/Android/Windows Mobile) as well as a universal Windows 10 app. Think about this. Many large organizations pay thousands upon thousands of dollars for monitoring software that includes capabilities to push to mobile, and even then they’re usually shrunken mobile websites. This is a native application functionality in the palm of your hands at a fraction of the price, so cool!

What else is in the future for EPMF? I’ll be rebuilding the data model from the ground up to make it more efficient for reporting. The current model is very simplistic, and works, but as enterprises adopt EPMF and monitor more and more, the current model has certain scale issues. What do you think? Are you excited for new developments in EPMF? Are you using EPMF today? Let me hear it in the comments!

Categories
Policy Based Management Syndication

EPM Framework v4 Released!

The Enterprise Policy Management Framework (EPMF) team is proud to announce the updated release of “Enterprise Policy Management Framework 4.0 for project EPMFramework. This is a major release with lots of performance tweaks and

The EPM Framework 4.0 includes the following updates:

  • For enhanced support of large environments
    • Reviewed database design, including views and indexing
    • Redesigned data load procedure
  • PowerShell execution now deletes XML files as soon as load is done – improves space usage on temp folder
  • Redesigned reports
  • Tested/verified against SQL Server 2005-2014

Note: an upgrade script for all the relevant database objects is provided, supporting direct upgrade from v3. Please check the documentation for further information.

HUGE thanks to the hard work from Pedro Azevedo (Twitter | Blog) who developed and tested this new version. Do you use Policy-Based Management (PBM) at work? Give this new version of EPMFramework a spin and let us know what you think.
Categories
Policy Based Management Syndication

Expiring Databases and Policy-Based Management

Today on Twitter my friend Jes “Run Forrest Run” Schultz Borland (Blog | Twitter) asked the Community “How do you clean up your dev environments? Let DBs sit out there forever? Delete after X months? Other? This seemed like an interesting issue to tackle and me being the PBM freak that I am, immediately I had a light bulb moment for a policy. In this post I’ll show you a policy you can run against your databases (can work in dev or whatever environment suits you) and will tell you which databases are older than 30 days old. As an added bonus, I’ll also show you how to add a custom extended property to set a custom expiration date.

Categories
Featured Policy Based Management Syndication

Make It Simple With Policy-Based Management

At work recently, I explained to a coworker the reasons and benefits of having databases in Simple mode in your development environment. Funnily enough, a few hours later, I see my friend Andie Letourneau (Blog | Twitter) posted a blog on how to adjust your recovery modes in development. While Andie’s homework assignment is to substitute it with a WHILE loop, I have MUCH simpler solution using Policy-Based Management!

First, let’s create the condition and policy. As always you can either use this T-SQL or you can simply download the XML policy and import it:

Download Policy Here

TSQL:

--Create Condition First
Declare @condition_id int
EXEC msdb.dbo.sp_syspolicy_add_condition @name=N'Simple Recovery Mode', @description=N'Ensures databases are set to SIMPLE recovery mode', @facet=N'Database', @expression=N'
  Bool
  EQ
  2

    Numeric
    RecoveryModel

    Numeric
    Enum
    Numeric
    2

      String
      System.String
      Microsoft.SqlServer.Management.Smo.RecoveryModel

      String
      System.String
      Simple

', @is_name_condition=0, @obj_name=N'', @condition_id=@condition_id OUTPUT
Select @condition_id

GO

--Create Policy
Declare @object_set_id int
EXEC msdb.dbo.sp_syspolicy_add_object_set @object_set_name=N'Simple Mode Check_ObjectSet', @facet=N'Database', @object_set_id=@object_set_id OUTPUT
Select @object_set_id

Declare @target_set_id int
EXEC msdb.dbo.sp_syspolicy_add_target_set @object_set_name=N'Simple Mode Check_ObjectSet', @type_skeleton=N'Server/Database', @type=N'DATABASE', @enabled=True, @target_set_id=@target_set_id OUTPUT
Select @target_set_id

EXEC msdb.dbo.sp_syspolicy_add_target_set_level @target_set_id=@target_set_id, @type_skeleton=N'Server/Database', @level_name=N'Database', @condition_name=N'', @target_set_level_id=0

GO

Declare @policy_id int
EXEC msdb.dbo.sp_syspolicy_add_policy @name=N'Simple Mode Check', @condition_name=N'Simple Recovery Mode', @policy_category=N'', @description=N'This policy ensures that the databases evaluated against are set to SIMPLE recovery mode.', @help_text=N'', @help_link=N'', @schedule_uid=N'00000000-0000-0000-0000-000000000000', @execution_mode=0, @is_enabled=False, @policy_id=@policy_id OUTPUT, @root_condition_name=N'', @object_set=N'Simple Mode Check_ObjectSet'
Select @policy_id

GO

Now that you’ve created your policy, you can either evaluate it directly on that server or we can leverage the Central Management Server feature to evaluate this policy against one or more servers in your environment! Since we’re looking at this from an enterprise-level standpoint, we’re going to go over the CMS way of doing this.

Once you’ve established your CMS, you’re going to want to register servers to it. You could register everything directly under the CMS but then it becomes one giant list and that’s not very nice. One method I like to implement in my CMS is to create new server groups for each environment (e.g. DEV, QA, PROD). Within each group, I also create additional groupings for each version of SQL Server (e.g. 2000, 2005, 2008). Once the groups/folders have been created I then register my servers in their appropriate spots.

The advantage of breaking groups up like this is that CMS allows you to query against multiple servers at once. For instance if you were to right-click the folder for DEV, which in my case contains two registered servers in groups beneath it, SSMS will connect to both instances so that you can query them at the same time. It is this mechanism that we’ll use with Policy-Based Management since that feature allows us to evaluate policies against multiple servers in the same way.

To evaluate this policy, right-click the DEV folder and select Evaluate Policies. For source, click on the ellipses button and select either the folder location of the XML file or select the SQL Server instance which you imported the policy to. Once you’ve selected the policy’s location, you’ll see the Evaluate Policies screen with a list. Tick the box for the Simple Mode Check and then click the Evaluate button.

Check that out, you just evaluated policies against all of the databases in DEV! The databases that are NOT in simple mode (per our condition check) fail the policy evaluation and show up with red X’s. To quickly switch those to simple mode simply check the boxes for those that failed the check, then click the Apply button. This will enforce the policy on those databases and switch them for you to Simple mode! Imagine doing this that quickly and easily against hundreds of databases!

If you’re wondering if you can automate this process, the answer is absolutely! Check out the great open-source project Enterprise Policy-Management Framework over at Codeplex. This project allows you not only automate this process and policy enforcement but it also offers some really nice reporting of all this as well!

UPDATE: After chatting with Andie it looks like I forgot to mention a few things. Namely that for those wondering if this works on down-level servers (2000, 2005, etc.) the answer is YES! So long as you have a SQL Server 2008 server acting as your CMS, you can evaluate (certain) policies against down-level servers. Given this confusion I’ll write up a post on how all this works and what the caveats are.

The other point to make is that CMS uses Windows authentication only. If you have multiple domains, and there is no trust established between the domains, then this solution won’t work. If you DO have multiple domains and trust established, so long as your credentials allow you to traverse domains and the proper security authorizations are in place on the target SQL Servers then it will work.

Categories
Policy Based Management Syndication

SQL Server Denali CTP3: Policy Based Management

Whether you know it or not, Policy-Based Management is an integral part of the new release of SQL Server “Denali”. Did you know that the HADR feature actually leverages PBM for checks? Yup, that’s right folks. If you’re an administrator and you’re NOT using Policy-Based Management yet, I highly recommend you pick up a book and check it out!

So what’s new in SQL Server “Denali” CTP3 for Policy-Based Management? I just got the shrink wrap off of it (I’m not quite the go-getter that Aaron Nelson is) but at a quick glance here’s the changes I see. These comparisons were made with listing of facets between SQL Server 2008 R2 and the new “Denali” CTP3 release.

 

We now have 84 facets (as opposed to 76 in 2008 R2). These are our new facets and their applicable target types!

Availability Database – Exposes properies of the Availability Database object
Applicable Target Types: AvailabilityDatabase
Availability Group – Exposes properies of the Availability Group object
Applicable Target Types: AvailabilityGroup
Availability Group State – Exposes properies of the Availability Group
Applicable Target Types: AvailabilityGroup
Availability Replica – Exposes properties of the Availability Replica object.
Applicable Target Types: AvailabilityReplica
Database Replica State – Exposes the properties of physical database replicas participating in an availability group.
Applicable Target Types: DatabaseReplicaState
Search Property List – Exposes properties of the Search Property List object
Applicable Target Types: SearchPropertyList
Sequence – Exposes properties of the sequence object.
Applicable Target Types: Sequence
Server Role – Exposes properties of the ServerRole object.
Applicable Target Types: ServerRole

 

Again, this still has that new install smell so I haven’t had time to mess around with it but I’ll soon be posting some new policies using these facets so you can get an idea of what you can do administratively using Policy-Based Management and these new facets.

Categories
Policy Based Management Syndication

Policy-Based Management and Local Password Policy

This post is based on an interesting question/situation that was posted over at ASKSSC.com today. The user asked how to create a policy condition that enforces local sql accounts to adhere to password expiration policies.

First off, to create the condition itself is relatively easy. Below I’ve provided the T-SQL code so that you can create the condition that way. I’ve also included a quick list on how to create it via SSMS GUI.

T-SQL method:

Declare @condition_id int
EXEC msdb.dbo.sp_syspolicy_add_condition
    @name=N'Password Policy Enforced', @description=N'', @facet=N'Login',
    @expression=N'<Operator>
  <TypeClass>Bool</TypeClass>
  <OpType>AND</OpType>
  <Count>2</Count>
  <Operator>
    <TypeClass>Bool</TypeClass>
    <OpType>EQ</OpType>
    <Count>2</Count>
    <Attribute>
      <TypeClass>Bool</TypeClass>
      <Name>PasswordExpirationEnabled</Name>
    </Attribute>
    <Function>
      <TypeClass>Bool</TypeClass>
      <FunctionType>True</FunctionType>
      <ReturnType>Bool</ReturnType>
      <Count>0</Count>
    </Function>
  </Operator>
  <Operator>
    <TypeClass>Bool</TypeClass>
    <OpType>EQ</OpType>
    <Count>2</Count>
    <Attribute>
      <TypeClass>Bool</TypeClass>
      <Name>PasswordPolicyEnforced</Name>
    </Attribute>
    <Function>
      <TypeClass>Bool</TypeClass>
      <FunctionType>True</FunctionType>
      <ReturnType>Bool</ReturnType>
      <Count>0</Count>
    </Function>
  </Operator>
</Operator>', @is_name_condition=0, @obj_name=N'',
@condition_id=@condition_id OUTPUT

Select @condition_id

GO

SSMS method:

  1. Under PBM node, right-click conditions folder and select New Condition
  2. Name your new condition something useful
  3. Select Login facet from drop-down menu
  4. Click on field box and select @PasswordExpirationEnabled from properties list
  5. Set the operator value to True
  6. Click on ‘Click here to add clause’ to add another clause to policy
  7. Click on field box and select @PasswordPolicyEnforced from properties list
  8. Set the operator value to True
  9. Click OK. You’ve now just created a new condition!

Now we’re left with another question: Where does this password policy come from? For details on that you can refer to the Books Online article about it (link). If your box is on a domain that has Active Directory policies regarding password expiration, when you select the box for ‘Enforce password policy’ as well as ‘Enforce password expiration’, these settings will come from that policy. Don’t have an Active Directory policy? No problem! If a policy isn’t provided from Active Directory, Windows then looks to its local security policies for these values.

image

To see the local values, click on your Start button, then type in ‘secpol.msc’ (don’t type type the single-quotes). This will open up the Local Security Policy MMC Snap-in. Expand the Account Policies folder and then click on the Password Policy folder. In the right side pane you will see the various password-related options you can set such as Maximum password age or password length.

image

While policy-based management can help you check whether or not the accounts have the option enabled to enforce the policy checks, Policy-based management itself has not bearing on the Local Security Policy settings. This is something you, as an administrator, will have to set and configure outside of SQL Server.

Categories
Policy Based Management Syndication

Find Table Heaps Using Policy-Based Management

This is just a quick post in regards to a conversation I just had via Twitter. If you don’t already use Twitter, the SQL Community has setup a great resource on there using the hashtag of #sqlhelp.

Today a conversation came up due to a forum question over at SQLServerCentral regarding applying policies to databases with tables that have heaps. If you’re not familiar with the term, a heap is a table that has no clustered index on it. This can be problematic from a performance stand point so it might benefit you to find a way to identify these potential problem children. Enter Policy-Based Management.

This is a simple policy that you can run against your servers and it will identify your tables that are heaps. Just to clarify this policy identifies if your table has a clustered index on it. If it doesn’t then it will fail policy check. I’ve provided two ways to get the policy.

Download policy by clicking here

OR (Updated 4/15/11 to include creation script for condition)

--CREATE CONDITION
Declare @condition_id intEXEC msdb.dbo.sp_syspolicy_add_condition @name=N'Find heaps', @description=N'', @facet=N'Table', @expression=N'<Operator>  <TypeClass>Bool</TypeClass>  <OpType>EQ</OpType>  <Count>2</Count>  <Attribute>    <TypeClass>Bool</TypeClass>    <Name>HasClusteredIndex</Name>  </Attribute>  <Function>    <TypeClass>Bool</TypeClass>    <FunctionType>True</FunctionType>    <ReturnType>Bool</ReturnType>    <Count>0</Count>  </Function></Operator>', @is_name_condition=0, @obj_name=N'', @condition_id=@condition_id OUTPUTSelect @condition_id
GO

--CREATE POLICY
Declare @object_set_id int
EXEC msdb.dbo.sp_syspolicy_add_object_set @object_set_name=N'Find Table Heaps_ObjectSet', @facet=N'Table', @object_set_id=@object_set_id OUTPUT
Select @object_set_id

Declare @target_set_id int
EXEC msdb.dbo.sp_syspolicy_add_target_set @object_set_name=N'Find Table Heaps_ObjectSet', @type_skeleton=N'Server/Database/Table', @type=N'TABLE', @enabled=True, @target_set_id=@target_set_id OUTPUT
Select @target_set_id

EXEC msdb.dbo.sp_syspolicy_add_target_set_level @target_set_id=@target_set_id, @type_skeleton=N'Server/Database/Table', @level_name=N'Table', @condition_name=N'', @target_set_level_id=0
EXEC msdb.dbo.sp_syspolicy_add_target_set_level @target_set_id=@target_set_id, @type_skeleton=N'Server/Database', @level_name=N'Database', @condition_name=N'', @target_set_level_id=0

GO

Declare @policy_id int
EXEC msdb.dbo.sp_syspolicy_add_policy @name=N'Find Table Heaps', @condition_name=N'Find heaps', @policy_category=N'', @description=N'Heaps are tables without clustered indexes. Read the link below to learn more about heaps.', @help_text=N'Fragmentation (part 4):what are heaps? by Paul Randal', @help_link=N'http://blogs.msdn.com/b/sqlserverstorageengine/archive/2006/09/19/761437.aspx', @schedule_uid=N'00000000-0000-0000-0000-000000000000', @execution_mode=0, @is_enabled=False, @policy_id=@policy_id OUTPUT, @root_condition_name=N'', @object_set=N'Find Table Heaps_ObjectSet'
Select @policy_id

GO
Categories
Policy Based Management Syndication

Policy for Ad-hoc Workloads

During my presentation at SQLSaturday 62 in Tampa I was asked by an attendee about having a policy to check the setting for ad-hoc optimization settings. At the time since I was in a bit of a time crunch (and I couldn’t remember the exact facet to look under) I couldn’t properly demo how to check for it. In this post I’ll show you how to check for that specific setting. In a future post I’ll show you how to check on many more settings.

Before we begin, I highly recommend you familiarize yourself with what exactly this setting changes and how it affects your SQL Server environment. Remember this setting affects the entire instance so all databases installed here will be affected by this change. Read this great post by Bob Pusateri (Blog | Twitter) to get an understanding of what Optimizing for Ad Hoc Workloads really does.

Creating the Policy/Condition

  • In SQL Server Management Studio browse down to and expand your management node, expand the Policy-Based Management node, right-click the Policies folder and select New Policy.
  • Name your new policy and then from the Check Conditions drop down menu select New Condition.
  • Give your new condition a name and from the Facet drop down menu select the Server Configuration facet.
  • In the Expression editor, click the area below the column title of field and you will be presented with a drop-down of all the properties available for this facet. Select @OptimizeAdhocWorkloads.
Creating our new condition

  • Under the heading of Value, you will have two options: True or False. When you create a policy you want to establish a condition you want so for the purposes of this demonstration we want our servers to have this setting set to off (which is default setting) so we’ll select the option for FALSE. Click OK to create your condition and return to the new policy window.
  • Next we’ll select our Evaluation Mode. This policy, based on the facets and properties we’ve selected offer us three options: On demand, on schedule and On Change: log only. The last option, if enabled, will allow this policy to be active and log any changes made to this particular setting. One cool thing you can do with this is you can create alerts to automatically email you if this particular condition is violated. Check out Ken Simmons (Blog | Twitter) article on Configuring Alerts for Policy-Based Management to learn more. Leave the Evaluation Mode to On Demand and click OK.

Now that we have our policy created simply right-click on it (located under your Policies folder) and select Evaluate to try it out!

GUI? We Don’t Need No Stinking GUI!

In this post I walked you through how to create this policy using the GUI but if you prefer to script this out, you can do that too! Here is the T-SQL script that you can run in lieu of walking through the SSMS screens, to create this particular policy:

[code lang=”sql” wraplines=”true”]
Declare @object_set_id int
EXEC msdb.dbo.sp_syspolicy_add_object_set @object_set_name=N’Ad-hoc Workload Check_ObjectSet’, @facet=N’IServerConfigurationFacet’, @object_set_id=@object_set_id OUTPUT
Select @object_set_id

Declare @target_set_id int
EXEC msdb.dbo.sp_syspolicy_add_target_set @object_set_name=N’Ad-hoc Workload Check_ObjectSet’, @type_skeleton=N’Server’, @type=N’SERVER’, @enabled=True, @target_set_id=@target_set_id OUTPUT
Select @target_set_id
GO

Declare @policy_id int
EXEC msdb.dbo.sp_syspolicy_add_policy @name=N’Ad-hoc Workload Check’, @condition_name=N’adhoc optimization check’, @policy_category=N”, @description=N’This policy checks the server setting to see if Optimize for Ad-Hoc Workload is enabled. The default setting is disabled.’, @help_text=N’To learn more about this policy check out Jorge Segarra”s blog post on this’, @help_link=N’https://sqlchicken.com/2011/01/policy-for-ad-hoc-workloads/’, @schedule_uid=N’00000000-0000-0000-0000-000000000000′, @execution_mode=0, @is_enabled=False, @policy_id=@policy_id OUTPUT, @root_condition_name=N”, @object_set=N’Ad-hoc Workload Check_ObjectSet’
Select @policy_id
GO[/code]

Conclusion

Again, I can’t iterate enough NOT to blindly go changing settings on your servers without understanding the effects of your actions! Policy-Based Management is a very powerful and easy-to-use tool but be sure to use it wisely! In a later post I will show you how to modify even more server-level settings and let you customize policies to check exactly the settings you want audit.

Categories
Policy Based Management Syndication

EPM Framework and SQL 2008 R2

This weekend at SQLSaturday 62 in Tampa, I presented my policy-based management presentation. During my presentation
one of the cool things I cover is how policy-based management can be extended utilizing Reporting Services and PowerShell
through the use of an amazing tool called the Enterprise Policy-Management Framework available on Codeplex.

Enterprise Policy Management Framework, or EPMF, is completely free and was developed by the folks at Microsoft who created
policy-based management. I absolutely love telling folks about this project because it really helps sell the idea of
policy-based management’s application within an organization. What’s cool about this project is the built in reports make it easy to see the health state of your environment at a glance as well as let you drill down further in to each report piece to find more granular information on policy states.

One caveat of EPMF is that in order to run on SQL Server 2008 it requires SP1 Cumulative Update 3 or higher installed on your Central Management server in order to function properly. This requirement is in place in order for EPMF to be able to properly handle policy evaluation on down level systems (e.g. SQL Server 2000, 2005). An interesting question was asked during the presentation: “Does EPMF support SQL Server 2008 R2 RTM (10.50.1600)?” The answer is YES, it does!

I tested this on my local install of SQL Server 2008 R2 at RTM level and it works. Even though it works at RTM, I highly recommend you update your SQL Server 2008 R2 instance to at least Cumulative Update 3 or higher. I know, you’re thinking “but you just told me it works at RTM!” Yes, it does, however the RTM edition of R2 came with quite a nasty little bug that wasn’t fixed until the CU3 patch. This bug is outlined in this Connect issue by Aaron Bertrand (Blog | Twitter). The bug is that SSMS will not allow you to edit or create a job step after you’ve created an initial one. How does this affect you? Well when you setup EPMF you need to create a new scheduled job that executes the PowerShell script that evaluates the policies against your environment. This particular bug will stop you from editing or creating new job steps which could severely affect you trying to fix things. There is a workaround wherein you can close/reopen SSMS to make the error disappear but this can become quite cumbersome very quickly.

Policy-based management is an extremely powerful and easy to use feature in SQL Server 2008 and EPM Framework extends its awesomeness even further. If you’d like to learn more about Policy-based management you can check out some webinars I’ve done over at Pragmatic Works (webinar link) or at SQLLunch (webinar link) on the topic.
Categories
Events Policy Based Management Syndication

PragmaticWorks Webinar – Policy-Based Management

This is just a quick note to remind everyone I’ll be presenting tomorrow, Tuesday October 26th 2010 for the PragmaticWorks Free Monthly Webinar Series. The topic (big surprise here) is Policy-Based Management!

The webinar runs from 11:00 AM – 12:00 PM EDT and you can register for free here. Hope to see you all there!