Richard Siddaway's Blog

Multiple test files

Back here http://msmvps.com/blogs/richardsiddaway/archive/2009/11/05/creating-temporary-files.aspx or http://richardsiddaway.spaces.live.com/blog/cns!43CFA46A74CF3E96!2595.entry I showed how to create temporary test files.

We can simply create multiple test files like this

1..10 | foreach {new-tempfile -path c:\test\test1 -size $(Get-Random -Maximum 1mb -Minimum 1kb)}

A one line PowerShell script that generates 10 files in the given folder with random names and sizes.

PowerShell WMI events

In my previous post ( http://richardsiddaway.spaces.live.com/blog/cns!43CFA46A74CF3E96!2598.entry  or http://richardsiddaway.spaces.live.com/blog/cns!43CFA46A74CF3E96!2598.entry ) I started to look at WMI events in PowerShell v2.  The win32_process class was used but all that showed us was that a process had started. We need a bit more information.  A bit of digging brought up the Win32_ProcessStartTrace class that seems to do what we want. 

Register-WMIEvent allows us to specify the class we want to use rather than a query – however if we try that we don’t get anything returned  - oops.

Looking through the help for Register-WMIEvent shows that we have the possibility of performing an action when the event occurs.  The action scriptblock can use a number of variables including $Event, $EventSubscriber, $Sender, $SourceEventArgs, and $SourceArgs automatic variables.  Wanting to understand these variables I tried dumping it though get-member. 

PS> Register-WmiEvent -Query "Select * FROM Win32_ProcessStartTrace" -Action {$Event | gm}

Id              Name            State      HasMoreData     Location             Command
--              ----            -----      -----------     --------             -------
2               c1016218-f80... NotStarted False                                $Event | gm

The subscription runs as a PowerShell job. Using the opening of Notepad to trigger the event we can see that data is returned.

PS> Get-Job

Id              Name            State      HasMoreData     Location             Command
--              ----            -----      -----------     --------             -------
2               c1016218-f80... Running    True                                 $Event | gm

And see that we have a few properties to play with.  ComputerName may come in useful if we are dealing with remote machines.

PS> Receive-Job -Id 2

   TypeName: System.Management.Automation.PSEventArgs

Name             MemberType Definition
----             ---------- ----------
Equals           Method     bool Equals(System.Object obj)
GetHashCode      Method     int GetHashCode()
GetType          Method     type GetType()
ToString         Method     string ToString()
ComputerName     Property   System.String ComputerName {get;}
EventIdentifier  Property   System.Int32 EventIdentifier {get;}
MessageData      Property   System.Management.Automation.PSObject MessageData {get;}
RunspaceId       Property   System.Guid RunspaceId {get;}
Sender           Property   System.Object Sender {get;}
SourceArgs       Property   System.Object[] SourceArgs {get;}
SourceEventArgs  Property   System.EventArgs SourceEventArgs {get;}
SourceIdentifier Property   System.String SourceIdentifier {get;}
TimeGenerated    Property   System.DateTime TimeGenerated {get;}

The properties look similar to those we saw in the last post.  Lets dig into SourceEventArgs

PS> Register-WmiEvent -Query "Select * FROM Win32_ProcessStartTrace" -Action {$Event.SourceEventArgs | gm}

Id              Name            State      HasMoreData     Location             Command
--              ----            -----      -----------     --------             -------
3               8d4246a5-5f8... NotStarted False                                $Event.SourceEventArgs...

PS> Get-Job

Id              Name            State      HasMoreData     Location             Command
--              ----            -----      -----------     --------             -------
3               8d4246a5-5f8... Running    True                                 $Event.SourceEventArgs...

PS> Receive-Job -Id 3

   TypeName: System.Management.EventArrivedEventArgs

Name        MemberType Definition
----        ---------- ----------
Equals      Method     bool Equals(System.Object obj)
GetHashCode Method     int GetHashCode()
GetType     Method     type GetType()
ToString    Method     string ToString()
Context     Property   System.Object Context {get;}
NewEvent    Property   System.Management.ManagementBaseObject NewEvent {get;}

Only thing here that look interesting is NewEvent

PS> Register-WmiEvent -Query "Select * FROM Win32_ProcessStartTrace" -Action {$Event.SourceEventArgs.NewEvent | gm}

Id              Name            State      HasMoreData     Location             Command
--              ----            -----      -----------     --------             -------
4               0857a744-1d3... NotStarted False                                $Event.SourceEventArgs...

PS> Receive-Job -Id 4

   TypeName: System.Management.ManagementBaseObject#\Win32_ProcessStartTrace

Name                MemberType Definition
----                ---------- ----------
ParentProcessID     Property   System.UInt32 ParentProcessID {get;set;}
ProcessID           Property   System.UInt32 ProcessID {get;set;}
ProcessName         Property   System.String ProcessName {get;set;}
SECURITY_DESCRIPTOR Property   System.Byte[] SECURITY_DESCRIPTOR {get;set;}
SessionID           Property   System.UInt32 SessionID {get;set;}
Sid                 Property   System.Byte[] Sid {get;set;}
TIME_CREATED        Property   System.UInt64 TIME_CREATED {get;set;}
__CLASS             Property   System.String __CLASS {get;set;}
__DERIVATION        Property   System.String[] __DERIVATION {get;set;}
__DYNASTY           Property   System.String __DYNASTY {get;set;}
__GENUS             Property   System.Int32 __GENUS {get;set;}
__NAMESPACE         Property   System.String __NAMESPACE {get;set;}
__PATH              Property   System.String __PATH {get;set;}
__PROPERTY_COUNT    Property   System.Int32 __PROPERTY_COUNT {get;set;}
__RELPATH           Property   System.String __RELPATH {get;set;}
__SERVER            Property   System.String __SERVER {get;set;}
__SUPERCLASS        Property   System.String __SUPERCLASS {get;set;}

Now we have got to the information we need.  So how can we use this.  Up to now we have just allowed the job to run and then picked the data from the job.  One option is to write the data to the prompt as shown in this example  http://blogs.msdn.com/powershell/archive/2009/08/30/exploring-wmi-with-powershell-v2.aspx.  A lot of this digging was because I didn’t understand how this was put together.  PowerShell really is the best way to discover how to use PowerShell!!

This gets us to this script which is modified from the PowerShell Team blog

Turns out the ComputerName parameter doesn’t work but a comment on the blog shows how Jeffrey Hicks solved the problem.

What we get now is a listing at our PowerShell prompt  when a new process starts. We can keep working and the data comes through when the prompt is idle.

Next we will look at closing a process and recording the data in a log

PowerShell Eventing

This isn’t the latest sport added for the 2012 Olympics but a way to dig deeper into what is happening on your machine.  There is a continuous stream of events occurring on a computer – programs stop or start, files open or close etc etc.  Some, but all, of these events are recorded in the event logs. If we want to understand what is happening we can track this using the PowerShell Event engine that is introduced in PowerShell v2.

Three types of events can be registered – PowerShell engine, .NET and WMI using the following cmdlets respectively

We can use the following cmdlets to discover the events that actually happen.

Get-Event
Get-EventSubscriber
New-Event
Remove-Event
Unregister-Event
Wait-Event

We’ll start by looking at WMI events. We can use Register-WmiEvent to register the event we want to track. In this case we want to know when new processes are started. We can create an event registration using

Register-WmiEvent              -Query "Select * from __instancecreationevent within 5 where targetinstance isa 'Win32_Process'"             -MessageData "Process Started" -SourceIdentifier "New Process"

__instancecreationevent  is a WMI System Class.  5 refewrs to the system being scanned every 5 seconds

WMI System classes are created on a per WMI namespace basis i.e. a new set of system classes is created for each WMI namespace. The full list of WMI system classes can be seen at http://msdn.microsoft.com/en-us/library/aa394583(VS.85).aspx or can be browsed using PowerGUI's WMI browser.

We can view the system classes relating to WMI events.

Get-WmiObject -Namespace 'root\cimv2' -List "__*Event"

and we will see that there is a __InstanceDeletionEvent class as well.  if we want to track process creation and deletion (program open and close) we will need to register this as well.

Register-WmiEvent                   -Query "Select * from __instancedeletionevent within 5 where targetinstance isa 'Win32_Process'"       -MessageData "Process Stopped" -SourceIdentifier "End Process"

When we run these commands nothing seems to happen. We can see the event registrations (or subscriptions)

PS> Get-EventSubscriber

SubscriptionId   : 1
SourceObject     : System.Management.ManagementEventWatcher
EventName        : EventArrived
SourceIdentifier : New Process
Action           :
HandlerDelegate  :
SupportEvent     : False
ForwardEvent     : False

SubscriptionId   : 2
SourceObject     : System.Management.ManagementEventWatcher
EventName        : EventArrived
SourceIdentifier : End Process
Action           :
HandlerDelegate  :
SupportEvent     : False
ForwardEvent     : False

If we start notepad and and then check the process

PS> Get-Process notepad | select name, starttime

Name                                                        StartTime
----                                                        ---------
notepad                                                     07/11/2009 14:20:19

we can compare this to the event information

PS> Get-Event -SourceIdentifier "New Process"

ComputerName     :
RunspaceId       : 2a581963-55cd-4e46-82ab-ddb6a38fa9a2
EventIdentifier  : 27
Sender           : System.Management.ManagementEventWatcher
SourceEventArgs  : System.Management.EventArrivedEventArgs
SourceArgs       : {System.Management.ManagementEventWatcher, System.Management.EventArrivedEventArgs}
SourceIdentifier : New Process
TimeGenerated    : 07/11/2009 14:20:23
MessageData      : Process Started

Which doesn’t seem to tell is much beyond the fact that a process has started – it specifically doesn’t tell us which process has started.

Similarly when we stop a process

PS> Stop-Process -Name notepad
PS> Get-Event -SourceIdentifier "End Process"

ComputerName     :
RunspaceId       : 2a581963-55cd-4e46-82ab-ddb6a38fa9a2
EventIdentifier  : 29
Sender           : System.Management.ManagementEventWatcher
SourceEventArgs  : System.Management.EventArrivedEventArgs
SourceArgs       : {System.Management.ManagementEventWatcher, System.Management.EventArrivedEventArgs}
SourceIdentifier : End Process
TimeGenerated    : 07/11/2009 14:27:47
MessageData      : Process Stopped

We get a message that the process has stopped but no identification as to which process.

Events only exist in the current session and the subscriptions are lost if the PowerShell session is closed.

Couple of quick points

The event queue can be quickly cleaned using

Get-Event | Remove-Event.

We can remove event subscriptions using

Unregister-Event -SourceIdentifier "New Process"
Unregister-Event -SourceIdentifier "End Process"

We will dig further into the eventing capabilities in future posts

SQL Server books

A couple of SQL Server books that I would recommend having been involved with both of them.

First up is SQL Server 2008 Administration in Action by Rod College (Manning - http://www.manning.com/colledge/) which I reviewed several times before publication. It covers SQL Server from sizing and installation, through configuration and day-to-day administration.  Lots of good best practice information written in a easy to read style.

The second is SQL Server MVP Deep Dives (Manning - http://www.manning.com/nielsen/) to which I contributed a chapter on PowerShell and SQL Server. This book has contributions from 53 MVPs (mainly SQL Server) and is edited by Paul Nielsen, Kalen Delaney, Greg Low, Adam Machanic, Paul S. Randal, and Kimberly L. Tripp.  The royalties from the book go to War Child International - (www.warchild.org)

Two good books to add to your SQL Server library

Removing empty folders

I blogged about this a long time back. As part of my tidy up program (me? tidy up? - - stop laughing!) I’ve been moving scripts into PowerShell v2 modules.  I’ve added these three functions to my FileFunctions module.

The Get-EmptyFolder function accepts a folder path and uses the Scripting.FileSystem object for the folder. This is passed to Test-EmptyFolder which tests the subfolders for size and adds the paths of empty ones to the output file. 

Test-EmptyFolder is then called recursively with each subfolder. Recursion is a very powerful technique for re-using the same function and looping through a nested structure.

A switch parameter is used to determine if the contents of the file should be displayed at the end. Switch parameters can be used to provide true\false values. In this case we display unless told not to.

The output file can then be processed by Remove-EmptyFolder. The file contents are read into an array which is then read bottom to top.  This enables me to delete subfolders before high level folders. Splitting the processing also allows me to review the file contents before folder removal.  As an additional check the –whatif parameter is used unless the -remove switch is used as a parameter.

The module manifest only publishes get-emptyfolder and remove-emptyfolder keeping test-emptyfolder as a hidden helper function.

I have a few more functions to add to this module then I’ll re-publish it on my sky drive.

Creating Temporary files

One thing I need to do when testing file system scripts is generate a bunch of temporary files for experimenting with.  In the past I have just copied in whatever I could find to use as test data.  It is possible to easily generate test data of this sort using the New-TempFile function below.

The function takes two arguments – a path and a file size. My default values are given but they can be easily changed to suit you system.

After checking that the path exists I generate a random file name using GetRandomFile().  We could use GetTempFileName() but that creates an empty file in the temporary folder which isn’t quite what I want.

If the size is zero an empty file is created on the given path. Otherwise if size is supplied e.g.

New-TempFile -size 5kb

then a number of 128byte lines are written into the file.  If larger files are being routinely created increase the size of the string to 512 bytes – or put a test on say 100kb size and use the bigger string in that case.

Add-Content is used to write the data into the file.

One enhancement is to allow the setting of the file extension.

Cleaning Temp folder

Windows uses a temporary folder as a scratch pad and dumping ground for all sorts of temporary files.  There are a couple of issues with the temporary folder. Firstly by default its part your profile and each user has their own. Secondly there isn’t any real clean up performed on it.

The first issue can be dealt with by editing the contents of the TEMP and TMP environmental variables. I’ll show a script for this another time.  The second we can deal with quite easily with these functions.

We can use Get-TempContents to determine the current contents of the temporary folder.  I’ve grouped by extension just to get a feel of the file distribution.  If you want to see the whole listing then leave the group command out.

PowerShell exposes a number data stores via providers. One of these is the environmental variables. These can be accessed using

Get-ChildItem env:

where env: is the PowerShell drive for the environmental variables. Using aliases this becomes

ls env:   or  dir env:

Now we have an idea of what files we have we can remove them.

We can start by defining a list of file extensions that we want to delete. I have decided to only delete files older than 7 days so we generate a date for this time last week using the AddDays method of the DateTime object that Get-Date returns.

We can then get the contents of the temporary folder. Check the extension and the LastWriteTime and remove those files that match the criteria.

Next job is to schedule this job which is a topic for another day.

No PowerShell = No Job?

James has an excellent post - http://blogs.technet.com/jamesone/archive/2009/11/02/you-can-t-be-a-21st-century-admin-without-powershell.aspx – explaining why PowerShell is going to be a must have skill for IT admins going forward.

As far as I am concerned he is preaching to the choir :)

But he has a very valid point. PowerShell is the Microsoft automation tool that will be built into all major products. Eventually we can expect administrative GUIs to be layered over PowerShell cmdlets and expose the commands they are running when we execute a command. This is the pattern established by the Exchange team.

James follows with a list of Microsoft products that already contain PowerShell – BTW SQL Server 2008 uses PowerShell v1 already we don’t need to wait for SQL 2008 R2 for PowerShell support.

What makes the PowerShell story even more compelling is the third party companies that are adopting PowerShell:-

Quest, Special Operations Software, IBM, Symantec, F1, Citrix, VMware

that I can think of immediately.

This makes PowerShell a tool that will be everywhere in a Microsoft environment.

The new features in PowerShell v2 – especially remoting and background jobs – make the case even more compelling for the admin of the future.

Of course the question now is how can I learn PowerShell – plus I don’t have time to automate.

Make the time the rewards are many.

Will the title come true? Probably not but if you want to do more than spend your day clicking on the GUI – learn PowerShell.

Hamming Distance

Catching up on my reading I came across this post - http://blogs.msdn.com/powershell/archive/2009/09/20/what-s-up-with-command-prefixes.aspx which is a good explanation for why we should use command prefixes when creating cmdlets.  There is a reference to prefixes increasing the Hamming Distance of noun names.

Not knowing what a Hamming Distance is I followed the link to http://en.wikipedia.org/wiki/Hamming_distance and found that it is a measure of the difference between two strings i.e. how many substitutions are required to turn one string into another.

We can find the Hamming Distance as follows

Input a couple of strings, check they are the same length and then compare on a character by character basis.

The script can be used like this

PS> .\Get-HammingDistance.ps1 toned roses
Hamming Distance for toned and roses is 3

PS> .\Get-HammingDistance.ps1 baston boston
Hamming Distance for baston and boston is 1

PS> .\Get-HammingDistance.ps1 2173896 2233796
Hamming Distance for 2173896 and 2233796 is 3

Date and Time

I recently came across a script that used date and time information in this manner

This gives a result of

12Oct,2009,15:20:32,Stuff

I didn’t like the way get-date was used three times – its messy .  In most cases it won’t make much difference but if the script runs across a day boundary you just might get some odd results.

The same results can be achieved with

which will give a result of

12Oct,2009,15:20:38,Stuff

The time differences are related to me clicking across different tabs in ISE

In this way we only call Get-date once and use the .NET string formatting to give the results we want.  Much neater and we don’t need to worry about time or date boundaries

Word module

I’ve posted a PowerShell module that includes the functions in earlier posts to my skydrive  -   http://cid-43cfa46a74cf3e96.skydrive.live.com/browse.aspx/PowerShell%20Scripts

Enjoy

Unblocking files

When I download files from the Internet to Vista or Windows 7 the file is blocked and I can’t execute it until its unblocked. This is especially annoying with zip files because if I unzip them all the contents are blocked. I have been meaning to do something in PowerShell for this and finally got round to it.  The blocking is because of a zone identifier in an alternate data stream (ADS) attached to the file. If you have been around PowerShell for any length of time you may remember MoWs post on this subject - http://thepowershellguy.com/blogs/posh/archive/2007/01/27/powershell-accessing-alternative-data-streams-of-files-on-an-ntfs-volume.aspx

This was my starting point but when went to download the file containing the .NET classes we need to access the ADS from http://www.codeproject.com/KB/cs/ntfsstreams.aspx I found that the classes had been updated. So first off download the .NET stuff and extract the dll containing the classes (after unblocking the zip file).

I decided to add this functionality into my module for working with files.

First point is that I need to load the .NET assemblies from the dll.  I can do this in the .psd1 file by adding this line

RequiredAssemblies = 'Trinet.Core.IO.Ntfs.dll'

I then added two functions into the module. One to test if a zone identifier exists and another to delete it

The Test-ZoneIdentifier function will accept a file from the pipeline and test the path to the file.  If the file exists the function will then test to see if a Zone.Identifer ADS exists. If it does it will read the ADS and work out the zone using the ZoneIdentifier enum which we create in the module as well.

This is adapted from MoWs code at http://thepowershellguy.com/blogs/posh/archive/2008/06/02/powershell-v2-ctp2-making-custom-enums-using-add-type.aspx

A hash table could also be used at this point if preferred.

The function Remove-ZoneIdentifier will delete the ADS and unblock the file.  By enabling these functions to work on the pipeline we can process a number of files in one hit.

The functions are used as follows

PS> Import-Module filefunctions
PS> Get-ChildItem -Path c:\test\NtfsStreams.zip | Test-ZoneIdentifier
Zone.Identifier present. Type Internet  If of Type Internet the file will be blocked.
PS> Get-ChildItem -Path c:\test\NtfsStreams.zip | Remove-ZoneIdentifier
True
PS> Get-ChildItem -Path c:\test\NtfsStreams.zip | Test-ZoneIdentifier

The ntfs streams dll you will need to download separately.

Codeplex AD Replication Module

The other codeplex project that caught my eye was a brand new one to create a PowerShell module to manage AD replication.  This one will be very useful and one I will be using a lot.

There is still time for suggestions as to content for this project – see http://adreplicationmodule.codeplex.com/

Codeplex PowerShell Configurator

I was looking at codeplex (Microsoft’s Open Source site) and decided to do a search for projects relating to PowerShell. 161 projects were returned.  The first 110 had an obvious PowerShell connection.  This is a tremendous number and really does demonstrate the strength of the PowerShell community.

One project that caught my eye was James O’Neill’s PowerShell configurator for Server 2008 R2 Core and Hyper Server R2.  Its done as a PowerShell v2 module with the following functions

Managing installed software , drivers and updates

Managing the windows Firewall

IP Networking

Licensing

Page file

Remote Desktop

Other Windows Configuration

There shouldn’t be any reason why it won’t work on full fat Windows so I’ll definitely be trying it out.  Download from http://psconfig.codeplex.com/

Word Add text

Keeping on with our look at Word we need to able to add text to the documents we are creating.

Add a paragraph then set the style and the paragraph text.  The text could come from a file using get-content. InsertParagraph() pushes the paragraph into the document.

Instead of setting the style we could explicitly set the font and its size.

Word add data to a table

We created a table in a recent post.  This time round we add some data to the table.

The row and column together with the text to enter into the cell are passed as parameters. 

$table.Cell($row, $col).Range.Style = “style_name” can be used to set the style of the table cell contents.  We can even set the style at the table level if required.

Module Contents

I’ve been writing a few modules recently and sometimes forget what I have in each module. Simple way to view the functions each module makes available.

List the available modules and for each one import the module and then use Get-Command to find the functions each one loads. We can then remove the module.

Word Close Document

So far we have created a document and started to add data into the document.

We pass a file name in as a parameter. Notice that the parameter is mandatory. If we don’t supply the file name we will be prompted. If just a file name is given it will be saved into your standard word location (usually My Documents). If you want it to save somewhere else then use the full path.

The SaveAs() method is used for the save.  If you give the name of a file that exists it will overwrite!

The final line of the script closes the Word application.

PowerShell v2 help file

In PowerShell v2 we get a nice graphical help system that includes the cmdlets, the about files and and the User and Getting Started Guide. It doesn’t include the help for the optional modules though which is a pity.

The default locations on the Start Menu for PowerShell are Accessories – Windows PowerShell.  That is not very good placement considering Windows only really exists to give us somewhere to run Powershell :-)  

In these locations the help files are not visible. If you open up ISE then you can access the help file.

If you pin PowerShell to the start menu on Windows 7 then you get access to the help through the recent files menu.  The help file is located (on my system) at C:\Windows\Help\mui\0409\WindowsPowerShellHelp.chm

I keep a shortcut on my desktop and a shortcut to the version 1 graphical help file from the script center.  that way I can compare changes.

Word Add a table

There seems to be two types of paragraph I create in Word documents – text or tables.  Lets start by adding a table.

The function Add-Table takes two integers as parameters. They define the rows and columns of the table.  The way that seems to work best is to add a paragraph to contain the table and then add the table into the paragraph.  If we just add the table to the document it overwrites the contents of the document with the table. oops.

Notice that I’m using global variables again so that they can be used across the environment.

Next time we will add some data to the table.