Martin Zugec blog : Scripting

Create file with specified size

martin — Mon, 16 Jun 2008 07:46:00 GMT

Yesterday I run into very strange problem with one of my friends - Echo. stopped working.

In batch scripts, you use Echo to output something to screen - and of course you can redirect it to file.

By using Echo. (mind the dot), you specify that you want to output empty line. So if you want to create empty file from batch, you can use Echo. > c:\Temp\Empty.txt

Yesterday one script stopped working - Echo. was not working. It was working for everything else (including log files etc), just not for creating files in one folder. I have absolutely NO IDEA how that was possible (and if someone does, share any ideas, I am very curious about this one).

I remembered of less known feature from FSUtil - FSUtil can create for you empty file with predefined size. And it worked:

FSUtil File CreateNew C:\Temp\Empty.txt 0

In my case I used it only as alternative method because Echo. failed, however it can be useful if you want to benchmark performance or do something similar.

For example if I want to create 10 files with 10GB size, I would run following command:

for /l %i IN (1,1,5) Do FSutil File CreateNew C:\Temp\PerfTest\Test%i 1073741824

This would create 5 files with 10GB size. What I like about it that all files are created within miliseconds.

Lnk, shortcuts - real disaster??

martin — Thu, 29 May 2008 22:20:57 GMT

Maybe you noticed that one of my projects I would like to finish in summer is also offline shortcuts editor - and you probably think "why some editor when I can build shortcuts using Explorer?"...

Think twice - it's not as easy as it looks like and you will be maybe surprised how many gotchas are there.

Problem with .lnk files is that they are WYSINWYG (What You See Is NOT What You Get). Windows is sometimes too "intelligent" - and it can be very, very annoying.

My blog is primarily focused on scripting, deployment and SBC - so I expect that my audience have also some experiences with this. And most people tried to create some shortcut and copy it somewhere else (you create shortcut to some drive letter on your PC and then copy it to desktop of all users for example). Problem is that together with shortcut you provide more information that you know - you provide also last successful resolution of that shortcut and you won't see this information anywhere in properties of such shortcut.

Let me show you example. You map home drives of your users to H: drive. You want to have shortcut to H:\MyDocs on your desktop. So if you create shortcut to H:\MyDocs, then you just need to distribute it to all your workstations and its done. You open properties of this shortcut and you see that it points to H:\MyDocs, so it should work everywhere, right?

WRONG - don't forget, Windows can be sometimes unpredictable ;) If you open properties of your .lnk file, you can see that it is (still) pointing to H:\MyDocs - however .lnk file also contains information that H:\MyDocs is in fact \\Servers\Administrator$\MyDocs and it will automatically open this folder even from different PC! Quite nasty, right?

And thats not all - try to remove that drive letter and open shortcut - because it remembers that location, it will automatically map H: to \\Servers\Administrator$\MyDocs

If you will try to map something else (\\Servers\Projects) to H: drive, Windows will surprise again - not only it will map that shortcut to different drive (X:), but it will also modify your shortcut so it is automatically changed from H:\MyDocs to X:\MyDocs... Which means that if you have 50 shortcuts pointing to H: drive, 1 accidental click can corrupt that shortcut for you.

You can find many (well, not tons of, but still few of them exists) offline shortcuts editors, however most of them covers only situation I described above (and not of all them covers this). There are some bugs that are (as far as I know) not solved by any existing product - for example if you use variables to define your folders (for example %Programs% for D:\Programs), you WON'T be able to create such shortcut if variable is not defined. When you try to create it, C:\ prefix is automatically appended (so %Programs% is in fact pointing to C:\D:\Programs, which is invalid value).

Well, this was just quick overview why I want to have offline editor where all such bugs will be fixed (btw I already have first version of code and it works correctly there, so it must be something in Explorer itself)... There are tons of other bugs - I will try to spend some time testing different results and share results with you. I am now very sorry Microsoft didn't implement something like XLNK in Windows Vista :(

When is storing configuration in Windows directory good idea?

martin — Thu, 29 May 2008 09:30:00 GMT

Well, I know this is quite controversial, but in some cases (specially scripting frameworks) I think it is good idea to use Windows directory to store some settings\configuration...

Why that? Generally you can always divide such solutions to push or pull (and mixed of course) - that means that (if we talk about servers), sometimes you push some data to server (from your workstation\laptop) and sometimes you want server to retrieve information from some central storage\share.

Windows directory is suddenly only directory that is easily accessible both locally and remotely - from remote system you can use \\Server\Admin$ and from server itself you can simply use %WinDir% variable.

I know most people would say that you can use \\Server\C$ and %SystemDrive%, but you never know if C: drive really exists (this is happening in SBC environment when you want to have client's C: drive mapped at C letter).

So for these reasons I like to use Windows directory - and then I make symlink\junction to some other location (so for example C:\Windows\CentralStorage\MyFramework --> C:\MyFramework). That way all required files are easily accessible for scripts (both push and pull) and also for administrators or people working with such environment.

There are of course another ways how to achieve that - create new shares etc, however this method is most reliable and can be easily accessible for all scripts\tools and also for all administrators.

SystemSherlockGUI - one package that rules them all ;)

martin — Mon, 26 May 2008 20:42:13 GMT

Good news everyone - I just received email from Roger (a.k.a. Kephyr) - he is author of brilliant SystemSherlock. He allowed me to include SystemSherlock in one package with my GUI - so now you can download one package with both GUI and command line interface :)

To remind you, SystemSherlock is snapshoting utility (like Regshot), that can track changes on your filesystem or registry...

Snapshoting is sometimes (often) better than using ProcMon\RegMon\FileMon - you only see real changes, no reads etc... So I prefer snapshoting when I for example want to find where in registry are some changes hapenning.

Compared to RegShot, SystemSherlock is I think much better - you can compare unlimited dumps (not only 1st & 2nd snapshot) and I like GUI more ;)

Download and enjoy :)

P.S.: If you have any feature requests or bugs to report, feel free to contact me, I would like to keep SystemSherlock alive

SystemSherlock - snapshot using GUI or CMD

martin — Sat, 17 May 2008 21:32:00 GMT

Recently I wrote about SystemSherlock Lite - really nice snapshoting tool that supports command line... I also posted small utility for parsing log files.

After that I started to heavily use SystemSherlock - and I must say that it is really really great utility. Problem is that usually you want to have command line AND GUI interface - and SystemSherlock is cmd only :(

So I decided to create wrapper around it - and for me combination of GUI and SystemSherlock is much better snapshoting tool than RegShot or InstallRite...

SystemSherlock GUI consists of 3 different tabs - one for creating snapshots, second for comparing snapshots and third for displaying log files in friendly structure.

Create snapshot

Below is GUI used for taking snapshots:

It allows you to create snapshot configuration, specify output file and also to include exclusion list if you want to ignore particular entries. This is configured snapshot for detecting HKCU and C:\Temp folder:

I tried to implement quite logical interface, so there are feature like auto-suggest or auto-repair of entries (for example HKLM is automatically translated HKEY_LOCAL_MACHINE)... Another feature is that entry type is automatically detected (you can see it in video - I don't select whether entry is registry or filesystem, it is automatically filled):

Update: Embedded video from Jing doesn't work :( So click on following link instead).

Once you specified what you want to monitor, just click on create button - dump is automatically created. Then do whatever you want - and just click on Create button again. One of SystemSherlock advantages is that it allows you to create as many dumps as you want - you can even compare current dump with one created months ago...

When finished, move to Compare snapshots tab.

Compare snapshots

Below is screenshot of Compare snapshots tab:

It is divided to two parts - on left you can see source dumps and on right target dumps. Source dumps are the ones that were taken first - this is VERY important to understand that source must be always older.

System Sherlock Lite can report unexpected results if you are not aware of this behavior - if you will swap source and target (so target will be older file and source will be newer file), results will be opposite (for example if you deleted folder between snapshots, it will report that this folder was created etc.).

For this reason I implemented some logic to processing. When you select any file on left, ONLY newer files then selected are displayed on right.

Parse logs

When you compare two dumps, differences between these two entries are automatically displayed in GUI.

Current version should be already functional, but I am sure that there will be some issues - after all, this is really first version I just finished. If you encounter any problems or you have some features requests, feel free to post them in comments - if nothing, I will at least respond (but probably also implement such changes).

Advantages compared to RegShot:

Fully supports command line - can be scripted
Imho better GUI
Supports multiple dumps - not only comparison of 2 snapshots (easily review historical changes)
Exclusion list based on RegExes
GUI for reading log files

Download

Utility to display logs from SystemSherlock Lite

martin — Thu, 15 May 2008 10:29:00 GMT

As mentioned in my post about SystemSherlock Lite, I really love this tool after few days... Only problem I had is that it can take some time to realize what really happened - you need to read pretty big log files and you can spend precious time reading through temporary entries or documents and settings and miss one important entry saying that applications copied something to your System32 folder.

I was playing with Sherlock yesterday and I spent lot of time parsing through log files generated by SystemSherlock. Then I spend 20 minutes writing some automated parser and I want to share with you ;)

Usage is pretty simple - just click on Open log file, select your log file and output similar to following should appear:

I dont know what else to add - this utility is pretty simple, however can save your time ;) One more notice - log parse is able to handle also mixed logs (so if you have one log where you store different entries together with log output from SystemSherlock Lite, you can use it and it will only show entries from SystemSherlock Lite.

Any comments, requests? Feel free to post comments...

Another nasty Robocopy bug

martin — Wed, 14 May 2008 18:26:00 GMT

Recently I wrote about bug I found in Robocopy - one of my favorite utilities... And today I found another one.

I am using robocopy for synchronization - which is VERY important part of our framework (if nothing is synchronized, nothing can be executed).

We use one folder for storing all log files. During migration to newer version of framework, I wanted to see only new entries for easier troubleshooting (many log files are not overwriting, but rather appending). So I renamed my Log folder to Log.Backup and tried to run synchronization... I run only backend part - that is part that is missing all checking, creation of required folders etc.

To my surprise my synchronization failed - when I renamed Log.Backup to Log again it was working.

Robocopy reported very strange error:

ERROR : Invalid Parameter #4 : "/Log+:C:\Temp\Log\Robocopy.log"

I am using same syntax for many years, so I was really surprised. I double checked robocopy version to be sure that I didnt accidentaly replaced it with some 3.11 version - nope, I was using current version.

Then I started to have that scary feeling that I know what is wrong - I tried to run it again, this time specifying folder that didnt exist - and I got same error as before.

To summarize it - if Robocopy is not able to write to file, it reports "Invalid parameter". If you specify that you want to store your settings in log file in folder that doesn`t exist - bang, "Invalid parameter".

Tested with newer and older versions of Robocopy - XP010 (Resource Kit), XP026 and XP027 (Vista).

Make snapshots of filesystem and registry using command line

martin — Wed, 14 May 2008 12:43:00 GMT

Some installers can be very mysterious - this suddenly applies for MSI packages, even using full logging, it is not easy to find out which registry keys or files were modified\deleted\added. If you have application that is using some custom installer, this can be hell...

Colleague of mine came with really nice idea - what about using some tool like InstallRite or RegShot for monitoring all installations and reporting what was changed during that installation? From idea it is usually not far to implementation - you only need some tools to do it...

I used utility called System Sherlock Lite - it is very simple to use command line utility that can take snapshots from filesystem\registry and compare them. Of course it is freeware ;) It also supports exclusion list (using regexes), so you can easily configure it for your needs and ignore entries you are not interested in (usually log files or temporary folders).

So what I did is that I created 2 different scripts - StartMonitor and StopMonitor.

StartMonitor is creating first snapshot.

StopMonitor will create second snapshot, compare it with first snapshot and store differences in installation log file. Then it will delete both snapshots.

This implementation is really easy, but is working really nice. Now I can easily monitor all installation I want automatically without any knowledge about installer being used or installation method. It allows me to easily debug anything from logon\logoff scripts to software installations...

Martin

New (simple) tool - SimpleScheduler

martin — Sat, 10 May 2008 09:23:00 GMT

I run into simple problem - I needed some (very basic) implementation of scheduled tasks. Scheduled tasks that are built-in to windows are not my best friend - problem is that (before Windows Vista, here they are completely different beast) it was not that easy to manage (automatically) scheduled tasks.

For example if you wanted to reboot your servers by scheduled task and then enable them with another, you cannot be 100% sure that it will work. Problem is that if server is available when scheduled task should be executed, it will ignore it later on.

For this reason I wrote really small script for myself and named it SimpleScheduler. It is using DateX, EchoX and Tee from Bill Stewart`s Shell Scripting Toolkit (which again I can only recommend).

To enable it, just create one small scheduled task (give it any name you want, mine is SimpleSchedService) and configure it to run every few minutes (depends on what you want to run through it). I wrote about this concept few times ago and I still really like it, because it allows you to make very robust and reliable scripts (even if one run fails, another will usually finish its job).

Once SimpleScheduler is running, that all you need to do on backend. All you need to do now is just create new jobs for it.

Creating new jobs is pretty easy once you get used to it. Job always consists of 2 elements:

Job.cmd - script that will run. If you want to run anything else, to it from this script.
<DateID>.NextRun - this file identifies when you want to run scheduled task. <DateID> is date format used by DateX (with -t switch), so for example 950688458.NextRun.

To create NextRun entries you must\should use DateX to generate ID when you want to run script. If you find this confusing, just post comment and I can add small script that will translate it automatically from you.

By default all jobs that are not scheduled to run again are automatically deleted (you can change this in SimpleScheduler.ini configuration). Every job will automatically create log file (using Tee.exe from Bill).

Main goal of this small script is to make more reliable scheduled tasks (ones that are immune to server reboot) and also to make easy scheduled tasks creation (all you need to do here is just copy folder to add new task or transfer it from one server to another).

If you find it useful, please add some comments. If you want to add\change anything, please also write comments :)

Useful tools for scripts

martin — Thu, 27 Mar 2008 10:09:00 GMT

One of my favorites are from SST (Shell Scripting Toolkit) from Bill Stewart (http://www.westmesatech.com/), but I dont understand why they are not really known in community.

So I decided to post a little bit about them, maybe someone could be interested.

Toolkit contains following utilities:

CCase.exe
ColorX.exe
DateX.exe
DriveX.exe
EchoX.exe
FInfo.exe
IfX.exe
LineX.exe
S2V.EXE
ShellEsc.exe
SleepX.exe
Str.exe
Tee.exe
TempName.exe

CCase

Change Case - allows you to change case using pipe, pretty simple and sometimes can be VERY useful. Just pipe output through ccase you you get results you want:

C:\Temp\BillStewart>echo Testing cHAnGe Case | ccase -l
testing change case

C:\Temp\BillStewart>echo Testing cHAnGe Case | ccase -u
TESTING CHANGE CASE

ColorX

Do you want to use custom color scheme in your scripts? Or manipulate with it? ColorX allows you to detect current settings or change them.
If you use -l switch, ColorX generates really nice overview of available colors, so you can quickly choose your combination. For example I like to have specific color of output to stress some facts (for example Success is light green, failure is red and important messages are orange), so you immediately see if there is anything wrong. Problem is that many people change default color, for example using red foreground (so all errors and warnings got lost).

I can build simple workaround using ColorX:

:: Store original settings
For /f "UseBackQ tokens=*" %%c IN (`ColorX`) Do Set Settings.OriginalColor=%%c

:: Change to MY settings (white frontground, black background)
ColorX -c 0F

:: Do something....

::Restore original settings
ColorX -c %Settings.OriginalColor%

DateX

One of the biggest pains when handling with batches are date\time manipulation. Problem is that they are dependant on your regional settings and if you need to manipulate with multiple countries, scripts are getting VERY complex. Not to mention problems if you want to increase or decrease values - my script for getting (universal) file creation time is 90 lines long. You will also probably (sooner or later) discover some cave-eats of batches - for example batch Set /a behavior. For example you want to store date one month ago in variable. It starts pretty simply - you just parse %Date% variable or use Date/t.

Today is Thu 03/27/2008 - and you would like to run something like Set /a 03 - 1 (March - one month) and then have some check like If %Result% LSS 1 set /a %Result% + 12 (that is probably easiest method how to handle with months). Still not see problem? If you would implement it today, you script will work for 5 months - and 1st of August it will stop working. Because we see "03", Set /a automatically consider number is octet - and 08 doesnt exist in octat, so you will receive error:
Invalid number. Numeric constants are either decimal (17),
hexadecimal (0x11), or octal (021).

Well, to make long story short, just ignore batch functionality and use DateX instead. I am using DateX for two purposes - first one is for time comparison (here is it extremely useful) and also for predefining format I expect.

Example of time comparison:

For /f "usebackq tokens=*" %%x IN (`DateX -t`) Do Set Temp.Start=%%x

::Do something

For /f "usebackq tokens=*" %%x IN (`DateX -t`) Do Set Temp.End=%%x

Set /a Temp.Duration=%Temp.End% - %Temp.Start%

Echo Script was running for %Temp.Duration% seconds

Of course changing it to minutes\hours etc is not problem at all.

DriveX

Allows you to manipulate with drive and retrieve type of drive. Only problem I encountered (but there is probably nothing that can be done to fix this) is that it is not able to detect substed drives (mapped using Subst). Returned errorlevel identifies drive type - so 3 is fixed, 5 is CD-ROM etc..

It is most usefull if you want to process particular drive types. For example unmap all network drive command is pretty simple using DriveX:
For /f "UseBackQ Tokens=*" %%d IN (`DriveX -l 4`) Do Net Use %%d: /Delete

EchoX.exe

EchoX is probably my favorite utility and I use it with most of my scripts. What it allows you to do? Its pretty simple, you can change color of output (not WHOLE console window, only one line). Also I really like usage of escape characters that allows you to include line feed and return commands in one line.

Consider scenario where something goes reaaaaally wrong - and you want to notify user. You want to display following message:

:: Do something

WARNING: Problem was detected when using script X.
WARNING: Script will now abort.
WARNING: Please contact your administrator

:: Do some recovery

Using batches, you must write following "code":

:: Do something

Echo.
Echo WARNING: Problem was detected when using script X.
Echo WARNING: Script will now abort.
Echo WARNING: Please contact your administrator
Echo.

:: Do some recovery

Which is not bad, problem is that same color is used and it is really easy to miss such important information. With EchoX you use

EchoX -c 0C "~nWARNING:Problem was detected when using script X.~nWARNING: Script will now abort.~nWARNING: Please contact your administrator~n"

Of course you can split it to three lines. What is important is that this message is obviously error. It is really nice if you use it together with If Else:

If /i #%Result%# EQU #OK# (
EchoX -c 0A "Operation finished successfully"
) Else (
EchoX -c 0C "WARNING: Something wrong happened"
)

That way you can change your batch script to appear more like utilities. Highly recommended, when I was searching for something like this I found SST. You can find (very few) tools that do the same, but all I tested had problems with buffer (text was truncated, sometimes corrupted etc)

FInfo.exe

FInfo allows you to dump some basic information about files. You specify file and you can get path only, file name etc. This functionality is not that important - you can achieve the same by implementing %~ (if you are interested, leave some comment and I can write how to use it).

What is more interesting is option to get last modification time in same format as DateX (universal, no parsing needed).
Also option to retrieve file size is really nice - output is obviously made for parsing (BTW Bill, if you read it, I cant stress enough how I appreciate parseable output ;)).

IfX.exe

Not really useful today, because built-in If contains same functionality. Obviously it was build for 9x OSs, so dont expect anything really fancy here. It would be nice if it would contain features like IfX Starts With, IfX Ends With, IfX Contains etc...

LineX.exe

Allows you to handle with output, generally Windows tails\head tools. Nice feature is that you can retrieve line n (so if your second line is always ::Created by, you can easily retrieve this). Usually you need to write some For /f with counter that tells you when to stop parsing - which is quite time consuming and not really reliable, so LineX can really help.

S2V.EXE

MS-DOS utility to store output in variable. Replaced in NT by For /f, I never had chance to try it.

ShellEsc.exe

Could be really nice :( You can shellescaped any text - which is really great if you automate automation like I do ;) Problem is that ScellEsc was never really working for me - when it encountered empty line, it always crashed with message

Runtime error 216 at $0040107F
$0040107F
$0006FFA8
$FFFFFFFF

I completely forgot about this tool and I will try to write Bill if he still supports SST and if he could fix it - because I like to organize my code to make it easily readable.

SleepX.exe

Nice replacement of original Sleep. While original sleep allows you to specify time only, SleepX allows you to display message also and add option (and this is why I like it) to abort sleep by any key. One feature I am missing is that I would like SleepX to sleep forever if no timeout is specified and option to abort was enabled. I know I could use Sleep & Pause combination, but the idea is that you could allow people to define timeouts for different messages like this and use one code for all configurations.

Str.exe

Str allows in general two operations - first one is to change to uppercase\lowercase and echo number of characters in string. For upper\lower case I prefer to use CCase, because it accepts pipe, but number of characters is sometimes very useful (especially if you want to do something where limited characters are allows). Second functionality is to detect string inside string and return starting position. I prefer using Str to handling with Find (you can detect if string exists by checking errorlevel), also because it allows me to easily detect if string STARTS with different string (which is sometimes quite important).

Tee.exe

Tee allows you to split output - so you can create log file that contains everything that is displayed to console. I like this concept, but prefer MTee to Tee because it contains more functionality.

TempName.exe

This tool could be quite useful, but I am still using script to do this:

:GenerateCacheFolder
Set Temp.ID=%Random%
If Exist "%Folder.Root%\Cache\%Temp.ID%" Goto GenerateCacheFolder
Set Temp.Cache=%FolderRoot%\Cache\%Temp.ID%
MkDir "%Temp.Cache%"

Ugly bug in Robocopy - ignoring security on file level

martin — Mon, 03 Mar 2008 20:34:00 GMT

I really like RoboCopy (robust file copy) utility from Microsoft - if you dont know this tool (I would be surprised), it adds tons of features to standart Copy command - and probably the most useful is /MIR (mirror) that creates mirror of source folder.

One of main benefits of mirror is that it is really fast and you can always use it to easily replicate different folders. Problem is that Mir is ignoring NTFS security.

I know, I know, you can use /SEC - but you can still run into issues. Problem is that when using MIR switch, robocopy ignores ACL - so if you use RoboCopy Source Target /MIR /SEC, security settings are transfered - BUT if they are changed on source or target, these ACLs are not mirrored. This is only problem of files, not folders(!).

This can be really annoying, currently you can use following as workaround:

RoboCopy /Mir <Source> <Target>
RoboCopy /E /Copy:S /IS /IT <Source> <Target>

Second line will copy files again, but now with security...

Hope so Microsoft will fix this in next (if any) version of RoboCopy :(

UPDATE: So, there are two newer versions than XP010 - version XP026 (that is part of Robocopy GUI) and XP027 (this is part of Windows Vista). Problem is that XP027 is not working under Windows XP\2003 (not tested, if someone have Vista, please test and let me know). Version XP026 is part of (not really useful - my opinion) Robocopy GUI. You can either install it and get Robocopy from System32... You can then use syntax RoboCopy <Source> <Target> /SecFix /xo /xn /xc /e /COPY:S
This wont copy files, it will just reapply security settings.

UPDATE2: Due to fact that it is not that easy to get Robocopy XP026 and you must install (quite ugly and useless) GUI to get it, I uploaded it to Microsoft SkyDrive:

Empty tokens in For /f?

martin — Tue, 13 Nov 2007 10:38:00 GMT

I received email with following question from Otto:

I have file with delimiters, however not every token have values. For example following file:

test1;test2;test3
t1;;t3
;te2;te3
tes1;tes2;

If I try to use For /f to parse it (For /f "usebackq delims=; tokens=1,2,*" %%i IN (`Type Test.txt`) Do Echo a=%%i b=%%j c=%%k, following is output:

a=test1 b=test2 c=test3
a=t1 b=t3 c=
a=te2 b=te3 c=
a=tes1 b=tes2 c=

As you can see, For /f ignores empty tokens... So how to solve it?

Only solution that came to my mind was to recommend using search&replace utility to replace these values first... However after some time I thought - "Hey, this is task ideal for regex replacement".

So I tried yesterday to write EmptyTokenParser.cmd that should be able to handle it (using SED from GnuWin32).

@Echo off
Set Int.File=%~1
Set Int.Delimiter=%~2

type "%Int.File%" | sed s/"^%Int.Delimiter%"/"#empty#%Int.Delimiter%"/g;s/"%Int.Delimiter%%Int.Delimiter%"/"%Int.Delimiter%#Empty#%Int.Delimiter%"/g;s/"%Int.Delimiter%$"/"%Int.Delimiter%#Empty#/g

Usage is quite easy - EmptyTokenParser "Test.txt" ";" and result will appear like this:

test1;test2;test3
t1;#Empty#;t3
#empty#;te2;te3
tes1;tes2;#Empty#

You must download SED to make it work. SED among many GREAT command line utilities can be found at http://gnuwin32.sourceforge.net/.

So the final script:

For /f "usebackq delims=; tokens=1,2,*" %%i IN (`EmptyTokenParser "Test.txt" ";"`) Do Echo a=%%i b=%%j c=%%k

Output?

a=test1 b=test2 c=test3
a=t1 b=#Empty# c=t3
a=#empty# b=te2 c=te3
a=tes1 b=tes2 c=#Empty#

P.S.: Do you also use your blogs to write reminders? ;)

HookApplication - some real-life examples

martin — Thu, 01 Mar 2007 12:59:00 GMT

I still dont think its time for public release, however I just wanted to share with you some different situations where I liked to use HookApplication...

For most of this actions I needed to write some plugins - I decided to keep HookApplication as simple as possible to have it as error-prone as possible.

First I tried to replace processes - I replaced Task Manager with Process Explorer and after some time also Notepad with Crimson Editor...

Right now I am thinking about installing Maxthon 2.0 and if I found it useful, I will replace Internet Explorer with it.

This was quite interesting, however nothing really special. For example Process Explorer supports replacement of Task Manager in its options...

Then I found out I can use HookApplication to solve one of my problems I got right now - locking processes with passwords.

Why do I need this? I am citrix administrator and if I am debugging something, I would like to be able to run task manager - however I want to prevent users to do the same. Problem is that I often need to run task manager under THEIR account, not mine. So I wrote small stupid plugin, that will pop-up password message box when you try to run specific process and if you dont know password.

I got it working with task manager - and it works great :)

So, I was thinking what were other ways how users were "hacking" citrix boxes usually - one of them is somehow (taskmgr, word etc - there are many ways) running cmd and then you can do almost what you want.

Created new plugin - if process is launched without any parameters (or vice versa), pop-up input box with password :) That means that scripts will run (cmd + parameters), however without password you are not able to use it...

Another usage was related to allowing only one instance running - I have few applications that doesnt always exit correctly (GUI not visible, however application still running) and some of them are extremely stupid when handling such situation (Miranda wont show GUI for example...). New plugin - check instances - if more than x running, do something (for some processes just popup message box, for some kill current process, for some kill rest of processes....). Quite simple, however it helped me a lot, since this I havent experienced any problems :)

Then I have some network related applications - for example I am killing all processes that are "calling home" before running VPN and then I am running them again or I am checking if NIC is enabled when trying to connect through EDGE (sometimes it fails on Unknown error if card is disabled)...

Another useful scenario for me was using Hook Application together with AutoIt scripts - sometimes you just want to automate something, however its not as simple as it looks like - you can end up with never-ending script (which is quite stupid) that will wait under condition appears (window title appears for example) or you can run your program using script (which is nothing for me as sometimes run programs using Start Menu, Launchy or shortcuts ;)). So with hook application I just assigned few AutoIT scripts with related processes and viola - its working exactly as I was expected :)

Also I got some processes that I sometimes accidentaly close, but I want to keep them running all the time - for example GUI for my scripts. So I created entry that will run the command again when it is closed and it is working :)

Introduction to batch services

martin — Mon, 12 Feb 2007 09:28:00 GMT

This blog post is going to be interesting mostly for people that got already a lot of experiences with batch scripting.

I am going to talk about quite interesting stuff I call batch services - I love this technology and it saved me and helped me many times in all my previous projects.

Batch service is in fact quite simple - it is just batch, that got actions linked to different folders and it is running periodically.

In my current job, we are using batch services for automatic rebooting of servers and for sending mails, however it was extremely useful for deployment because you were able to do a lot of stuff with very simple scripts.

Consider scenario where you want to allow your technicians to unlock some accounts for example without giving them permissions.

You write script called UnlockAccountEngine.cmd. This script will consist of following lines:

For /f "usebackq delims=. tokens=1,*" %%i IN (`Dir /s /b "\\server\UnlockAccounts$\Account.*"`) Do (

Echo %Date%;%Time%;%UserDomain%;%%j>>"\\Server\UnlockAccounts$\Unlock.log"

Net User %UseDomain%\%%j /Domain /Active:Yes

)

Extremely simple, right? What it means is that it will unlock every account when file Account.UserName is available. For example if I would like to unlock my account, I will just create file Account.Mzugec.

Now we will create client script call UnlockAccount.cmd

This file contains following lines:

@Echo off

Set /p Int.AccountName=Please specify account name which you would like to unlock:

If Exist \\Server\UnlockAccounts$\Account.%Int.AccountName% (

Echo WARNING: Request to unlock this account was already sent

) Else (

Echo.>\\Server\UnlockAccounts$\Account.%Int.AccountName%

)

If Exist \\Server\UnlockAccounts$\Account.%Int.AccountName% (

Echo Request to unlock account %Int.AccountName% is registered, please wait few minutes and try it again

) Else (

Echo WARNING: Request to unlock account %Int.AccountName% was not created, please contact your IT support

)

This script is really simple, you just specify account name and then you will see if request (file) was successfully created or not.

Now we need to perform just last step - create scheduled task called UnlockAccountEngine and schedule it to run every 3 minutes (!).

What will happen? Every 3 minutes script UnlockAccountEngine will run and it will unlock any account that is locked. Extremely simple, but also extremely useful in fact :)

There are many advantages - this script wont hang, you don’t need to delegate any privileges, you can use scripts, web pages, even emails to generate such files (to do something, you just create one file, can it be simplier?)…

There are many different scenarios what you can do with batch engines. For example if you are deployment guy, you can let users "request" software based on their needs, or you can run some program on remote machine (VNC server?) and many many other options.

How to detect if folder exist (and ignore file with same name ;))

martin — Tue, 30 Jan 2007 08:50:00 GMT

Today I experienced really annoying bug with one of my scripts. It is checking if one folder (in quite complex structure) exists(If Exist Folder...), if it does, it perfoms some actions and send few mails. However after some time it started to behave strange - it was sending sometimes mail when folder was definitely not there etc.

It didnt take me long time to discover what was the problem - another team was using file without extension with same name as my folder :) So I rewrote today this script to detect ONLY folder and ignore files with same name.

How to do it? Its quite simle

If Exist Folder\nul.ext Send-Mail

For example If Exit C:\Temp\test\nul.ext will be True if folder test exists, however not if file test exists.

You just need to be very careful - Windows doesnt allow you to have folder and file with same name in same folder:

C:\Temp>mkdir Test

C:\Temp>Echo.>test
Access is denied.

C:\Temp>rmdir Test

C:\Temp>Echo.>test

C:\Temp>mkdir Test
A subdirectory or file Test already exists.

C:\Temp>

How to dial using default RAS connection

martin — Mon, 29 Jan 2007 21:14:00 GMT

TuxExplorer send me question how to dial connection using default RAS connection - I wrote small script (two liner) to do that:

For /f "usebackq tokens=1-3" %%i IN (`reg query "HKCU\Software\Microsoft\RAS AutoDial\Default" /v DefaultInternet ^| find /i "REG_SZ"`) Do If /i #%%j# EQU #REG_SZ# Set Int.DefaultRASConnection=%%k

If Defined Int.DefaultRASConnection Start RasPhone -d %Int.DefaultRASConnection%

It should work, I just connected using this script :)

Martin

How to specify timeout for script?

martin — Mon, 29 Jan 2007 17:01:00 GMT

Maybe you run into this issue before - you want to run some script/utility (for example query user), but you know that it hangs sometimes and you would like to have some kind of timeout utility. There is way how to handle this in batch scripts using two concurent scripts:

Create three scripts, for example Scripts.cmd, Pause.cmd and Detection.cmd

Content of Script.cmd is

Set Int.Command=%~1
Set Int.Pause=%~2

Set Int.PauseTitle=%Random%PauseScript
Set Int.DetectionTitle=%Random%DetectionScript

Start /Min Cmd /c Detection.cmd
Start cmd /c exit /b 1
Start /Min /Wait cmd /c Pause.cmd
If %ErrorLevel% EQU 0 (
Echo Failure
) Else (
Echo Success
)

Content of Pause.cmd is

Title %Int.PauseTitle%
Sleep %Int.Pause%
Taskkill /f /t /fi "WINDOWTITLE eq %Int.DetectionTitle%"

Content of Detection.cmd is

Title %Int.DetectionTitle%
%Int.Command%
Taskkill /f /t /fi "WINDOWTITLE eq %Int.PauseTitle%"
Exit /b 0

Then call Script.cmd with two arguments, command to perform and timeout. For example Script.cmd "pause" "10".

It will try to execute command "Pause", and if it took more then 10 seconds, it will kill this process and report it using errorlevel.

Quite simple, however functional. This way it is also fast, because if your scripts finished faster, it will automatically remove Sleep subroutine that will hold main script execution.

I am using this procedure right now to detect hanging sessions on servers - and it turns out to be really simple and functional, so I wanted to share :)

Security issue with PowerShell

martin — Thu, 25 Jan 2007 08:47:00 GMT

Hi guys,

after (too) long time I can finally return to Powershell (I spend few months creating batch framework to manage citrix servers, right now it is about 980 scripts).

I was playing with Get-Credential:

$Operator = get-credential

I provided my username/password. If I have a look at object, I can see Password property and GetPassword() method. I tried this, Output is in System.Security.SecureString.

I was just thinking how PowerShell is handling explicit authentication, so I had a look at available properties and methods - GetNetworkCredential grab my eye, so I tried it. And I was really surprised by output:

mzugec>$operator.GetNetworkCredential()

UserName                   Password                   Domain
--------                           --------                         ------
mzugec                       YouPwdInPlainText     MyDomain

Replace YouPwdInPlainText with your actual password! So be aware, if you are using get-credentials, it is very easy to retrieve your actual password!

Using functions in cmd (advanced scripting no.3)

martin — Fri, 31 Mar 2006 08:22:00 GMT

Comment from Alun Jones (thx!) reminds me I forget to mention "functions" in batches! We were talking about script block, which are great if you combine them with call... Lets have a look what I have in my mind:

Echo Beginning...
...
...
Call :CheckForUserName %UserName%
...
...
Echo End...
Goto EndOfScript

:: Subroutines
:CheckForUserName
Net User %1 /Domain > nul
If %ErrorLevel% NEQ 0 (
   Echo User doesnt exist
   Goto EndOfScript
   ) Else (
   Echo User exist
   )

:EndOfScript

If you use it this way, you can see that it will execute CheckForUserName with parametere username and then it will return back (where Call was used), which can be extremely usefull sometimes.

However sometimes you need to end function (for example if error is encountered). Many people are asking me why I am always using EndOfScript when I can simply use Goto :EOF command - the reason is that if Goto :EOF is used inside this function, it WONT just to end of script - it will simply jump to end of function!

Script blocks in CMD (advanced scripting no.2)

martin — Thu, 30 Mar 2006 12:39:00 GMT

Well, I know I talked about this few times, but I decided to write more comprehensive article about this (this will follow previous about delayed expansion).

Many people think batches are stupid - I dont agree, I think you can do miracles with batches.

One of not really often used features are script block - I mean commands that are on more than one line. You can often see this:

If %Errorlevel% NEQ 0 Echo Error %ErrorLevel% occured!
If %Errorlevel% NEQ 0 FixError.cmd %ErrorLevel%
If %Errorlevel% NEQ 0 Goto EndOfScript

You can write this in more intelligent way:If %Errorlevel% NEQ 0 (
   Echo Error %ErrorLevel% occured!
   FixError.cmd %ErrorLevel%
   Goto EndOfScript
   )

And this is script block. You can use them either with If command or For command... And you can have many nested script blocks... For example this:

If %ErrorLevel% NEQ 0 (
   For /f "usebackq delims== tokens=1-2" %%i IN (`Set Error.`) Do (
      Echo Error %%j occured
      )
   Goto EndOfScript
)

Of course this is just example - you can make much MORE complex script blocks, for example some of mine are about 50 lines long (and when they are running, they create like 300 lines of code). When you get familiar with script block, you wont use common GoTo structure anymore - there is no need to.

But there is also one disadvantage - you must be advanced script write, because the script block is evaluated at beggining, so if you have some error, it is sometimes really hard to find out whats wrong.