Smitfraud again, but this time its personal...
I had to battle another Smitfraud infection this weekend, but this time it was much closer to home and let me tell you that being so close to the situation emotionally is real bad for the stress levels.
What follows is the story of infection and removal as I can best reconstruct from my notes, logs and emails.
Mode of infection:
The network runs Citrix Metaframe XP on a Windows 2000 Server SP4 box. The verson of Office in use is MS Office XP SP3.
On the night of Thursday 8 June our network was spammed by many emails with subjectlines like "China domestic dogs and cats are raised to be killed for their fur", "Horrifying videos of the the cruel slaughter of dogs and cats in China" and "Moral? Love? Humanism? Just a beautifull sound of blood streaming from the warm pet's body".
One of the above emails was viewed using Outlook XP's Preview Pane and our network was immediately infected with no further interaction.
Symptoms:
Lots of Internet Explorer pop-ups in the terminal session of the infecting user pointing to www.razespyware.net (please do NOT go to that site).
Trend Antivirus for SMB V2.0 immediately reported the following infected files, showing a pop-up window in *all* terminal sessions (you can imagine what effect *that* had on the entire office):
s.exe (bkdr_small.bvr), winapi32.dll (troj_vb.alt), reger.exe (troj_adclick.aq) and mswinb32.dll (troj_generic).
Cleanup:
Unfortunately for Trend, there was far more to this infection than just those four files. Trend successfully removed s.exe on the first attempt, and reger.exe after 24 attempts. 24 failed attempts were made to remove winapi32.dll and one failed attempt was made to remove mswinb32.dll.
S!Ri's Smitfraudfix (run after Trend reported it had "quarantined" the above four files)detected:
C:\WINNT\system32\intxt.exe
C:\WINNT\system32\mswinb32.dll (also detected by Trend but not successfully removed)
C:\WINNT\system32\mswinb32.exe
C:\WINNT\system32\mswinf32.dll
C:\WINNT\system32\mswinf32.exe
C:\WINNT\system32\mswinup32.dll
C:\WINNT\system32\mswinxml.dll
C:\WINNT\system32\shell386.exe
C:\WINNT\system32\winapi32.dll (also detected by Trend but not successfully removed)
C:\WINNT\system32\winlfl32.dll
C:\WINNT\adw.htm
Files I found that were not detected by any programme were:
C:\WINNT\system32\svchw.exe
C:\WINNT\system32\tmp_n.dll (a hidden dll that I originally thought this was a legitimate file related to our document management system)
C:\WINNT\system3\page.htm
C:\WINNT\system3\dllsys.dll
C:\WINNT\system3\svcsys.dll
C:\WINNT\adw.htm
C:\WINNT\3no.exe
C:\WINNT\tmpfile027.exe
My grateful thanks to Dean of Calvert Technologies for all of his time, patience and input while I was trying to decide whether the above files were legitimate or malware - I'm betting the guy *so* wishes he wasn't on my IM list now ... all those pings ... "Hey Dean, can you check if dllsys.dll exists on your W2K box" ... and svcsys.dll ... and tmp_n.dll ... and lots of different *.exe files 
I also found some evidence of *old* malware infection, being:
c:\searchsquire.exe
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunServices\beta.exe
After checking, and doublechecking, and triple checking, and quadruple checking (killing the server or any of the programmes on it by deleting the wrong file would have been a *bad thing*) I left tmp_n.dll alone under the misapprehension that could be legitimate, and ran Killbox. I tell you now, my hands were shaking for the first time in all my years of malware fighting - if I killed the server or any installed programme by deleting the wrong file(s) it would have been a disaster.
I was able to delete all of the malware files except for svchw.exe using Killbox. svchw.exe was automatically recreated (by tmp_n.dll) and it took a couple of passes using Killbox before I was able to replace svchw.exe with a dummy copy.
After running Killbox to get rid of all of the malware files I rebooted the server and discovered we were in trouble. The server kept bluescreening within a minute or two, referencing the file win32k.sys. The only time it did not blue screen was when the server was booted into safe mode. This was very bad news because the other terminal server died on Monday (hard drive failure) and without this remaining server we were in big trouble.
Again, hooray for Dean. He knew what was the likely cause of the win32k.sys blue screen and knew exactly what to do. We had to disable nearly all of the automatically starting services to stabilise the system, and then reinstalled Service Pack 4 for Windows Server 2000 to repair the damage. Then, it was a matter of restarting one service at a time (again under Dean's guidance) until all were running and the system proved stable.
At this stage I thought the machine was clean, until 24 hours later when I saw the following error:
16 Bit MS-DOS subsystem
c:\WINNT\system32\svchw.exe
The NTVDM CPU encountered an illegal instruction.
CS:0000 IP:0077 OP:fo37050a02
Choose close to terminate the application.
Choosing "close" left NTVDM.EXE as a running process taking up 25% CPU usage... three more errors left CPU usage at 100% and extremely sluggish.
Ok, so something was using NTVDM.EXE to call svchw.exe and was crashing when encountering the dummy svchw.exe, but what? The only possible candidate was tmp_n.dll - the file I had originally assessed as possibly legitimate and too risky to remove.
tmp_n.dll was difficult to remove and was always in use - even in safe mode 
Not only that, the system instability it caused thanks to the ntvdm crashes made working on the server quite difficult.
By peeking into tmp_n.dll using Process Explorer (something I should have done in the first place, dammit) I discovered that its purpose was to download a file called 6.exe from URLs at jamming.cn and sonofyork.cn. An antispyware fighter who is way smarter than me dug deeper and reported that 6.exe was apparently renamed as svchw.exe (thanks Dave).
Every attempt to replace tmp_n.dll with a dummy using Killbox failed (although it did lose its hidden attribute).
There was an entry referencing tmp_n.dll in HKLM\SOFTWARE\Microsoft\Windows\NT\CurrentVersion\Windows\AppInit_DLLs. I deleted that entry, and and then got rid of the file the old fashioned way (DEL /F C:\WINNT\system32\tmp_n.dll)
Hopefully the server is now clean enough to keep it in service until it is decommissioned on Friday - there may still be an infection in wininet.dll lurking, but as long as it has no files to call, I think we'll be ok although I *will* be watching the server very closely - we survived, albeit with very frayed nerves and far too many new grey hairs.