Scheduled task would not run
The scheduled task for running ntbackup on our fileserver would not run anymore. It would stop with a return code of 0x80, and nothing worthwhile in the log file or event log. It ran fine for months, apart from the occasional hickup such as having no more free tapes in the loader, or a tape not being imported correctly.
I tried running the backup batch file manually, and that worked fine. I ran it under the dedicated backup account, and that worked fine too. But it wouldn't run automatically, and starting the task manually when logged in didn't work either. I couldn't find any useful debugging info so I turned to google. I found several people who has similar problems, but most had to do with a bug in the HID service which somehow interfered with ntbackup. I looked around some more and finally I found something that ended up giving me the final clue.
I opened up the task manager and noticed that there were 21 instances of xcopy still running, 23 instances of psexec, and 3 instances of ntbackup. These were probably left over from some tests I had done with the backup script of one of the new servers (a backup domain controller) which uses the 3 aforementioned programs.
After killing these dead processes I tried to run the backup task again, and everything worked fine this time. In my naivety, I had assumed that ending a task would end the processes it had spawned, but that seems to have been a mistake. Ah well, everything is working again, and now that I know the cause, I can prevent this from happening again.
This is why I try to understand a problem, rather than just rebooting the machine. A reboot would have fixed the problem in less time then I needed to figure this out. But then I wouldn't have known what happened (I hate that) and I wouldn't be able to prevent this from happening again. I would have been stuck in a regular but unknown problem -> reboot cycle.