Question about Unit Testing

Comments

# William said on December 10, 2005 8:46 AM:

Changing the design of a class for unit testing is indeed one of the goals of unit testing and TDD in general, but just changing private methods to public is not the solution.

Private methods usually indicate a class hidden inside the other class. You would want to look at that class with refactoring and see if a new class can be extracted. That new class can then become public and have its own unit tests.

# James Shaw said on December 11, 2005 6:50 AM:

Welcome to another religious debate :-)

In a nutshell, we've always added unit tests in a new folder inside the project so that it can access internal (not private) members.

http://www.coveryourasp.net/internal
http://www.coveryourasp.net/TestingPrivateMethods

http://www.coveryourasp.net/search.aspx?search=unit%20testing

# Bill said on December 11, 2005 9:08 AM:

Miki:

THanks for the response. I understand the goal here, but what i'm having trouble with is how do you test methods/properties that you are 10000% percent sure should be private if you can't see them outside of the class?

# William said on December 11, 2005 11:09 AM:

Miki - Thanks for your response. I understand where you're coming from but my question is with respect to how do you test methods that you are 100% sure you want to be private?

# Jake Good said on December 12, 2005 1:07 PM:

Unit Testing the face of the class should inheritely test the private methods of your class... if they don't get there.. then you might want to rethink the method in the first place. It's a "failing" test in itself.

Use a code coverage tool like NCover that can tell you how much your tests are covering in your classes. Very worthwhile.

Unit testing only takes you so far in writing good code. :)

# Jeremy Brayton said on December 12, 2005 3:15 PM:

I'd say test them both. The more you test, the better the product will be overall. Of course I don't know the HOW to test something private
vs. public. I thought TDD was to test every routine regardless of it's scope? Perhaps it is unit testing that can only handle public routines? I guess I haven't gotten my hands wet enough with NUnit to know why you can't.

I suppose it's that your test class calls methods from the Assembly and a class outside of the other class can only access it's public routines. Perhaps that's why there is a need for mock objects?

Then again it seems James has the answer. Use internal instead of private. I personally have a need to test every routine possible. Yes it helps refactor public methods but if I'm starting from the ground-up (which I plan on doing very soon) then I want to test it all to make sure I'm not letting common crap skate through. Plus testing everything makes sure all my logic is as refactored as possible, keeping the bloat out.

# Buckhunter said on December 12, 2005 5:28 PM:

Hey Bill,
just talking with some of the guys here at work this is the best we could come up with:

As you already know Private methods are a problem. You can exercise the private methods by using the public methods that call them. If no public methods call the privates, you have very few options. But they do exist.

Checkout the following URL:

http://www.artima.com/suiterunner/private.html

# Miki Watts said on December 12, 2005 8:10 PM:

If we're talking about a private method that is a helper method, then i'd extract it to a class and set the class to be internal.

If it's something that has logic in it, then i'd test it through a public function interface. The unit test code shouldn't know anything about the private parts of a class.

What I do instead create a class that handles that logic, create an interface from that class, and then pass the interface as a constructor parameter to be used later on. (aka Dependency Injection).

For example, I have an interface called DataSource, which has basic read and write functions, and an implementation of that interface against the filesystem.

Every class that needs to work with the file system takes that interface as a parameter. In production code I do "new FileDataSource()" and in the unit tests i pass a mock DataSource which then enables me to test the class with no side effects.

# Ted said on December 13, 2005 9:55 AM:

Test your public interfaces corrrectly and that should hit all paths through your private methods, provided your private methods to what they claim.

# Miki Watts said on December 13, 2005 7:58 PM:

I tried to post, but it got eaten...

For methods that are private, if the method is some helper method, then you can probably extract that to some helper class.

If it contains logic, then i'd test it through a public method. No one outside the class should know how the internals of the class work, and that's what unit testing enforces as a side bonus.

After all, if you could test internal methods, and then you refactored the guts of the class, then you just broke those tests. But why? the behaviour stayed the same, there is no reason to break the tests.

# Miki Watts said on December 13, 2005 7:59 PM:

I tried to post twice now but it got eaten, and doesn't show up...

# AndyV said on December 14, 2005 11:35 AM:

I'm in the camp with James. We're currently incorporating our unit tests in the same assembly and marking "private" methods we'd like to test as internal.

My question for James: We've had some internal debate as to whether the tests would reside in the same namespace within the assembly or in a subnamespace. For example, if my code were in "MyCoolStuff" then the tests would reside in "MyCoolStuff.Test". What do you think?

# Bill said on December 15, 2005 10:43 AM:

Miki - that makes total sense, I can't believe I missed it. I definitely appreciate your suggestions man!

# optionsScalper said on December 15, 2005 3:48 PM:

DoubleI,

I'm purposely late to this conversation. I wanted to see what others had written first as I have definite opinions on this.

You wrote:

". . . hopefully one of my brilliant readers can shed some light on it . . ."

NOT ME. I place these opinions out for ridicule like the rest of the brave souls on this post (as James Shaw states, a religious debate).

The first question to ask is "What are you testing?". I'll give the classic pitfall - testing the language instead of testing the application. There was a post sometime ago that echoes the basic ideas:

http://jayfields.blogspot.com/2005/06/what-are-you-testing.html

He's a Thoughtworks guy and I think he asks the fundamental question. I've been meaning to post on this topic as I have already posted on TDD and my problem with the "T" in TDD.

Think about the test.

Example -

If you are testing the domain of a value, do your tests portray that? Does your code portray that? Take a simple OLS regression. I want to know that the alpha and beta (or other attributes of the output of an OLS) are correct. How can I be certain of this? I could present a few different sets of data (as independent tests) and then assert against the known values (with a possible precision tolerance which is another candidate test). But does that test the OLS? What if the code doesn't maintain precision (truncation errors in numerical analysis) of an outlier? Is that acceptable? What domain of values are acceptable for the input range of the dependent and independent variable sets? What other factors could affect the success or failure of the OLS and in particular the calculation of alpha and beta? Is the domain continuous?

This is an example of a fallacy of proof by counterexample. If it doesn't fail "a few" examples", then it must not fail over the entire domain. If the "entire" domain is not known, then the requirements for the application are weak and unit test definition is likely not at fault. If the "entire" domain is known, then the unit tests can be constructed to test the domain. Proof by induction or equivalent is useful. Design By Contract (and the principles that lead to this practice; http://archive.eiffel.com/doc/manuals/technology/contract) is a proactive approach and this gets into topics like Hoare Logic where the existence of a unit test can be shown to be mathematically unnecessary, i.e. the code is a proof for its runtime behavior (whether in partial or total correctness). Haskell "guards" and Mathematica "constraints" (which I find useful) are an example of similar language constructs where the "contract" can be placed within the function or method.

So it follows that the next question to ask is "Is the domain of values known for the public properties/methods?". If the answer is yes, then you have strong requirements and can test for those conditions. If the answer is no, then the requirements are weak and unit testing is of little value. In this example, while I don't entirely agree with Miki, I think the principles are clear. If a private method in question is truly an atomic unit, is it's responsibility placed in the correct class, i.e. is a refactoring necessary? If a unit test is needed on each private method, ask "why?". If the domain of a method is known for a private method and can be determined of its impact on public methods, then the public methods reflect the results of that private method and its usage and linkage to the public method in question. If the private method is a primitive, then the refactoring as mentioned by Miki should be considered. That the private method operates correctly outside of the scope of the public method IS OF NO CONSEQUENCE. OO rules (not functional decomposition; see http://c2.com/cgi/wiki?FunctionalDecomposition) should dictate the way that these methods are allotted and their scope. If a private method is used by a number of public methods and the domain is not consistent for usage, this provides for evidence to refactor. It is important to remember that a class provides a definition for objects that 1) expose a service and 2) carry the responsibility of that service. The private methods that are involved in that service delivery are internal to the state of the class (and object) at runtime. Consideration for the Liskov Substitution Principle (if one believes in such things) should also be made if the refactoring in question is done with inheritance.

TOO MANY PEOPLE WRITE UNIT TESTS FOR THE SAKE OF TESTING WITHOUT CONSIDERATION OF WHAT THEY TEST. Most of the unit tests (> 75%) that I see in the field make this and other common mistakes. I'm annoyed that competent developers cannot distinguish these differences (and I'm not pointing a finger at anyone here; the comments here are all useful). The discovery of the unit tests that are necessary, in my experience, are the discovery and definition of requirements and are the means to improve the code proactively.

As far as deployment of unit tests and the assembly "of residence", I'm torn on this topic. By default, all unit tests that I write are in a separate assembly (with no private visibility into the assembly that they test). They have an extension of the namespace that they test and it is usually the suffix ".test", e.g. JJBR.stuff has a test namespace of JJBR.stuff.test. I have friends and colleagues whom I respect that provide for tests within the release assembly, so that should it be necessary, the tests are available in the production environment (and all other QA environments). I value that approach, but being a security-first kinda guy, having this type of code in a production assembly allows for another potential attack vector. H4ckb01z are always looking for an "in".

Regards,

---O

p.s. Of course, I'm being obscenely pedantic, but that is one way that you reduce errors.

# William said on December 19, 2005 1:06 PM:

O - you gave me a lot to think about. A lot. A lot. I'd like to put a Thank You in a for loop with x = 1000000 or something b/c well, I realize I have not put nearly enough thought into this and I need to. I'll probably be hitting you up for some advice shortly but many many thanks!

# Cory Foy said on December 19, 2005 7:15 PM:

Hey Bill,

My suggestion, as others have put, would be to move the methods to a public class (or internal if you are paranoid) and test it through that public interface.

Of course, if you feel you can adequetly exercise the methods using the existing public interface then I see no reason to extract it.

Remember, do the simplest thing possible, no more.